Analyzing errors and referral pairs to characterize common problems and improve web reliability

Abstract

In this paper, we analyze web server error logs and the corresponding referral pairs from web access logs to identify and characterize common web errors. We identify missing files as the primary type of web errors and classify them according to their incoming referral links into internal, external, and user errors. We also identify major missing file types within each error category. Based on these analysis results, we recommend different quality assurance initiatives to deal with different types of web problems for the effective improvement to web reliability. © Springer-Verlag Berlin Heidelberg 2003.