Design and implementation of an effective web server log preprocessing system

Abstract

WWW constitutes huge repository, distributed and dynamically growing hyper medium, supporting access to information and services. As more organizations rely on WWW to conduct business, user behavior analysis becoming difficult in web-based applications. Information about user's interactions with website is stored in server logs and serves as huge electronic survey of website. Web usage mining deals with discovering usage patterns fromserver logs in order to understand and better serve the needs of web users. The raw information contained in log file represents noisy data. Preprocessing includes cleaning, user identification, sessionization, path completion & structurization and is a prerequisite for improving accuracy and efficiency of the subsequent mining process. This paper emphasizes on an effective web log preprocessing system. Experimental results proved that the proposed system reduces the size of log file down to 12% and improves the performance of preprocessing in identifying users, sessions, path completion and structurization. © 2012 Springer-Verlag GmbH Berlin Heidelberg.