Evaluating information filtering techniques in an adaptive recommender system

Abstract

With the huge increase in the volume of information available in digital form and the increasing diversity of Web applications, the need for efficient, reliable information filtering is critical. New algorithms that filter information for specific tastes are being developed to tackle the problem of information overload. This paper proposes that there is a substantial relative difference in the performances of various filtering algorithms as they are applied to different datasets, and that these performance differences can be leveraged to form the basis of an Adaptive Information Filtering System. We classify five different datasets based on a number of metrics, including sparsity, ratings distribution and user-item ratio, and develop a regression function over these metrics to predict the suitability of a particular recommendation algorithm for a previously unseen dataset. Our results show that the predicted best algorithm does perform best on the new dataset. © Springer-Verlag 2004.