Collaborative filtering and recommendation systems

Abstract

The purpose of this document is to introduce the wide area of intelligent filtering, mainly concentrating on collaborative filtering (CF), user profiling and recommendation systems. The background of collaborative filtering and its advantages and limitations, as well as examples and suggestions on how to improve collaborative filtering ideas for analyzing product data in the LOUHI- project have been presented in this report. The development of collaborative filtering or its super-concept intelligent filtering is still ongoing. “Word-of-mouth” type of opinion and information sharing “systems” have been in use for ages, but from the first half of 1990s, the pervasion of the internet enabled new ways to carry out the idea of sharing opinion with a wide number of people via the net. The first known applications were Grouplens in 1992 and Firefly in 1994. Later Yahoo and Barnesandnoble signed up to use Firefly's technology. Finally book dealer Amazon.com introduced the idea of collaborative filtering (they had the BookMatcher system later spread to cover also other items) to a wider number of people in 1998. This is a basic knowhow report for the case studies within Phase 2 of the Louhi-project. Further development of the ideas will be implemented within the case studies in 2002- 2003. The major findings of this report are that collaborative filtering is a good possibility when building personalized recommendation systems. There are still many problems that have not been solved, e.g. scalability and reliability questions. Also the bi-directional development towards both user-based and item-based filtering have created new possibilities that have not yet been totally implemented. These are also the challenges for LOUHI cases. Collaborative filtering ideas also touch the ideas of semantic web. By semantics one can improve the abstraction level of recommendation systems. Besides comparing individual profiles one can also find other connections between the users and the items.1