Accuracy analysis of similarity measures in surprise framework

Abstract

Recommender Systems (RS) are growing technologies, which can be very useful for consumers in finding items of their interest on the web. Collaborative filtering(CF), a popular approach of Recommender Systems, recommends items to users based on other users with the same taste. The key step of the collaborative filtering method is to compute the similarity between users and items. The success of any recommender model hugely depends on how accurately the notion of similarity has been modelled. There are many open-source Python frameworks, which are useful in building and experimenting with RS models in the industry as well as academics such as Surprise, Python-recsys, Case Recommender, Polara, Spotlight, and RecQ. This work provides a brief introduction to the Surprise, a Python library for Recommender System, explaining its architecture, implementation and main features. Furthermore, a comparative study of the Surprise framework with other related frameworks is provided to demonstrate the fact why it is better than other frameworks in terms of implementing and handling new and complex recommender models. We have evaluated the accuracy of various built-in similarity measures provided by the Surprise framework using the real-world benchmark MovieLens datasets (100 k and 1 M). Thereafter, the accuracy of the recommendation is measured in terms of Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE). The empirical results demonstrate that Pearson-baseline outperforms other built-in similarity measures available in the Surprise framework.