Replication and reproduction in recommender systems research - evidence from a case-study with the rrecsys library

Abstract

Recommender systems (RS) are a real-world application domain for Artificial Intelligence standing at the core of massively used e-commerce and social-media platforms like Amazon, Netflix, Spotify and many more. The research field of recommendation systems now has already a more than 20 years long tradition and issues like replication of results and reproducibility of algorithms become more important. Therefore this work is oriented towards better understanding the underlying challenges of reproducibility of offline measurements of recommendation techniques. We therefore introduce rrecsys, an open-source package in R, that implements many popular RS algorithms, expansion capabilities and has an integrated offline evaluation mechanism following an accepted methodology. In addition, we present a case study on the usability of the library along with results of benchmarking the provided algorithms with other open-source implementations.