Efficiency-Effectiveness Trade-offs in Recommendation Systems

Abstract

Throughout the years, numerous recommendation algorithms have been developed to address the information filtering problem by leveraging users' tastes through implicit or explicit feedback. In this paper, we present the work undertaken as part of a PhD thesis focused on exploring new evaluation dimensions centred around the efficiency-effectiveness trade-offs present in state-of-the-art recommendation systems. Firstly, we highlight the lack of efficiency-oriented studies and we formulate the research problem. Then, we propose a mapping of the design space and a classification of the recommendation algorithms/models with respect to salient attributes and characteristics. At the same time, we explain why and how assessing the recommendations on an accuracy versus training cost curve would advance the current knowledge in the area of evaluation, as well as open new research avenues for exploring parameter configurations within well-known algorithms. Finally, we make the case for a comprehensive methodology that incorporates predictive efficiency-effectiveness models, which illustrate the performance and behaviour of the recommendation systems under different recommendation tasks, while satisfying user-defined quality of service constraints and goals.