Towards a comprehensive evaluation of recommenders: A cognition-based approach

Abstract

Evaluating Recommender Systems (RSs) is a challenging issue that is significantly magnified by the multifaceted properties of RSs, which makes it insufficient to use only one metric to evaluate recommenders. This challenge necessitates the need for a unified evaluation model that comprehensively assesses multiple aspects of the recommender. This position paper proposes a cognition-based comprehensive evaluation to evaluate the main activities of RSs. We innovated the proposed model based on the cognitive dimension of Bloom’s taxonomy, a widely used model for classifying learning objectives in the teaching area. We created a phase-wise mapping between RSs and Bloom’s taxonomy to come up with an overall evaluation for recommenders. Based on these connections, we believe that the proposed evaluation model would have the potential to support the decision of selecting the most appropriate recommender systems by giving a benchmarked score for different aspects of RSs.