Learning and predicting user pairwise preferences from emotions and gaze behavior

Abstract

Most people believe that their choices are based only on a rational analysis of available alternatives. Actually, emotions affect our choices and are activated as feedback during the decision process. In Recommender Systems research, preference learning is an important element to produce good recommendations. Preferences can be acquired by explicitly asking the user or by analyzing his implicit feedback that can be inferred from his behavior. Following the latter vein of research, we developed a novel approach to automatically extract these preferences by analyzing the gaze behavior and the facial expressions of the users while looking at item's images form making a choice between them. Specifiically, we focused on pair-wise image comparisons in a preference elicitation experiment and exploited a Process Mining approach to learn preferences. In particular, we investigated whether behavior patterns can be learned and used to predict the user choice. We evaluated strategies based on gaze behavior and experienced emotions and results obtained so far show promising prediction performance.