From Facial Expressions to User Experience (UX): How User Emotions Define the Design of Intelligent Systems

Abstract

Emotions are a key determinant of User Experience (UX). This work investigates the relationship between facial emotions and UX. It presents ALPACA, a predictive system that estimates user satisfaction from valence–arousal (VA) signals. Dataset DT22, a database of 112 participants, is used to evaluate models on the external test set. An identity-free processing pipeline that uses only VA features achieves ~77% accuracy/weighted F1-score. When contextual information is also available—Movie Clip ID and perceived User Quality—the context-aware pipeline increases performance to ~85–92% in ablation analyses. This gain is obtained by ensembles with a majority-voting decision rule. This indicates that contextual features provide complementary information to VA. To assess system robustness, Leave-One-Participant-Out (LOPO) and Leave-One-Movie-Out (LOMO) schemes are also investigated, as well as emotion-anchored evaluations (first- and last-emotion). The approach is modular and adheres to privacy-by-design principles by operating on identity-free descriptors rather than storing facial frames. Overall, the findings suggest that emotion signals can support adaptive UX. They can also enable dynamic content personalization in applications such as video streaming, customer-experience management, and e-learning. This research presents a practical study on integrating emotion-aware intelligence while respecting user privacy.