Personality Trait Prediction From Facial Sketch Leveraged by Expression Muscles

Abstract

This work explores whether facial sketches can be used to predict personality traits, representing, to our knowledge, the first systematic investigation of this approach in the literature. Unlike traditional RGB facial images that capture detailed features, sketch-based images emphasize the structure and movement of facial expression muscles, thereby providing a novel perspective for personality prediction. Our approach introduces three key innovations: expression muscle-guided feature weighting to improve prediction accuracy by prioritizing biologically relevant patterns; data augmentation through intermediate sketches generated via the 25-Step Sketching Approach to mitigate data scarcity; and comprehensive validation on a dataset of 12,320 individuals. Experimental results demonstrate that our sketch-based model achieves comparable accuracy to image-based models for specific personality traits, while ablation studies confirm the complementary benefits of both expression muscle weighting and sketch augmentation strategies. These findings, coupled with the newly constructed sketch datasets, offer valuable multimodal resources and methodological insights for researchers in affective computing and behavioral science.