Speech-based personality prediction using deep learning with acoustic and linguistic embeddings

Abstract

This study introduces a novel method for predicting the Big Five personality traits through the analysis of speech samples, advancing the field of computational personality assessment. We collected data from 2045 participants who completed a self-reported Big Five personality questionnaire and provided free-form speech samples by introducing themselves without constraints on content. Using pre-trained convolutional neural networks and transformer-based models, we extracted embeddings representing both acoustic features (e.g., tone, pitch, rhythm) and linguistic content from the speech samples. These embeddings were combined and input into gradient boosted tree models to predict personality traits. Our results indicate that personality traits can be effectively predicted from speech, with correlation coefficients between predicted scores and self-reported scores ranging from 0.26 (extraversion) to 0.39 (neuroticism), and from 0.39 to 0.60 for disattenuated correlations. Intraclass correlations show moderate to high consistency in our model’s predictions. This approach captures the subtle ways in which personality traits are expressed through both how people speak and what they say. Our findings underscore the potential of voice-based assessments as a complementary tool in psychological research, providing new insights into the connection between speech and personality.