A Personalized Spatial-Temporal Cold Pain Intensity Estimation Model Based on Facial Expression

Abstract

Objective: Pain assessment is of great importance in both clinical research and patient care. Facial expression analysis is becoming a key part of pain detection because it is convenient, automatic, and real-time. The aim of this study is to present a cold pain intensity estimation experiment, investigate the importance of the spatial-temporal information on facial expression based cold pain, and study the performance of the personalized model as well as the generalized model. Methods: A cold pain experiment was carried out and facial expressions from 29 subjects were extracted. Three different architectures (Inception V3, VGG-LSTM, and Convolutional LSTM) were used to estimate three intensities of cold pain: No pain, Moderate pain, and Severe Pain. Architectures with Sequential information were compared with single-frame architecture, showing the importance of spatial-temporal information on pain estimation. The performances of the personalized model and the generalized model were also compared. Results: A mean F1 score of 79.48% was achieved using Convolutional LSTM based on the personalized model. Conclusion: This study demonstrates the potential for the estimation of cold pain intensity from facial expression analysis and shows that the personalized spatial-temporal framework has better performance in cold pain intensity estimation. Significance: This cold pain intensity estimator could allow convenient, automatic, and real-time use to provide continuous objective pain intensity estimations of subjects and patients.