Remote Heart Rate Estimation Using Attention-targeted Self-Supervised Learning Methods

Abstract

Heart rate measurement is a crucial factor for assessing the overall health status of an individual. Abnormal heart rates, whether lower or higher than baseline, can indicate potential pathological or physiological abnormalities. As a result, it is necessary to have reliable technology for monitoring heart rates in various fields, including medicine, biotechnology, and healthcare. With recent advancements in deep learning research, it is now possible to monitor heart rate conveniently and hygienically without specialized equipment, using facial video photo volume measurement. This new technology employs a deep learning-based video analysis method that requires a large data set to achieve high performance. However, collecting and labeling a vast amount of data is often impractical and costly. Therefore, researchers have been searching for alternative ways to achieve high performance with smaller datasets. This paper proposes a novel self-supervised learning approach suitable to the face video process. Our proposed method can effectively acquire a deep latent expression from a face image sequence and apply it to a target task through transfer learning. Using this method, we aim to improve the remote heart rate estimation performance in a limited-size dataset. Our proposed method is specialized for facial image sequences and focuses on the color change of the face to achieve high performance in existing attention-based deep learning models. The proposed self-supervised learning method has several advantages. First, it can learn useful features from unlabeled data, reducing the reliance on annotated datasets. Second, it can help overcome the problem of insufficient labeled data in specific domains, such as medical image analysis. Third, the proposed method can improve the performance of the target task using pre-trained models on different datasets. Finally, our approach improves the remote heart rate estimation performance by extracting useful features from facial images.