Heart rate estimation from facial videos with motion interference using T-SNE-based signal separation

Abstract

Remote photoplethysmography (RPPG) can detect heart rate from facial videos in a non-contact way. However, head movement often affects its performance in the real world. In this paper, a novel anti-motion interference method named T-SNE-based signal separation (TSS) is proposed to solve this problem. TSS first decomposes the observed color traces into pulse-related vectors and noise vectors using the T-SNE algorithm. Then, it selects the vector with the most significant spectral peak as the pulse signal for heart rate measurement. The proposed method is tested on a self-collected dataset (17 males and 8 females) and two public datasets (UBFC-RPPG and VIPL-HR). Experimental results show that the proposed method outperforms state-of-the-art methods, especially on the videos containing head movements, improving the Pearson correlation coefficient by 5% compared with the best contrasting method. To summarize, this work significantly strengthens the motion robustness of RPPG, which makes a substantial contribution to the development of video-based heart rate detection.