Extracting Heart Rate Variability from a Smartphone Camera

Abstract

217 infarction [6], diabetes [7], and chronic heart failure [8]. Particularly it was found that patients with a restrictive left ventricular filling pattern had a lower HRV index compared to patients with a non-restrictive pattern [7]. The HRV analysis is also a relevant indicator of athletic conditions [9]. Besides presenting heart beat detection algorithm, the goal of this paper is to compare HRV extracted from a video recorded by smartphone camera and HRV extracted from an electrocardiogram (ECG). Due to the fact that accuracy of estimated HRV depends on the sampling rate we investigate whether the limited camera's frame rate produces HRV comparable to HRV extracted from ECG. We also calculate and compare statistics of both groups of HRV signals. These statistical characteristics vary for people with healthy heart and heart with congestive heart failure or other heart conditions. Therefore, it is important to verify that these characteristics remain unaltered regardless of the sensor used in recording of a cardio-signal. II. HEART RATE VARIABILITY EXTRACTION A. Data Collection The experimental setup consists of a Samsung Galaxy S2 (Samsung, Seoul, Korea) and Mitsar EEG 201 recorder that was employed as an ECG recorder. To record video a palmar side of right index finger was placed over the camera lens. The flash was turned on while the video is being recorded. To simulate recording in a real environment, subjects were told to apply pressure that feels comfortable to them. For the purpose of reducing computational load we selected the lowest available resolution of 176 × 144 and to increase the precision of extracted HRV we set the sampling rate to the maximum available frame rate of 30 fps. Simultaneously with video recording, we recorded the ECG with Mitsar EEG 201 that was a ground truth signal in our experiment. Two electrodes were placed on the patient's body to record ECG. One of the electrodes was attached to patient's ear and the other placed on the skin surface around the heart. Five male subjects participated in our experiment ranging from 21 to 32 years old. All the signal analysis was conducted in MATLAB (MathWorks, Natick, MA, USA) as described below. B. Beat Detection We developed a heartbeat detection algorithm to analyze and compare HRV extracted from the video signal. The ECG signal is relatively clean and by simply finding maxima that are located above a predefined threshold we were able to detect R-beats. On the other hand it is not trivial to detect R-beats form video frames due to the fact that the color intensity is affected by the pressure applied by the finger. Moreover, the color response is different for each color channel. Scully et al. [2] suggested calculating average for a window of 50 × 50 pixels in green channel. We replicated the experiment and found that in case when the subject apply significant pressure to the finger against the camera lens, values of the green channel become close to zero and therefore not applicable for heart beat detection in Fig. 2(b). On the contrary calculating the average for a blue channel produces a noise like signal when the finger is resting on the lens without any pressure. The heartbeat disappears when the subject presses the finger against the lens. An average of the red channel produces a relatively clean signal regardless of the pressure. Although the signal often fluctuates close to the maximum and sometimes gets threshold. Through trial and error we found that composite signal obtained as a sum of normalized averages of each color channel produces a signal applicable for heart beat detection (Fig. 2). We faced a number of difficulties associated with finding heartbeats in the composite signal (Fig. 2). As it can be seen the beginning and the end of the signal contains high amplitude spikes that should be filtered out. These spikes are caused by the transition of intensity adjustment from the time when camera is turned on and the finger is yet not placed over the camera and the time when finger is over the camera. To remove the spikes we applied Random Sample Consensus (RANSAC) algorithm [10]. RANSAC finds all the data points that conform a selected model. The data points that do not fit the model are considered as outliers and therefore removed. The second problem comes from significant amplitude variations (Fig. 2). To overcome this problem we proposed the following beat detection algorithm: