Validation of minute-to-minute scoring for sleep and wake periods in a consumer wearable device compared to an actigraphy device

Abstract

Actigraphs are widely used portable wrist-worn devices that record tri-axial accelerometry data. These data can be used to approximate amount and timing of sleep and wake. Their clinical utility is limited, however, by their expense. Tri-axial accelerometer-based consumer wearable devices (so-called fitness monitors) have gained popularity and could represent cost-effective research alternatives to more expensive devices. Lack of independent validation of minute-to-minute accelerometer data for consumer devices has hindered their utility and acceptance. We studied a consumer-grade wearable device, Arc (Huami Inc., Mountain View CA), for which minute-to-minute accelerometer data (vector magnitude) could be obtained. Twelve healthy participants and 19 sleep clinic patients wore on their non-dominant wrist, both an Arc and a research-grade actigraph (Actiwatch Spectrum, Philips, Bend OR) continuously over a period of 48 h in free-living conditions. Time-stamped data from each participant were aligned and the Cole-Kripke algorithm was used to assign a state of “sleep” or “wake” for each minute-long epoch recorded by the Arc. The auto and low scoring settings on the Actiwatch software (Actiware) were used to determine sleep and wake from the Actiwatch data and were used as the comparators. Receiver operating characteristic curves were used to optimize the relationship between the devices. Minute-by-minute Arc and Actiwatch data were highly correlated (r = 0.94, Spearman correlation) over the 48-h study period. Treating the Actiwatch auto scoring as the gold standard for determination of sleep and wake, Arc has an overall accuracy of 99.0% ± 0.17% (SEM), a sensitivity of 99.4% ± 0.19%, and a specificity of 84.5% ± 1.9% for the determination of sleep. As compared to the Actiwatch low scoring, Arc has an overall accuracy of 95.2% ± 0.36%, a sensitivity of 95.7% ± 0.47%, and a specificity of 91.7% ± 0.60% for the determination of sleep. The Arc, a consumer wearable device in which minute-by-minute activity data could be collected and compared, yielded fundamentally similar sleep scoring metrics as compared to a commonly used clinical-grade actigraph (Actiwatch). We found high degrees of agreement in minute-to-minute data scoring for sleep and wake periods between the two devices.