Multimodal Attention Networks for Human Activity Recognition From Earable Devices

Abstract

Earables (a.k.a ear-worn wearable devices) are gaining traction in the wearables ecosystem for monitoring user health. Human activity recognition (HAR) is a promising use case of earables due to their placement on the head and the combination of sensors. In this paper, we explore using multimodal attention-based neural networks for HAR from the ear. Attention networks have had a large impact on other disciplines' machine learning tasks and we believe they present opportunities in HAR from earable data. Different methods of utilising attention mechanisms in the literature are discussed as well as the benefits and challenges of using such networks in the context of HAR on real systems.