A Sensor-Independent Multimodal Fusion Scheme for Human Activity Recognition

Abstract

Human Activity Recognition is a field that provides the fundamentals for Ambient Intelligence and Assisted Living Applications. Multimodal methods for Human Activity Recognition utilize different sensors and fuse them together to provide higher-accuracy results. These methods require data for all sensors employed to operate with. In this work we present a sensor-independent, in regards to the number of sensors used, scheme for designing multimodal methods that operate when sensor-data are missing. Furthermore, we present a data augmentation method that increases the fusion model’s accuracy (up to 11% increases) when operating with missing sensor-data. The proposed method’s effectiveness is evaluated on the ExtraSensory dataset, which contains over 300,000 samples from 60 users, collected from smartphones and smartwatches. In addition, the methods are evaluated for different number of sensors used at the same time. However, the max number of sensors must be known beforehand.