Robust and Affordable Deep Learning Models for Multimodal Sensor Fusion

Abstract

Deep fusion networks have received considerable attention lately due to the growing adoption of IoT devices, smartphones, and wearables that incorporate multiple sensing modalities, and their promising applications from human activity recognition to smart home automation. Despite recent advances in this area, there are several practical requirements that are often overlooked. Specifically, fusion networks must maintain their performance during momentary and long-term changes in the environment, be robust to sensor data quality issues, and have a reasonable size so that they can be deployed on resource-constrained devices. My PhD research aims to address these challenges by building robust multimodal fusion networks that rapidly generalize to new environments and have a smaller number of trainable weights, hence lower memory and carbon footprints.