Learning in the Wild: When, How, and What to Learn for On-Device Dataset Adaptation

Abstract

Although deep learning has been widely applied in practical applications, many embedded, mobile, and IoT systems performing deep learning tasks in the wild suffer from the problem of dataset shift, where their train and real data distributions are different, which has an adversarial effect on the quality of deep models. In this paper, we introduce a holistic online dataset adaptation strategy for on-device deep models running in the wild by tackling three challenges of dataset adaption: 1) when to start learning for adaptation, 2) how to learn for adaptation, and 3) what (examples) to learn for adaptation. To enable computing resource-and learning example-efficient online model update, we perform a model adaptation based on peroutput feature distributions, allowing efficient representation and high-level learning of new datasets. All the decisions on adaptation, i.e., when, how, and what to learn, as well as adaptation itself, are performed with the outputs of feed-forward execution without an explicit back-propagation of the model, enabling lightweight adaptation on resource-constrained devices. We implement a deep IoT device with a microcontroller and evaluate it with three datasets under various dataset shift scenarios. Our evaluation shows that the deep models effectively adapt to new data on the device, which achieves up to 28% higher and 2.5x efficient learning performance.