Evaluating Multi-Label Machine Learning Models for Smart Home Environments

Abstract

Context: Smart home devices have become increasingly popular in modern households, powered by the Internet of Things (IoT) advances. The data generated by smart devices can provide valuable insights into users' behavior and preferences. By analyzing the data, one can understand how people interact with their homes, thus creating a “smart home profile”. To comprehend the complete IoT ecosystem dynamics of an intelligent environment, it is necessary to learn from each IoT device to predict its status in the future time. Nevertheless, dealing with real-world IoT data structure requires considerable preprocessing tasks and the employment of classifiers that can learn multiple IoT inputs from a single IoT message. Objective: Aware of these challenges, this paper proposes a novel methodology to process multi-label IoT data and provide a comprehensive comparison of multi-label classifiers for forecasting the status of smart devices, considering their efficiency and accuracy. Method: We propose a data transformation method to preprocess the IoT data to be used by multi-label classifiers. This method is based on real data structure. Results: We evaluate our proposal in two real-world scenarios and various multi-label classifiers. The promising findings indicate that efficient classifiers can generate many correct predictions for a comprehensive IoT ecosystem in a small fraction of a second. Conclusions: Our proposed data transformation can fit the context of prediction to smart homes and work with multi-label classifiers to understand user behavior.