VisKit: Web-based interactive IoT management with deep visual object detection

Abstract

Various technologies and standards for the Internet of Things (IoT) have defined the way devices should interact with each other to provide an intelligent IoT service to users in an efficient manner. Although the usability of system interface between the platform and users is one of the key factors for the success of IoT ecosystems, the manner in which IoT platforms should interact with users has not been well studied. Current IoT platforms provide a simple list-based interface to manage devices, which result in the degradation of their usability as the number of devices increases. In this paper, we propose an interactive web-based IoT management system where deep object detection techniques are adopted to facilitate user's device selection and management. The proposed system automatically recognizes the device type from video streams and accordingly generates smart controllers. The users can choose a device by touching an object in the video stream and use a smart controller to control the selected device. Finally, we show the feasibility of the proposed system through the implementation of a prototype which demonstrates a set of user scenarios.