RobWoTAR: Web of Things Approach to Universally Control Robots in Augmented Reality

Abstract

The rise of the Internet of Things (IoT) presents significant opportunities to enhance automation and control of various devices. However, it has also increased complexity in onboarding, controlling, and interacting with devices, particularly with complex devices such as robot arms. The W3C Web of Things (WoT) standards aim to reduce this complexity by describing interactions with IoT devices in a unified and abstracted way. We present the first Augmented Reality (AR) system, RobWoTAR, that leverages WoT Thing Descriptions to enable universal, interoperable robot arm control without device-specific implementations. Our novel approach automatically generates a visual AR Digital Twin (DT) based solely on the robot arm's standardized Thing Description (TD) and Unified Robot Description Format (URDF), eliminating the need for manual setup or proprietary interfaces. By bridging WoT standards with AR visualization, users can program any WoT-compliant robot arm through natural gestures and test motions within the actual environment before deploying them to the physical device. We evaluate our approach through a case study demonstrating seamless onboarding of a new robot arm using only the TD and URDF, and validate intuitive control through a user study.