Haptic and mixed reality enabled immersive cockpits for tele-operated driving

Abstract

In the last few years, the use of automated guided vehicles (AGVs) and autonomous mobile robots (AMRs) has experienced a sustainable increase in different verticals such as factories and logistics. However, they still have some technical limitations that hamper their autonomous operation in unpredictable or dynamic environments, requiring them to be supervised and/or controlled by human operators. In such situations, current tele-operated driving (ToD) systems lack the required stimulation and spatial perception to precisely manipulate the AGVs/AMRs, besides suffering from real-time challenges that limit the accuracy of movement. This chapter describes a proposal to solve these problems, by combining low-latency 5G-IoT networks and immersive cockpits equipped with haptic and mixed-reality devices. It also explains how such devices provide intuitive feedback for ToD and facilitate context-aware decision-making. The results are validated in the context of two innovative demonstrations deployed in the environment of a seaport, where ToD of multiple AGVs/AMRs is supported by a 5G mmWave network infrastructure.