Remote Driving Control With Real-Time Video Streaming Over Wireless Networks: Design and Evaluation

Abstract

There is a large gap between current AI-based autonomous-driving cars and fully autonomous cars, where the remote control of vehicles can be a unique solution to fill the gap. The remote control enables valuable operational data to be obtained, thus laying the groundwork for gradually increasing autonomous driving performance in the future. Moreover, human assistance through remote driving can offer more flexibility and intelligence than a single artificial intelligence. However, the real-time transmission of data and images is particularly crucial for remotely driven vehicles. The latency between a vehicle and the controller depends on the video streaming communication methods and transport protocols. Furthermore, the control or driving performance also depends on the vehicle speed likewise. Therefore, in this paper, we explore the impact of different communication methods of video streaming and vehicle 'speed' on the performance of remote driving. We design a vehicle remote driving system based on ROS (Robot Operating System) with ROS as the core communication architecture to realize remote control of vehicles. The video stream is transmitted using three different streaming methods: ROS multi-computer communication, TCP protocol, and UDP protocol. To be specific, we implement a simple remote driving system for a 'model' car, drive it at different speeds, and analyze how the drivers perform in terms of whether the vehicle gets off the track while driving. Based on the results, we find that 'UDP-based video streaming' achieves 720P video streaming with a latency of less than 50ms, which is helpful for further research on remote driving. The experiment by 'directly' observing the vehicle with eyes instead of using video streaming is also conducted to remove the video streaming latency. The experiment results show that the speed of the vehicle and the video streaming methods have a significant impact on the driving performance, and UDP protocol-based video streaming method is better suited for remote driving. The results imply that remote driving should be used in a low-speed environment rather than at high speed.