E2AR: An Energy-Efficient Augmented Reality Framework for Collaborative Multi-Drone Systems

Abstract

The safety, energy efficiency, and small size of smart drones have led to the broad use of autonomous Unmanned Aerial Vehicles (UAVs) across various applications, creating opportunities for human-machine collaboration. Machine Learning (ML) algorithms like Neural Networks (NNs) offer promising solutions for vision-based navigation and autonomous systems. However, these algorithms are computationally intensive, making it challenging to deploy them on robots expanded by resource-constrained edge devices with limited computational power and low energy consumption requirements. In this paper, we propose an Energy-Efficient Framework for Video Streaming and Augmented Reality called E2AR to enable ML-based multi-edge device video streaming to a AR device while applying augmented reality to enhance human-machine teaming. For this aim, a YOLO is deployed on the edge device for energy-efficient computation and higher performance. Moreover, video streaming to HoloLens is optimized to improve communication latency and power consumption. To evaluate the proposed method, we implemented it on edge devices such as Crazyflie drone while streaming video to the HoloLens. Crazyflie drones with LiDAR sensors and the GAP8 processor has consisted of an octa-core RISC-V. We measured the power consumption, latency, and core usage of the GAP8 processor while implementing the proposed approach. View a video demonstration of the E2AR concept at: Video.