DeepPilot4Pose: a fast pose localisation for MAV indoor flight using the OAK-D camera

Abstract

We present DeepPilot4Pose, a compact convolutional neural network for visual pose estimation that runs onboard novel smart camera, the OAK-D. We aim at using it for micro aerial vehicle (MAV) localisation, which flies in an indoor environment, where neither GPS nor external sensors are available. This calls for onboard processing, which demands a combination of software and hardware that could run efficiently onboard the MAV. To this end, we exploit the use of this novel sensor that can be carried by the MAV, the OAK-D camera, capable of performing neural inference on its chip in addition to providing colour, monochromatic and depth images. We show that our DeepPilot4Pose can run efficiently on the OAK-D at $$65\,{\text {Hz}}$$ with a localisation performance comparable to that obtained with RGB-D ORB-SLAM using the OAK-D and running onboard the MAV on the Intel Compute Stick at $$12 \,{\text {Hz}}$$ . We have evaluated our approach with benchmark datasets and in real MAV flights in an indoor facility with a challenging visual appearance.