UAVStereo: A Multiple Resolution Dataset for Stereo Matching in UAV Scenarios

Abstract

Stereo matching is a fundamental task in 3-D scene reconstruction. Recently, deep learning-based methods have proven effective on some benchmark datasets, such as KITTI and SceneFlow. Unmanned aerial vehicles (UAVs) are commonly used for surface observation, and the images captured are frequently used for detailed 3-D reconstruction because of their high resolution and low-altitude acquisition. Currently, mainstream supervised learning networks require a significant amount of training data with ground-truth labels to learn model parameters. However, owing to the scarcity of UAV stereo-matching datasets, learning-based stereo matching methods in UAV scenarios are not fully investigated yet. To facilitate further research, this study proposes a pipeline for generating accurate and dense disparity maps using detailed meshes reconstructed based on UAV images and LiDAR point clouds. Through the proposed pipeline, we constructed a multiresolution UAV scenario dataset called UAVStereo, with over 34 000 stereo image pairs covering three typical scenes. To the best of our knowledge, UAVStereo is the first stereo matching dataset for UAV low-altitude scenarios. The dataset includes synthetic and real stereo pairs to enable generalization from the synthetic domain to the real domain. Furthermore, our UAVStereo dataset provides multiresolution and multiscene image pairs to accommodate various sensors and environments. In this article, we evaluate traditional and state-of-the-art deep learning methods, highlighting their limitations in addressing challenges in UAV scenarios and offering suggestions for future research.