Recent advances in drone-view object detection

Abstract

Given the support of artificial intelligence technology，drones have initially acquired intelligent sensing capabilities and have demonstrated efficient and flexible data collection in practical applications. Drone-view object detection，which aims to locate specific objects in aerial images，plays an irreplaceable role in many fields and has important research significance. For example，drones with highly mobile and flexible deployment have remarkable advantages in accident handling，order management，traffic guidance，and flow detection，making them irreplaceable in traffic monitoring. As for disaster emergency rescue，drones with aerial vision and high mobility can achieve efficient search and safe rescue in large areas，locate people quickly and accurately in distress，and help rescuers control the situation，thereby ensuring the safety of people in distress. This study provides a comprehensive summary of the challenges in object detection based on the unmanned aerial vehicle（UAV）perspective to portray further the development of drone-view object detection. The existing algorithms and related datasets are also introduced. First，this study briefly introduces the concept of object detection in drone view and summarizes the five imbalance challenges in object detection in drone view，such as scale imbalance，spatial imbalance，class imbalance，semantic imbalance，and objective imbalance. This study analyzes and summarizes the challenges of drone-view object detection based on the aforementioned imbalances by using quantitative data analysis and visual qualitative analysis. 1）Object scale imbalance is the most focused challenge in current research. It comes from the unique aerial view of drones. The changes in the drone’s height and angle bring drastic changes to the object scale in the acquired images. The distance of the lens from the photographed object under the drone view is often far. This scenario results in numerous small objects in the image and makes capturing useful features for object detection difficult for the existing detectors. 2）Different regions of drone-view images have great differences，and most objects are concentrated in the minor area of images，i. e. ，the spatial distribution of objects is enormously uneven. On the one hand，the clustering of dense objects in small areas generates occlusion. The detection model needs to devote considerable attention to this occlusion to distinguish different objects effectively. On the other hand，treating equally different areas wastes many computational resources in vanilla areas，limiting the improvement of object detection performance. 3）The problem of class imbalance in the drone view is divided into two categories. One is the positive-negative sample imbalance problem caused by the gap between the front and rear views shared in the image. The other is the imbalanced numbers of different categories caused by the number of samples in the real world. 4）The semantic pieces of information defined by different category labels in the drone-view object detection dataset are often similar，resulting in only subtle differences between different categories. However，significantly different representations of objects exist in the same category，which together form the semantic imbalance problem. 5）Drone-view object detection often faces the problem of unbalanced optimization targets，i. e. ，the contradiction between the high computational demand for high-resolution images and the limited computing power of low-power chips is difficult to balance. These unbalanced problems bring enormous challenges to object detection from the UAV viewpoint. However，even the most advanced object detection algorithms currently available can hardly achieve an average accuracy rate of 40% on aerial images，which is far below the performance of general object detection tasks. Therefore，many scholars have conducted many studies. These research methods can be summarized as optimization ideas to solve these imbalance problems. In this study，we collect relevant research works，which are sorted and analyzed according to the countries of authors，institutions，published journals or conferences，years，the category of methods，and the solved problem. The present study presents the challenging problems solved by previous research and the development trends of existing methods. This study also focuses on the methods of improving drone-view object detection performance in terms of data augmentation，multiscale feature fusion，region searching strategies，multitask learning，and lightweight model. The advantages and disadvantages of these methods for different problems are systematically summarized and analyzed. Besides introducing existing methods，the present study compiles and introduces the applications of drone-view object detection in practical scenarios，such as traffic monitoring，power inspection，crop analysis，and disaster rescue. These applications further emphasize the significance of object detection in drone view. Then，this study collects and organizes UAV datasets suitable for object detection tasks. These datasets are present from various perspectives，such as year，published journals or conferences，annotation information，and number of citations. In particular，the present study provides the performance evaluation of the existing algorithms on two commonly used public datasets. The presentation of these performance data is expected to help researchers understand the current state of development of drone-view object detection and promote further development in this field. Finally，this study provides an outlook on the future direction of drone-view object detection by considering the aforementioned imbalance problems. The promising research includes the following：1）data augmentation：providing the network with enough high-quality learning samples by considering the specific characteristics of drone-view images based on the conventional data augmentation strategy is a good idea；2）multiscale representation：how to avoid the interference of background noise in feature fusion and effectively extract information at different scales using an efficient fusion strategy is an urgent problem to be solved；3）visual inference：using information unique to the viewpoint of drones，mining contextual information from images to facilitate image recognition，and using easy-to-detect objects to improve the performance of difficult-to-detect objects are directions worthy of deep consideration.