Aerial Fluvial Image Dataset for Deep Semantic Segmentation Neural Networks and Its Benchmarks

Abstract

Classification of aerial imagery is essential for water channel surveillance and waterfront land cover characterization. It is also beneficial to long-duration collaborative autonomous navigation of both unmanned aerial vehicles (UAVs) and autonomous surface vehicles (ASVs) to fulfill unmanned hydrologic data collection, environmental inspection, and disaster warning tasks. Deep semantic segmentation networks trained on aerial imagery have shown great results, however, they require finely labeled data. Existing aerial image datasets contain mostly urban scenes or fluvial images taken from ground level or collected from the Internet, there are no datasets that incorporate aerial and fluvial scenes with detailed annotation from different perspectives or include waterborne obstacles. To tackle this problem, aerial fluvial image dataset (AFID) is presented with multiple camera perspectives of fluvial scenes and is semantically labeled with emphasis on water and waterborne obstacles. Deep neural networks for binary (water and nonwater) semantic segmentation, with 12 different combinations of five encoders and three decoding architectures, are trained and tested in a curriculum learning scheme. Model performance is benchmarked on AFID, and the accuracy-efficiency tradeoff is discussed with the conclusion that the Unet architecture with a mix transformer encoder achieves the best segmentation performance with moderate computational consumption. The AFID dataset is publicly available to facilitate future work on developing new lightweight semantic segmentation models. Our immediate future plan will focus on the coordination of air and surface-water autonomous systems for navigable water detection and obstacle avoidance in high-risk challenging environments.