Identifying streetscape features using vhr imagery and deep learning applications

Abstract

Deep Learning (DL) based identification and detection of elements in urban spaces through Earth Observation (EO) datasets have been widely researched and discussed. Such studies have developed state-of-the-art methods to map urban features like building footprint or roads in detail. This study delves deeper into combining multiple such studies to identify fine-grained urban features which define streetscapes. Specifically, the research focuses on employing object detection and semantic segmentation models and other computer vision methods to identify ten streetscape features such as movement corridors, roadways, sidewalks, bike paths, on-street parking, vehicles, trees, vegetation, road markings, and buildings. The training data for identifying and classifying all the elements except road markings are collected from open sources and finetuned to fit the study’s context. The training dataset is manually created and employed to delineate road markings. Apart from the model-specific evaluation on the test-set of the data, the study creates its own test dataset from the study area to analyze these models’ performance. The outputs from these models are further integrated to develop a geospatial dataset, which is additionally utilized to generate 3D views and street cross-sections for the city. The trained models and data sources are discussed in the research and are made available for urban researchers to exploit.