StreetScouting: A Deep Learning Platform for Automatic Detection and Geotagging of Urban Features from Street-Level Images

Abstract

Urban environments are evolving rapidly in big cities; keeping track of these changes is becoming harder. Information regarding urban features, such as the number of trees, lights, or shops in a particular region, can be crucial for tasks, such as urban planning, commercial campaigns, or inferring various social indicators. StreetScouting is a platform that aims to automate the process of detecting, visualizing, and exporting the urban features of a particular region. Recently, the advent of deep learning has revolutionized the way many computer vision tasks are tackled. In this work, we present StreetScouting, an extensible platform for the automatic detection of particular urban features of interest. StreetScouting utilizes several state-of-the-art computer vision approaches including Cascade R-CNN and RetinaFace architectures for object detection, the ByteTrack method for object tracking, DNET architecture for depth estimation, and DeepLabv3+ architecture for semantic segmentation. As a result, the platform is able to detect and geotag urban features from visual data. The extracted information can be utilized by many commercial or public organizations, eliminating the need for manual inspection.