Towards vertical urban geometry extraction: occlusion-reduced estimation from street view images using diffusion models

Abstract

Understanding urban structure is crucial for analysing city dynamics, urban planning, energy efficiency, and environmental sustainability. Extracting urban vertical geometry, such as building heights, is essential to these efforts. However, as cities are constantly evolving, there is a growing need for cost-effective and efficient methods of updating data. Street view imagery, easily captured by road vehicles or voluntary sources, offers frequent updates and rich details of urban features, making it a valuable resource for urban vertical geometry acquisition. However, existing methods often struggle to accurately estimate building heights when street elements obstruct building façades, particularly in dense and complex urban environments. To address this, we propose a framework for building height estimation that adopts diffusion models to reduce occlusions, remove obstructing objects, and recover hidden building features through image inpainting. The framework also integrates single-view metrology and building footprint data to enhance accuracy by compensating for distance variations. Evaluated on a dataset of over 1,000 buildings and 3,814 images, our method shows a 9.96% increase in the number of height estimates within a 2-meter error margin, demonstrating its effectiveness. This approach offers new opportunities for urban vertical geometry extraction or updating, supporting urban studies and facilitating smart city development.