Challenges and advancements in image-based 3D reconstruction of large-scale urban environments: a review of deep learning and classical methods

Abstract

Over the past decade, the field of image-based 3D scene reconstruction and generation has experienced a significant transformation, driven by the integration of deep learning technologies. This shift underscores a maturing discipline characterized by rapid advancements and the introduction of numerous innovative methodologies aimed at broadening research boundaries. The specific focus of this study is on image-based 3D reconstruction techniques applicable to large-scale urban environments. This focus is motivated by the growing need for advanced urban planning and infrastructure development for smart city applications and digitalization, which requires precise and scalable modeling solutions. We employ a comprehensive classification framework that distinguishes between traditional and deep learning approaches for reconstructing urban facades, districts, and entire cityscapes. Our review methodically compares these techniques, evaluates their methodologies, highlights their distinct characteristics and performance, and identifies their limitations. Additionally, we discuss commonly utilized 3D datasets for large environments and the prevailing performance metrics in this domain. The paper concludes by outlining the current challenges faced by the field and proposes directions for future research in this swiftly evolving area.