HPAN: Hierarchical Part-Aware Network for Fine-Grained Segmentation of Street View Imagery

Abstract

Street view imagery (SVI) has become a valuable geospatial data source for urban analysis, offering rich information about urban environments from a human-centric perspective. However, existing segmentation methods face significant challenges due to the inherent complexities of SVI, including scale variations, occlusions, and diverse semantic hierarchies. Drawing inspiration from the hierarchical nature of human visual cognition, this study proposes the hierarchical part-aware network (HPAN) to address these challenges in the fine-grained segmentation of SVI. The HPAN framework integrates four key components: (1) a hierarchical consistency learning module (HCLM), which ensures consistency across different levels of segmentation through novel loss functions; (2) a topology-aware graph matching module (TGMM), designed to model spatial relationships between object parts; (3) an edge-guided feature enhancement module (EFEM), which incorporates fine-grained edge information; and (4) a multilevel joint attention module (MLJAM), which adaptively fuses global scene semantics with local object details. Extensive experiments conducted on the cityscapes panoptic parts dataset demonstrate that HPAN outperforms existing methods across multiple panoptic quality metrics, particularly excelling in part-level segmentation tasks. Further evaluations on the mapillary vistas dataset and the cityscapes dataset validate HPAN's robust semantic segmentation performance across diverse street scenes. Generalization tests on different SVI sources, including challenging scenarios, such as low-light conditions and occluded environments, highlight the model's strong adaptability and effectiveness.