CIM-WV: A 2D semantic segmentation dataset of rich window view contents in high-rise, high-density Hong Kong based on photorealistic city information models

Abstract

Large-scale assessment of window views is demanded for precise housing valuation and quantified evidence for improving the built environment, especially in high-rise, high-density cities. However, the absence of a semantic segmentation dataset of window views forbids an accurate pixel-level assessment. This paper presents a City Information Model (CIM)-generated Window View (CIM-WV) dataset comprising 2,000 annotated images collected in the high-rise, high-density urban areas of Hong Kong. The CIM-WV includes seven semantic labels, i.e., building, sky, vegetation, road, waterbody, vehicle, and terrain. Experimental results of training a well-known deep learning (DL) model, DeepLab V3+ , on CIM-WV, achieved a high performance (per-class Intersection over Union (IoU) ≥ 86.23%) on segmenting major landscape elements, i.e., building, sky, vegetation, and waterbody, and consistently outperformed the transfer learning on a popular real-world street view dataset, Cityscapes. The DeepLab V3+ model trained on CIM-WV was robust (mIoU ≥ 72.09%) in Hong Kong Island and Kowloon Peninsula, and enhanced the semantic segmentation accuracy of real-world and Google Earth CIM-generated window view images. The contribution of this paper is three-fold. CIM-WV is the first public CIM-generated photorealistic window view dataset with rich semantics. Secondly, comparative analysis shows a more accurate window view assessment using DL from CIM-WV than deep transfer learning from ground-level views. Last, for urban researchers and practitioners, our publicly accessible DL models trained on CIM-WV enable novel multi-source window view-based urban applications including precise real estate valuation, improvement of built environment, and window view-related urban analytics.