Open-Data-Driven Unity Digital Twin Pipeline: Automatic Terrain and Building Generation with Unity-Native Evaluation

Abstract

The creation of simulation-ready digital twins for real-world simulations is hindered by two key challenges: the lack of widely consistent, application-ready open access terrain data and the inadequacy of conventional evaluation metrics to predict practical, in-engine performance. This paper addresses these challenges by presenting an end-to-end, open-data pipeline that generates simulation-ready terrain and procedural 3D objects for the Unity engine. A central finding of this work is that the architecturally advanced Swin2SR transformer exhibits severe statistical instability when applied to Digital Elevation Model (DEM) data. We analyze this instability and introduce a lightweight, computationally efficient stabilization technique adapted from climate science—quantile mapping (qmap)—as a diagnostic remedy which restores the model’s physical plausibility without retraining. To overcome the limitations of pixel-based metrics, we validate our pipeline using a three-axis evaluation framework that integrates data-level self-consistency with application-centric usability metrics measured directly within Unity. Experimental results demonstrate that qmap stabilization dramatically reduces Swin2SR’s large error (a 45% reduction in macro RMSE from 47.4 m to 26.1 m). The complete pipeline, using a robust SwinIR model, delivers excellent in-engine performance, achieving a median object grounding error of 0.30 m and real-time frame rates (≈100 FPS). This study provides a reproducible workflow and underscores a crucial insight for applying AI in scientific domains: domain-specific stabilization and application-centric evaluation are indispensable for the reliable deployment of large-scale vision models.