Urban tree generator: spatio-temporal and generative deep learning for urban tree localization and modeling

Abstract

We present a vision-based algorithm that uses spatio-temporal satellite imagery, pattern recognition, procedural modeling, and deep learning to perform tree localization in urban settings. Our method resolves two primary challenges. First, automated city-scale tree localization at high accuracy typically requires significant acquisition/user intervention. Second, vegetation-index segmentation methods from satellites require manual thresholding, which varies across geographic areas, and are not robust across cities with varying terrain, geometry, altitude, and canopy. In our work, we compensate for the lack of visual detail by using satellite snapshots across twelve months and segment cities into various vegetation clusters. Then, we use multiple GAN-based networks to plant trees by recognizing placement patterns inside segmented regions procedurally. We present comprehensive experiments over four cities (Chicago, Austin, Indianapolis, Lagos), achieving tree count accuracies of 87–97%. Finally, we show that the knowledge accumulated from each model (trained on a particular city) can be transferred to a different city.