Dynamic Urban Scene Modeling with 3D Gaussian Splatting from UAV Full Motion Videos

Abstract

Reconstructing dynamic urban scenes from unmanned aerial vehicle (UAV) full-motion videos is a vital task with significant applications in urban planning, traffic analysis, and autonomous navigation. However, modeling these scenes is challenging due to their large scale and, more importantly, the ever-changing presence of dynamic objects such as vehicles and pedestrians. In recent years, emerging neural 3D scene representation approaches have gained popularity for their promising performance in novel view synthesis, and several recent works have further explored the potential of modeling large-scale and dynamic scenes. While most existing methods focus on indoor or street-level scenes, very little effort has been made to address the unique complexities of dynamic urban environments captured by UAVs. To investigate this problem, we apply a recently developed dynamic 3D Gaussian Splatting framework that decomposes urban scenes into static and dynamic elements, thereby achieving efficient and accurate modeling. We further reduce the need for auxiliary input data, thereby accommodating more general cases in which only video sequences are available. Specifically, we propose a pipeline for automatically tracking dynamic vehicles using trajectory optimization to model their natural movement, thereby eliminating the dependency on prior knowledge of vehicles — which is often unavailable in real-life scenarios. By integrating the dynamic 3D Gaussian Splatting framework with the photogrammetric reconstruction pipeline, our pipeline offers scalable and reliable 3D dynamic scene reconstruction. Our pipeline is evaluated on multiple UAV datasets, and the results demonstrate the promising quality of scene reconstruction and view synthesis.