Semantics-Driven 3D Scene Retrieval via Joint Loss Deep Learning

Abstract

Three-dimensional (3D) scene model retrieval has emerged as a novel and challenging area within content-based 3D model retrieval research. It plays an increasingly critical role in various domains, such as video games, film production, and immersive technologies, including virtual reality (VR), augmented reality (AR), and mixed reality (MR), where automated generation of 3D content is highly desirable. Despite their potential, the existing 3D scene retrieval techniques often overlook the rich semantic relationships among objects and between objects and their surrounding scenes. To address this gap, we introduce a comprehensive scene semantic tree that systematically encodes learned object occurrence probabilities within each scene category, capturing essential semantic information. Building upon this structure, we propose a novel semantics-driven image-based 3D scene retrieval method. The experimental evaluations show that the proposed approach effectively models scene semantics, enables more accurate similarity assessments between 3D scenes, and achieves substantial performance improvements. All the experimental results, along with the associated code and datasets, are available on the project website.