A Machine Learning Tool to Match 2D Drawings and 3D Objects’ Category for Populating Mockups in VR

Abstract

Virtual Environments (VE) relying on Virtual Reality (VR) can facilitate the co-design by enabling the users to create 3D mockups directly in the VE. Databases of 3D objects are helpful to populate the mockup and necessitate retrieving methods for the users. In early stages of the design process, the mockups are made up of common objects rather than variations of objects. Retrieving a 3D object in a large database can be fastidious even more in VR. Taking into account the necessity of a natural user’s interaction and the necessity to populate the mockup with common 3D objects, we propose, in this paper, a retrieval method based on 2D sketching in VR and machine learning. Our system is able to recognize 90 categories of objects related to VR interior design with an accuracy up to 86%. A preliminary study confirms the performance of the proposed solution.