3D object recognition for Virtual Reality based Digital Twins

Abstract

Abstract: Recent advances in VR/AR have fueled the interest in automating the development of 3D interfaces. An important step of this automation is 3D object recognition. This allows applications to pre-configure assets based on expected behaviours, rather than requiring human developers to hard code such assets. This paper presents a clustering approach for reliably identifying 3D objects. Model-based descriptors are used to extract key features and geometric data are used to develop models for 3D object recognition. Two clustering approaches are used in this paper: hierarchical and K-means. The models are tested on two industrial gearboxes and an accuracy score is calculated to test model performance. The two gearboxes were too small resulting in the accuracy of both algorithms being identical, however the computational efficiency of the K-means algorithm makes it a more suitable choice. The accuracy for both algorithms was 74% when tested on a combined dataset of both gearboxes. This increased to 81% when difficult to identify parts are removed. The model can reliably identify geometrically consistent parts like ball bearings, roller bearings and screws.