Comparing computer vision models for low resource dataset to develop a mixed reality based manual assembly assistant

Abstract

Assembling complex components often requires expert guidance. Augmented Reality (AR) offers intuitive visual assistance that can enhance user performance, yet existing AR-based systems largely focus on simple tasks, limiting their application to intricate assembly scenarios. This paper addresses the gap by developing a comprehensive pipeline for AI and AR-based assembly guidance. We analyzed various software architectures, selecting the optimal setup for advanced manufacturing based on latency, accuracy, and human factors. A novel AI-based hologram registration technique was implemented to provide real-time assistance in dynamic environments. Additionally, a multimodal user interface was designed to facilitate seamless interaction with the Mixed Reality (MR) system, allowing users to access instructions efficiently. Results demonstrate that users completed tasks faster with the MR system compared to traditional video-based methods. The proposed system also significantly reduces cognitive load and improves usability, positioning it as an effective tool for modern manufacturing.