Advancing industrial inspection with an automated computer vision solution for orthopedic surgical tray inspection

Abstract

This work aimed to automate the current manual inspection process of orthopedic joint reconstruction loaner trays, which contain multiple components securely placed in designated slots, by developing an end-to-end pipeline, including an object detection model followed by a tray layout verification algorithm. We collected video training data from 74 trays comprising 1039 unique components. We trained a single You Only Look Once (YOLOv7) model capable of detecting 1039 classes, where we customized the loss and non-max suppression (NMS) functions. For layout verification, we implemented an in-house algorithm based on a pretrained Local Feature Matching with Transformer (LoFTR) model to verify each component’s presence and correct placement within a tray. We evaluated our end-to-end pipeline on 139 testing inspection images and achieved an overall mean average precision (mAP@0.5) and false-positive rate of 0.94 ± 0.10 and 0.05 ± 0.08, respectively, across 12 test scenarios. Our enhanced YOLOv7-X architecture outperformed the Region-based Convolutional Neural Network (Faster-RCNN-ResNet101) baseline model by 24%. Remarkably, in 7 out of 12 scenarios, we achieved an mAP exceeding 0.99. Our proposed solution substantially reduces the inspection time by 47.3% and is highly scalable, allowing straightforward integration into broad industrial applications.