Artificial intelligence versus radiologist in the accuracy of fracture detection based on computed tomography images: a multidimensional, multi-region analysis

Abstract

Background: Extremities fractures are a leading cause of death and disability, especially in the elderly. Avulsion fracture are also the most commonly missed diagnosis, and delayed diagnosis leads to higher litigation rates. Therefore, this study evaluates the diagnostic efficiency of the artificial intelligence (AI) model before and after optimization based on computed tomography (CT) images and then compares it with that of radiologists, especially for avulsion fractures. Methods: The digital X-ray photography [digital radiography (DR)] and CT images of adult limb trauma in our hospital from 2017 to 2020 were retrospectively collected, with or without 1 or more fractures of the shoulder, elbow, wrist, hand, hip, knee, ankle, and foot. Labeling of the fracture referred to the visualization of the fracture on the corresponding CT images. After training the pre-optimized AI model, the diagnostic performance of the pre-optimized AI, optimized AI model, and the initial radiological reports were evaluated. For the lesion level, the detection rate of avulsion and non-avulsion fractures was analyzed, whereas for the case level, the accuracy, sensitivity, and specificity were compared among them. Results: The total datasets (1,035 cases) were divided into a training set (n=675), a validation set (n=169), and a test set (n=191) in a balanced joint distribution. At the lesion level, the detection rates of avulsion fracture (57.89% vs. 35.09%, P=0.004) and non-avulsion fracture (85.64% vs. 71.29%, P<0.001) by the optimized AI were significantly higher than that by pre-optimized AI. The average precision (AP) of the optimized AI model for all lesions was higher than that of pre-optimized AI model (0.582 vs. 0.425). The detection rate of avulsion fracture by the optimized AI model was significantly higher than that by radiologists (57.89% vs. 29.82%, P=0.002). For the non-avulsion fracture, there was no significant difference of detection rate between the optimized AI model and radiologists (P=0.853). At the case level, the accuracy (86.40% vs. 71.93%, P<0.001) and sensitivity (87.29% vs. 73.48%, P<0.001) of the optimized AI were significantly higher than those of the pre-optimized AI model. There was no statistical difference in accuracy, sensitivity, and specificity between the optimized AI model and the radiologists (P>0.05). Conclusions: The optimized AI model improves the diagnostic efficacy in detecting extremity fractures on radiographs, and the optimized AI model is significantly better than radiologists in detecting avulsion fractures, which may be helpful in the clinical practice of orthopedic emergency.