Assessment of the accuracy of imaging software for 3D rendering of the upper airway, usable in orthodontic and craniofacial clinical settings

Abstract

Background: Several semi-automatic software are available for the three-dimensional reconstruction of the airway from DICOM files. The aim of this study was to evaluate the accuracy of the segmentation of the upper airway testing four free source and one commercially available semi-automatic software. A total of 20 cone-beam computed tomography (CBCT) were selected to perform semi-automatic segmentation of the upper airway. The software tested were Invesalius, ITK-Snap, Dolphin 3D, 3D Slicer and Seg3D. The same upper airway models were manually segmented (Mimics software) and set as the gold standard (GS) reference of the investigation. A specific 3D imaging technology was used to perform the superimposition between the upper airway model obtained with semi-automatic software and the GS model, and to perform the surface-to-surface matching analysis. The accuracy of semi-automatic segmentation was evaluated calculating the volumetric mean differences (mean bias and limits of agreement) and the percentage of matching of the upper airway models compared to the manual segmentation (GS). Qualitative assessments were performed using color-coded maps. All data were statistically analyzed for software comparisons. Results: Statistically significant differences were found in the volumetric dimensions of the upper airway models and in the matching percentage among the tested software (p < 0.001). Invesalius was the most accurate software for 3D rendering of the upper airway (mean bias = 1.54 cm3; matching = 90.05%) followed by ITK-Snap (mean bias = − 2.52 cm3; matching = 84.44%), Seg 3D (mean bias = 3.21 cm3, matching = 87.36%), 3D Slicer (mean bias = − 4.77 cm3; matching = 82.08%) and Dolphin 3D (difference mean = − 6.06 cm3; matching = 78.26%). According to the color-coded map, the dis-matched area was mainly located at the most anterior nasal region of the airway. Volumetric data showed excellent inter-software reliability (GS vs semi-automatic software), with coefficient values ranging from 0.904 to 0.993, confirming proportional equivalence with manual segmentation. Conclusion: Despite the excellent inter-software reliability, different semi-automatic segmentation algorithms could generate different patterns of inaccuracy error (underestimation/overestimation) of the upper airway models. Thus, is unreasonable to expect volumetric agreement among different software packages for the 3D rendering of the upper airway anatomy.