Convolutional neural network and in-painting techniques for the automatic assessment of scoliotic spine surgery from biplanar radiographs

Abstract

Assessing the effectiveness of scoliosis surgery requires the quantification of 3D spinal deformities from pre- and post-operative radiographs. This can be achieved from 3D reconstructed models of the spine but a fast-automatic method to recover this model from pre- and post-operative radiographs remains a challenge. For example, the vertebrae’s visibility varies considerably and large metallic objects occlude important landmarks in postoperative radiographs. This paper presents a method for automatic 3D spine reconstruction from pre- and post-operative calibrated biplanar radiographs. We fitted a statistical shape model of the spine to images by using a 3D/2D registration based on convolutional neural networks. The metallic structures in postoperative radiographs were detected and removed using an image in-painting method to improve the performance of vertebrae registration. We applied the method to a set of 38 operated patients and clinical parameters were computed (such as the Cobb and kyphosis/lordosis angles, and vertebral axial rotations) from the pre- and post-operative 3D reconstructions. Compared to manual annotations, the proposed automatic method provided values with a mean absolute error <5.6° and <6.8° for clinical angles; <1.5 mm and <2.3 mm for vertebra locations; and <4.5° and <3.7° for vertebra orientations, respectively for pre- and post-operative times. The fast-automatic 3D reconstruction from pre- and post in-painted images provided a relevant set of parameters to assess the spine surgery without any human intervention.