Non-rigid Medical Image Registration using Physics-informed Neural Networks

Abstract

Biomechanical modelling of soft tissue provides a non-data-driven method for constraining medical image registration, such that the estimated spatial transformation is considered biophysically plausible. This has not only been adopted in real-world clinical applications, such as the MR-to-ultrasound registration for prostate intervention of interest in this work, but also provides an explainable means of understanding the organ motion and spatial correspondence establishment. This work instantiates the recently-proposed physics-informed neural networks (PINNs) to a 3D linear elastic model for modelling prostate motion commonly encountered during transrectal ultrasound guided procedures. To overcome a widely-recognised challenge in generalising PINNs to different subjects, we propose to use PointNet as the nodal-permutation-invariant feature extractor, together with a registration algorithm that aligns point sets and simultaneously takes into account the PINN-imposed biomechanics. Using 77 pairs of MR and ultrasound images from real clinical prostate cancer biopsy, we first demonstrate the efficacy of the proposed registration algorithms in an “unsupervised” subject-specific manner for reducing the target registration error (TRE) compared to that without PINNs especially for patients with large deformations. The improvements stem from the intended biomechanical characteristics being regularised, e.g., the resulting deformation magnitude in rigid transition zones was effectively modulated to be smaller than that in softer peripheral zones. This is further validated to achieve low registration error values of 1.90±0.52 mm and 1.94±0.59 mm for all and surface nodes, respectively, based on ground-truth computed using finite element methods. We then extend and validate the PINN-constrained registration network that can generalise to new subjects. The trained network reduced the rigid-to-soft-region ratio of rigid-excluded deformation magnitude from 1.35±0.15, without PINNs, to 0.89±0.11 (p< 0.001 ) on unseen holdout subjects, which also witnessed decreased TREs from 6.96±1.90 mm to 6.12±1.95 mm (p= 0.018 ). The codes are available at https://github.com/ZheMin-1992/Registration_PINNs.