Data-Driven Simulation for Augmented Surgery

Abstract

To build an augmented view of an organ during surgery, it is essential to have a biomechanical model with appropriate material parameters and boundary conditions, able to match patient specific properties. Adaptation to the patient’s anatomy is obtained by exploiting the image-rich context specific to our application domain. While information about the organ shape, for instance, can be obtained preoperatively, other patient-specific parameters can only be determined intraoperatively. To this end, we are developing data-driven simulations, which exploit information extracted from a stream of medical images. Such simulations need to run in realtime. To this end we have developed dedicated numerical methods, which allow for real-time computation of finite element simulations. The general principle consists in combining finite element approaches with Bayesian methods or deep learning techniques, that allow to keep control over the underlying computational model while allowing for inputs from the real world. Based on a priori knowledge of the mechanical behavior of the considered organ, we select a constitutive law to model its deformations. The predictive power of such constitutive law highly depends on the knowledge of the material parameters and the boundary conditions. In our first approach, material properties are modeled as stochastic parameters whose probability distributions will be modified in real-time using Kalman filters, given observations extracted from intraoperative data. The second option we propose is to directly learn material parameters and boundary conditions from patient-specific data using deep neural networks. This has been applied to the modeling of liver biomechanics, its real-time simulation, and parametrization to achieve patient-specific augmented reality during