Towards Regulatory-Compliant MLOps: Oravizio’s Journey from a Machine Learning Experiment to a Deployed Certified Medical Product

Abstract

Agile software development embraces change and manifests working software over comprehensive documentation and responding to change over following a plan. The ability to continuously release software has enabled a development approach where experimental features are put to use, and, if they stand the test of real use, they remain in production. Examples of such features include machine learning (ML) models, which are usually pre-trained, but can still evolve in production. However, many domains require more plan-driven approach to avoid hazard to environment and humans, and to mitigate risks in the process. In this paper, we start by presenting continuous software engineering practices in a regulated context, and then apply the results to the emerging practice of MLOps, or continuous delivery of ML features. Furthermore, as a practical contribution, we present a case study regarding Oravizio, first CE-certified medical software for assessing the risks of joint replacement surgeries. Towards the end of the paper, we also reflect the Oravizio experiences to MLOps in regulatory context.