Towards Explainable AI: Design and Development for Explanation of Machine Learning Predictions for a Patient Readmittance Medical Application

Abstract

The need for explainability of AI algorithms has been identified in the literature for some time now. However, recently became even more important due to new data protection act rules (GDPR 2018) and due to the requirements for wider applicability of AI to several application areas. BT’s autonomics team has recognized this through several sources and identified the vitality of AI algorithms explainability to ensure their adoption and commercialization. In this paper, we designed and developed a system providing explanations for a prediction of patient readmittance using machine learning. The requirements and the evaluation were set by BT through their projects with real-customers in the medical domain. A logistic regression machine learning algorithm was implemented with explainability “hooks” embedded in its code and the corresponding interfaces to the users of the system were implemented through a web interface. Python-based technologies were utilized for the implementation of the algorithm (Scikit-learn) and the web interface (web2py), and the system was evaluated through thorough testing and feedback. Initial trade-off analysis of such an approach that presents the overhead introduced by adding explainability versus the benefits was performed. Lastly, conclusions and future work are presented, considering experimentation with more algorithms and application of software engineering methods such as abstraction to the aid of explainable AI, leading further along to “explainability by design”.