Are treatment recommendations provided by cognitive computing supported by real world data (Watson for Oncology with Cota RWE) concordant with expert opinions?

Abstract

Background: Treatment selection requires review of patient (pt) and clinical features, medical literature, national guidelines, physician experience, and cost-value issues. The IBM Watson for Oncology with Cota RWE (WfO/Cota) point-of-care decision support tool ingests pt attributes from electronic health records and displays treatment recommendations (TRx) based on Memorial Sloan Kettering Cancer Center training and medical literature. The system has been recently supplemented with real world data curated by Cota listing historical treatments and survival outcomes drawn from the treating physician's institution and a national database. WfO/Cota is undergoing testing at John Theurer Cancer Center (JTCC, Hackensack, NJ, USA). Concordance of WfO/Cota with expert opinions is required to confirm that cognitive computing TRx match best practices. Methods: 88 early stage post-menopausal breast cancer (BC) cases from the JTCC BC clinic were presented to 3 JTCC BC experts (without using WfO/Cota). The cases were compared against pts with similar demographic and disease characteristics from the Cota database (matched using Cota Nodal Address [CNA] algorithms). Results: BC experts reviewed 223 cases (not all cases scored by each). WfO/Cota “recommended” option was concordant with selection by BC experts in 175 (78.5%) and “for consideration” option was selected in 21 (9.4%); experts agreed with WfO/ Cota in 87.9%. 7 of 88 cases (8%) generated 59% of non-concordant responses with->=2 doctors disagreeing with WfO. The BC expert who worked at MSKCC deviated the least from MSKCC trained WfO. In the Cota database 69.3% of matched historical controls were treated with “recommended,” 11.4% “for consideration”, 19.3% “not recommended.” Conclusions: WfO/Cota recommendations are largely concordant with disease expert best oncology practices. The observation that nearly a fifth of pts with similar disease (CNA) characteristics received non-recommended options in a real world database highlights a need. WfO/Cota is an innovative decision support tool that derives new insights based on existing real world evidence to reduce variations in practice.