Impact of e-ASPECTS software on the performance of physicians compared to a consensus ground truth: a multi-reader, multi-case study

Abstract

Background: The Alberta Stroke Program Early CT Score (ASPECTS) is used to quantify the extent of injury to the brain following acute ischemic stroke (AIS) and to inform treatment decisions. The e-ASPECTS software uses artificial intelligence methods to automatically process non-contrast CT (NCCT) brain scans from patients with AIS affecting the middle cerebral artery (MCA) territory and generate an ASPECTS. This study aimed to evaluate the impact of e-ASPECTS (Brainomix, Oxford, UK) on the performance of US physicians compared to a consensus ground truth. Methods: The study used a multi-reader, multi-case design. A total of 10 US board-certified physicians (neurologists and neuroradiologists) scored 54 NCCT brain scans of patients with AIS affecting the MCA territory. Each reader scored each scan on two occasions: once with and once without reference to the e-ASPECTS software, in random order. Agreement with a reference standard (expert consensus read with reference to follow-up imaging) was evaluated with and without software support. Results: A comparison of the area under the curve (AUC) for each reader showed a significant improvement from 0.81 to 0.83 (p = 0.028) with the support of the e-ASPECTS tool. The agreement of reader ASPECTS scoring with the reference standard was improved with e-ASPECTS compared to unassisted reading of scans: Cohen's kappa improved from 0.60 to 0.65, and the case-based weighted Kappa improved from 0.70 to 0.81. Conclusion: Decision support with the e-ASPECTS software significantly improves the accuracy of ASPECTS scoring, even by expert US neurologists and neuroradiologists.