Comparison of chest CT grading systems in COVID-19 pneumonia

Abstract

Purpose: To compare the performance and interobserver agreement of the COVID-19 Reporting and Data System (CO-RADS), the COVID-19 imaging reporting and data system (COVID-RADS), the RSNA expert consensus statement, and the British Society of Thoracic Imaging (BSTI) guidance statement. Materials and Methods: In this case-control study, a total of 100 symptomatic patients suspected of having coronavirus disease 2019 (COVID-19) were included: 50 patients with COVID-19 (59 years 6 17, 38 men) and 50 patients without COVID-19 (65 years 6 24, 30 men). Eight radiologists independently scored chest CT images of the cohort according to each reporting system. The area under the receiver operating characteristic curve (AUC) and interobserver agreements were calculated and statistically compared across the systems. Results: A total of 800 observations were made for each system. The level of suspicion of COVID-19 correlated with the reverse-transcription polymerase chain reaction (RT-PCR) positive rate except for the “negative for pneumonia” classifications in all the systems (Spearman coefficient: r = 1.0, P -.001 for all the systems). Average AUCs were as follows: CO-RADS, 0.84 (95% CI: 0.83, 0.85); COVID-RADS, 0.80 (95% CI: 0.78, 0.81); the RSNA statement, 0.81 (95% CI: 0.79, 0.82); and the BSTI statement, 0.84 (95% CI: 0.812, 0.86). Average Cohen k across observers was 0.62 (95% CI: 0.58, 0.66), 0.63 (95% CI: 0.58, 0.68), 0.63 (95% CI: 0.57, 0.69), and 0.61 (95% CI: 0.58, 0.64) for CO-RADS, COVID-RADS, the RSNA statement, and the BSTI statement, respectively. CO-RADS and the BSTI statement outperformed COVID-RADS and the RSNA statement in diagnostic performance (P,.05 for all the comparison). Conclusion: CO-RADS, COVID-RADS, the RSNA statement, and the BSTI statement provided reasonable performances and interobserver agreements in reporting CT findings of COVID-19.