Performance of an artificial intelligence algorithm for reporting urine cytopathology

Abstract

Background: Unlike Papanicolaou tests, there are no commercially available computer-assisted automated screening systems for urine specimens. Despite The Paris System for Reporting Urinary Cytology, there still is poor interobserver agreement with urine cytology and many cases in which a definitive diagnosis cannot be made. In the current study, the authors have reported on the development of an image algorithm that applies computational methods to digitized liquid-based urine cytology slides. Methods: A total of 2405 archival ThinPrep glass slides, including voided and instrumented urine cytology cases, were digitized. A deep learning computational pipeline with multiple tiers of convolutional neural network models was developed for processing whole slide images (WSIs) and predicting diagnoses. The algorithm was validated using a separate test data set comprised of consecutive cases encountered in routine clinical practice. Results: There were 1.9 million urothelial cells analyzed. An average of 5400 urothelial cells were identified in each WSI. The algorithm achieved an area under the curve of 0.88 (95% CI, 0.83-0.93). Using the optimal operating point, the algorithm's sensitivity was 79.5% (95% CI, 64.7%-90.2%) and the specificity was 84.5% (95% CI, 81.6%-87.1%) for high-grade urothelial carcinoma. Conclusions: The authors successfully developed a computational algorithm capable of accurately analyzing WSIs of urine cytology cases. Compared with prior studies, this effort used a much larger data set, exploited whole slide–level and not just cell-level features, and used a cell gallery to display the algorithm's output for easy end-user review. This algorithm provides computer-assisted interpretation of urine cytology cases, akin to the machine learning technology currently used for automated Papanicolaou test screening.