Inter-rater reliability for diagnosis of geographic atrophy using spectral domain OCT in age-related macular degeneration

Abstract

Purpose: To evaluate the inter-rater reliability for identification of complete retinal pigment epithelium and outer retinal atrophy (cRORA) on SD-OCT images as defined by the Classification of Atrophy Meetings (CAM) group. Methods: Fifty images of anonymized SD-OCT line scans of eyes with cRORA due to AMD were selected. Each.tiff image was saved in both black-on-white (BW) and white-on-black (WB) format. Five retina-trained clinicians graded both sets of images twice for the diagnosis of cRORA based on the CAM group definition. Fleiss kappa statistic was calculated for inter-rater reliability and Cohen’s kappa statistic for intra-grader and inter-grader reliability between any two graders. Results: The inter-grader reliability varied from as low as 0.28 to 0.92 for WB images and 0.34 to 0.86 for BW images. However, the inter-grader and intra-grader agreement was ĸ WB 0.92; ĸ BW 0.86 and ĸ 0.92 respectively, for graders accustomed to the CAM criteria. Fleiss kappa was ĸ 0.49 (p value < 0.0001) for WB images and ĸ 0.34 (p value < 0.0001 for BW images. Overall, the agreement was better using WB images for all parameters except RPE attenuation/loss. Conclusion: There is significant variability in diagnosis of cRORA on SD-OCT by retina-trained ophthalmologists in the real world. The study highlights the need for training to recognise the different features of cRORA prior to its implementation in clinical practice.