Development and Validation of an Open-Source Grading Tool for Outcome Assessment in Limbal Stem Cell Treatment

Abstract

Purpose: To design a grading system and validate an open-source tool to improve objective quantification and follow-up of limbal stem cell deficiency (LSCD) after treatment. Methods: A custom-made web-based grading system was developed for grading stem cell deficient eyes, termed the "Vascularisation, Haze, and Integrity"tool. For validation purposes, 60 corneal slit-lamp images of 30 limbal stem cell deficient eyes were graded by 3 groups of examiners: 3 corneal specialists (group A), 3 ophthalmologists with an expertise other than cornea (group B), and 3 nonclinicians (group C). The intragrader and intergrader agreement was evaluated using Fleiss weighted kappa coefficients and concurrent assessment of interrater and intrarater reliability (IRR) coefficients. Results: The overall intergrader agreement was 0.78, 0.61, and 0.42 for superficial corneal vascularization, corneal haze, and epithelial integrity, respectively. All groups had good agreement for the vascularization parameter with the highest intergrader reliability in group A (IRR = 0.80) and the lowest in group C (IRR = 0.72). When assessing "haze,"there was good agreement in groups A (IRR = 0.75) and B (IRR = 0.76) but low agreement in group C (IRR = 0.37). Conclusions: We report the development and evaluation of a novel method for grading results of limbal stem cell deficient eyes after treatment and provide this system as a free, open-source online tool. The grading tool offers an easy and standardized way of assessing the corneal surface in patients with LSCD, enables evaluation of progression over time, reduces assessment bias, and - if adopted universally - will harmonize outcome being reported between groups.