POS0918 VALIDATION OF A MODIFIED NEW YORK CRITERIA RETIC E-TOOL FOR INCREASING SCORING PROFICIENCY OF RADIOGRAPHS OF THE SACROILIAC JOINTS IN AXIAL SPONDYLOARTHRITIS

Abstract

Background: Radiographic assessment of the sacroiliac joints (SIJ) according to the modified New York (mNY) criteria is essential in the classification of ankylosing spondylitis, but low to moderate inter‐reader agreement and limited improvement by calibration have been reported [1]. A web‐based real‐time iterative calibration (RETIC) module for scoring SIJ radiographs according to the mNY criteria has been developed by experienced readers to allow remote standardized calibration. Objectives: To test the performance of this online RETIC calibration module in enhancing scoring proficiency with an expert reader as gold standard reference. Methods: This RETIC module consisted of 50 cases with radiographs of the SIJ at one timepoint with an integrated scoring interface with real time feedback to the reader's grading compared to the expert readers' grading. The mNY grade (0‐4) was assessed for each joint. Reliability for fulfilling versus not fulfilling the mNY criteria was assessed in real‐time by kappa data being provided every 10 cases, until proficiency targets (kappa > 0.5) were achieved. In this study, 19 readers from the EuroSpA Imaging project were randomized to one of two reader calibration strategies (groups A and B) that comprised of 3 calibration steps, each followed by a reading exercise (Exercises 1‐3; 25 patients per exercise). Calibration steps were as follows: Group A; 1. Review of original manuscripts describing the mNY method. 2. Review of PowerPoint summary of mNY method and video with grading examples PLUS completion of mNYRETIC module. 3. Re‐review of PowerPoint summary and video. Group B; same 3‐step strategy as group A except that the mNYRETIC module was completed at step 3, i.e. immediately before Exercise 3. The reliability of scoring was compared to an expert radiologist (RL) using kappa statistics for positivity of mNY criteria and intraclass correlation coefficients (ICCs) for the grades of the mNY criteria. Results: The mean agreement for all readers on fulfillment of the mNY criteria with kappa was 0.69 in exercise 1, 0.72 in exercise 2 and 0.83 in exercise 3. Corresponding ICCs for mNY scoring 0‐4 were 0.77, 0.77 and 0.82 in exercises 1, 2 and 3, respectively, see Table 1 and Figure 1. For Reader Group A, kappas for exercises 1‐3 were 0.65, 0.76 and 0.83, respectively and ICCs 0.71, 0.82 and 0.81 (exercise 3). Interestingly, the main increase in agreement occurred between Exercises 1 and 2, i.e. after the RETIC module was performed. For Reader group B, corresponding kappas were 0.71, 0.77 and 0.83 and ICCs 0.80, 0.72 and 0.80, for exercises 1‐3, respectively. Similarly, agreements predominantly improved directly after the RETIC module was performed. Conclusion: Agreement on scoring radiographs of the SIJ according to the mNY criteria was noticeably improved when using a systematic online calibration module and in general readers achieved very good agreement with experts implying that systematic calibration of readers before scoring of images can enhance scoring proficiency.