Interobserver agreement in describing adnexal masses using the International Ovarian Tumor Analysis simple rules in a real-time setting and using three-dimensional ultrasound volumes and digital clips

Abstract

Objective To estimate the agreement between an expert and a non-expert examiner using the International Ovarian Tumor Analysis (IOTA) simple rules for classifying adnexal masses on real-time ultrasound and when using three-dimensional (3D) ultrasound volumes and digital clips. Methods Forty-two non-consecutive women diagnosed as having an adnexal mass were evaluated by transvaginal power Doppler ultrasound as part of their diagnostic work-up. In each woman, examination was first performed by a non-expert examiner (a trainee) and immediately afterwards by an expert examiner. Both used the IOTA simple rules to describe the mass, blinded to each other's results. After finishing the examination, each examiner classified the mass as benign, malignant or inconclusive, according to the IOTA simple rules. Additionally, the expert recorded a short videoclip and acquired a static 3D volume of each mass, which were subsequently assessed by four trainees in obstetrics and gynecology with different levels of training, who also classified the mass as benign, malignant or inconclusive according to the IOTA simple rules. Agreement was assessed by calculating weighted and standard kappa index values with 95% CI and the percentage of agreement between observers. Results Agreement between the observers who performed real-time ultrasound examination was good (weighted kappa = 0.76; 95% CI, 0.61-0.90; agreement = 78.6%). Agreement between trainees using videoclips plus 3D volumes was moderate (kappa values ranged from 0.45 to 0.58, depending on pair comparison). Conclusion Interobserver agreement of the IOTA simple rules for classifying adnexal masses as benign, malignant or inconclusive using real-time ultrasound, between an expert and a non-expert examiner, might be considered good. Agreement using a videoclip plus a 3D volume was moderate for trainees with different degrees of training. Copyright © 2013 ISUOG. Published by John Wiley & Sons Ltd.