Interpreting the real-time dynamic 'sliding sign' and predicting pouch of Douglas obliteration: an interobserver, intraobserver, diagnostic-accuracy and learning-curve study

Abstract

OBJECTIVE: To determine inter- and intraobserver agreement, diagnostic accuracy and the learning curve required for interpreting the 'sliding sign' and predicting pouch of Douglas (POD) obliteration. METHODS: This was an inter-/intraobserver, diagnostic-accuracy and learning-curve study involving six observers with different medical backgrounds, clinical skill sets and prior gynecological ultrasound experience: five non-specialist observers who had performed 0-750 previous gynecological scans and an expert sonologist who had performed  > 15 000. Following a formal theoretical and practical training session, they each viewed 64 offline transvaginal ultrasound (TVS) 'sliding-sign' videos from two anatomical locations (retrocervix and posterior uterine fundus (PUF)) in 32 women presenting with chronic pelvic pain, interpreting the videos as positive or negative for sliding sign and predicting, on that basis, the POD status. For intraobserver agreement analysis they re-analyzed the same video sets, in a different order, at least 7 days later. The expert sonologist was the reference standard for interpreting the sliding sign and the gold standard, laparoscopy, was used for the POD analysis. Learning-curve cumulative summation (LC-CUSUM) tests were conducted to assess if observer performance reached acceptable levels, using LC-CUSUM score < -2.45 as a cut-off. CONCLUSIONS: Performance of a minimum number of gynecological ultrasound examinations is necessary for interpreting offline videos of the real-time dynamic sliding sign and predicting POD obliteration. Non-specialist observers with prior experience of 200 or more gynecological scans were more consistent in interpreting the sliding sign at the retrocervix vs PUF. Copyright © 2015 ISUOG. Published by John Wiley & Sons Ltd. RESULTS: With respect to interpretation of the sliding sign, the overall multiple-rater agreement was moderate (Fleiss' kappa, K = 0.499). Observers were more consistent in their interpretation of the second compared with the first observation set (K = 0.547 vs 0.453) and for the retrocervical compared with the PUF region (K = 0.556 vs 0.346). Regarding prediction of POD status, the accuracy, sensitivity, specificity and positive and negative predictive values for individual observers ranged from 65.4 to 96.2%, 80.0 to 100%, 64.7 to 100%, 50.0 to 100% and 94.7 to 100%, respectively. Using LC-CUSUM score < -2.45, the observer with experience of 200 previous gynecological scans reached acceptable levels for predicting POD obliteration and interpreting the sliding sign at each region (retrocervix and PUF) at 39, 54 and 28 videos and the observer with experience of 750 scans at 56, 53 and 53 videos.