High interrater reliability of overnight pulse oximetry interpretation among inexperienced physicians using a structured template

Abstract

Study Objectives: To assess the interrater reliability and accuracy of overnight pulse oximetry (OPO) interpretations by pulmonary fellows using a comprehensive structured template and after a brief educational session. Methods: Using the template, four pulmonary and critical care (PCC) fellows interpreted OPO saturation waveforms and parameters extracted from 50 adult consecutive in-laboratory sleep studies. The template included three saturation parameters (mean saturation, oxygen desaturation index [ODI], and cumulative desaturation time) and description of the saturation waveform. A scoring system was proposed combining waveform characteristics and ODI to determine the suspicion for moderate to severe sleep apnea. Waveform description and mean saturation determined the suspicion for cardiopulmonary disease (CPD). Cumulative desaturation time determined need for oxygen prescription. Apnea-hypopnea index was extracted from the sleep study results. Results: The overall interrater reliability for final recommendations (sleep apnea suspicion, presence of CPD, and oxygen prescription) was high (kappa =.81, 95% confidence interval [CI].76-.88). Good agreement was noted in CPD evaluation and suspicion of moderate to severe sleep apnea (kappa =.70, 95% CI.46-.86 and kappa =.65, 95% CI.56-.77 respectively). The interrater reliability for oxygen prescription was in an excellent range (kappa =.98, 95% CI.91-1.00). The accuracy of a high sleep apnea suspicion score in detecting apnea-hypopnea index ≥ 15 events/h ranged from 88.0% to 94.0% (sensitivity 91.3% to 95.7%, specificity 81.5% to 92.6%). Desaturations due to CPD were identified by 75% of the raters as desaturations due to conditions other than sleep apnea. Conclusions: A structured template for OPO interpretation can produce a high interrater agreement and good accuracy, and make it a reliable clinical tool.