Interrater reliability in interpretation of electrocorticographic seizure detections of the responsive neurostimulator

Abstract

Objective Electrocorticographic (ECoG) recordings from patients with medically intractable partial-onset seizures treated with a responsive neurostimulator system (the RNS System) that detects and stores physician-specified ECoG events provide a new data resource. Interpretation of these recordings has not yet been validated. The purpose was to evaluate the interrater interpretation of chronic ambulatory ECoG recordings obtained by the RNS System. Methods Five pairs of five experts independently classified 7,221 ECoG recordings obtained from 128 patients with medically intractable partial seizures who participated in a randomized controlled trial of the safety and efficacy of the RNS System. ECoG detections - "long episodes" or "saturations" - were classified as "seizures" or "not seizures" based on a reference definition. Interrater agreement rates and kappa score reliabilities were calculated between rater pairs from the ECoG sample as a whole and within individual patients who had more than the median number of individual ECoG recordings. Results The overall interrater agreement was 79%, with a reliability κ = 0.57 (moderate agreement). Agreement between pairs of reviewers ranged from 0.69 to 0.85. Agreement rates were 94% or better for 50% of patients. Only 25% of patients had ECoG recordings agreement rates worse than 75%. ECoGs with mixed interpretations (one reviewer "seizure"/the other - "not seizure") consisted of periods of low amplitude activity that evolved in amplitude or periodic discharges near 2 Hz. Significance Although reliability as a whole was moderate, for the majority of patients, detections yielded highly reliably interpreted events of either electrographic seizures or nonictal epileptiform activity.