P151 - How variable is the Mayo score between observers and might this affect trial recruitment or outcome?

Abstract

Background:The Mayo score for ulcerative colitis(UC) activity has 4 components:stool frequency(SF),rectal bleeding(RB),flexible sigmoidoscopy(FS)and physician's global assessment(PGA). How interobserver variation(IOV)affects the Mayo score and its impact on criteria for inclusion or outcome of clinical trials are unknown. Method(s):100 patients with UC were seen independently, each on the same day,in random order,by 4 gastroenterologists. Both patient and clinician completed a proforma. A separate clinician performed videorecorded FS on the same day which was later scored by the 4 gastroenterologists. Each component of the score and total score were calculated for each patient. Comparison was made with inclusion criteria for ACT 1&2 trials (Mayo 6-12, endoscopy subscore >2),remis-sion outcome (Mayo <2, no subscore >1). For clinical relevance(CR),scores were categorised as remission(<2),mild(3-5),moderate(6-8),or severe(9-12) activity and an experienced, blinded clinician independently assigned an appropriate clinical category (ACC) to each patient by assessing symptoms, examination, blood results, FS and histology. Quadratic weighted kappa statistics assessed agreement within the Mayo score,where disagreements are weighted in relation to their magnitude. Result(s): Of 100 patients, there was complete agreement between 4 clinicians in total Mayo score in 6/96 (4 had no FS), varying by <2 points in 84/96, which changed clinical category in 23/84. Overall agreement for CR and ACC were good (kappa=0.88 and kappa=0.81 respectively). Between patients and clinicians there was 70% agreement for SF and RB. Between clinicians there was complete agreement in 65% for SF, 74% for RB, but only 21% for FS and 45% for PGA. Most disagreement was by one category (median 81%, range 74-93). For inclusion criteria, at least 1 clinician would have included 41/96, but all agreed in only 17/41(41%). At least 3/4 clinicians would have included 22 and excluded 67, so there was at least 25% disagreement in 26/96(27%) and 50% disagreement in 11/96(11%). 11% had FS score >=2 but total score 6 but a FS score <1. For remission,at least 1 clinician scored <2 in 43/96, but all agreed in only 20/ 43 (47%) and 3/4 agreed in 35/43 (81%). Agreement was not significantly improved by a total score <1 (at least 1 = 39/96; all agree 20/39; 3/4 agree 35/39). Conclusion(s): There is high variability in Mayo scoring between observers,despite good agreement on clinical category. Complete agreement between observers for recruitment to clinical trials or outcome occurs in <50% and 3/4 agreement in about 80% patients. IOV should be considered when calculating the power of clinical trials.