Reliability and validity of hospital pharmacists' clinical intervention data

Abstract

The reliability and validity of pharmacist intervention data documented at a hospital in England were studied. The study involved intervention data recorded from September 1 to December 31 in 1990 (a pilot study), 1991, and 1992 by a total of 23 pharmacists. The interventions in 1990 were coded by a single pharmacist manager and compared with coding by pharmacists of their own interventions during 1991 and 1992. To test for internal reliability of the intervention data, 25% (584) of the 2342 interventions recorded in 1991 and 1992 were randomly sampled after interventions by the 11 pharmacists no longer at the site were eliminated. In September 1993 the 12 remaining pharmacists were asked to recode the interventions they had previously entered. In March 1994 a panel of 12 pharmacists recoded 62 interventions randomly selected from the 584 recoded interventions. Each panel member also assigned financial values to interventions. Medical records corresponding to the 62 recoded interventions were inspected for evidence of the interventions. The distribution of interventions among the drug-use-process (DUP) indicators used in the coding differed significantly between 1990 (when a manager assigned all codes) and 1991 plus 1992 (when pharmacists coded their own interventions). There were similar differences between 1991 and 1992. Results from code-recode and panel testing showed the DUP indicators to be reliable. Clinical outcome indicators were unreliable, as were financial indicators. Medical records verified the occurrence of 50 (81%) of the 62 interventions selected, but in only one case was there evidence that the change in drug therapy was initiated by a pharmacist. The capacity of pharmacists at a British hospital to reliably code interventions was poor.