Evaluation of LabRespond, a new automated validation system for clinical laboratory test results

Abstract

Background: Manual validation of laboratory test re suits is time-consuming, creating a demand for expert systems to automate this process. We have started to set up the program 'LabRespond', which covers five validation levels: administrative, technical, sample, patient, and clinical validation. We present the evaluation of a prototype of an automated patient validation system based on statistical methods, in contrast to the commecially available program 'VALAB', a rule-based automated validation system. Methods: In the present study, 163 willfully altered, erroneous test results out of 5421 were submitted for validation to LabRespond, VALAB, and to a group of clinical chemists (n = 9) who validated these test results manually. The test results rejected by three or more clinical chemists (n = 281) served as a secondary reference standard. Results: The error recovery rates of clinical chemists ranged from 23.9% to 71.2%. The recovery rates of LabRespond and VALAB were 77.9% and 71.8%, respectively (difference not significant). The false-positive rates were 82.7% for LabRespond, 83.6% for VALAB, and 27.8-86.7% for clinical chemists. Using the consensus of three or more clinical chemists as the secondary reference standard, we found error recovery rates of 64.8% for LabRespond and 72.2% for VALAB (P = 0.06). Compared with VALAB, LabRespond detected more (P = 0.003) erroneous test results of the type that were changed from abnormal to normal. Conclusions: The statistical plausibility check used by LabRespond offers a promising automated validation method with a higher error recovery rate than the clinical chemists participating in this study, and a performance comparable to VALAB. (C) 2000 American Association for Clinical Chemistry.