Effects of diagnostic errors in pattern differentiation and acupuncture prescription: A single-blinded, interrater agreement study

Abstract

This study compared the interrater agreement for pattern differentiation and acupoints prescription between two groups of human patients simulated with different diagnostic outcomes. Patients were simulated using a dataset about zangfu patterns and separated into groups (n = 30 each) according to the diagnostic outcome determined by a computational model. A questionnaire with 90 patients was delivered to 6 TCM experts (4-year minimal of clinic experience) who were asked to indicate a single pattern (among 73) and 8 acupoints (among 378). Interrater agreement was higher for pattern differentiation than for acupuncture prescription. Interrater agreement on pattern differentiation was slight for both groups with correct (Light's = 0.167, 95% CI = [0.108; 0.254]) and incorrect diagnosis (Light's = 0.190, 95% CI = [0.120; 0.286]). Interrater agreement on acupuncture prescription was slight for both groups of correct (ι = 0.029, 95% CI = [0.015; 0.057]) and incorrect diagnosis (ι = 0.040, 95% CI = [0.023; 0.058], P = 0.075). Diagnostic performance of raters yielded the following: accuracy = 60.9%, sensitivity = 21.7%, and specificity = 100%. An overall improvement in the interrater agreement and diagnostic accuracy was observed when the data were analyzed using the internal systems instead of the pattern's labels.