Selecting diagnostic tests for ruling out or ruling in disease: The use of the Kullback-Leibler distance

Abstract

Background. To select a proper diagnostic test, it is recommended that the most specific test be used to confirm (rule in) a diagnosis, and the most sensitive test be used to establish that a disease is unlikely (rule out). These rule-in and rule-out concepts can also be characterized by the likelihood ratio (LR). However, previous papers discussed only the case of binary tests and assumed test results already known. Methods. The author proposes using the 'Kullback-Leibler distance' as a new measure of rule-in/out potential. The Kullback-Leibler distance is an abstract concept arising from statistics and information theory. The author shows that it integrates in a proper way two sources of information - the distribution of test outcomes and the LR function. The index predicts the fate of an average subject before testing. Results. Analysis of real and hypothetical data demonstrates its applications beyond binary tests. It works even when the conventional methods of dichotomization and ROC curve analysis fail. Conclusions. The Kullback-Leibler distance nicely characterizes the before-test rule-in/out potentials. It offers a new perspective from which to evaluate a diagnostic test.