American Journal of Epidemiology Copyright �� 1998 by The Johns Hopkins University School of Hygiene and Public Health All rights reserved Vol. 148, No. 11 Printed in U.S.A. Slopes of a Receiver Operating Characteristic Curve and Likelihood Ratios for a Diagnostic Test Bernard C. K. Choi1 This paper clarifies two important concepts in clinical epidemiology: the slope of a receiver operating characteristic (ROC) curve and the likelihood ratio. It points out that there are three types of slopes in an ROC curve���the tangent at a point on the curve, the slope between the origin and a point on the curve, and the slope between two points on the curve. It also points out that there are three types of likelihood ratios that can be defined for a diagnostic test that produces results on a continuous scale���the likelihood ratio for a particular single test value, the likelihood ratio for a positive test result, and the likelihood ratio for a test result in a particular level or category. It further illustrates mathematically and empirically the following three relations between these various definitions of slopes and likelihood ratios: 1) the tangent at a point on the ROC curve corresponds to the likelihood ratio for a single test value represented by that point 2) the slope between the origin and a point on the curve corresponds to the positive likelihood ratio using the point as a criterion for positivity and 3) the slope between two points on the curve corresponds to the likelihood ratio for a test result in a defined level bounded by the two points. The likelihood ratio for a single test value is considered an important parameter for evaluating diagnostic tests, but it is not easily estimable directly from laboratory data because of limited sample size. However, by using ROC analysis, the likelihood ratio for a single test value can be easily measured from the tangent. It is suggested that existing ROC analysis software be revised to provide estimates for tangents at various points on the ROC curve. Am J Epidemiol 1998,148:1127-32. data interpretation, statistical diagnostic tests, routine likelihood ratio models, statistical ROC curve It has been suggested that the slope of a receiver operating characteristic (ROC) curve represents a like- lihood ratio for a diagnostic test (1, 2). This seems to be intuitively correct, since the ROC curve is a plot of the true positive rate versus the false positive rate, and a likelihood ratio (for a positive test) is defined as the ratio of the true positive rate to the false positive rate. (Technically, a "true positive rate" or a "false positive rate" is not a rate but a proportion. However, since these terms are used widely in the epidemiologic lit- erature, they are used in this paper.) There are problems, however, in using the above generally recommended relation. First, the slope of an Received for publication December 2, 1996, and accepted for publication April 20, 1998. Abbreviations: FPR, false positive rate LR, likelihood ratio ROC, receiver operating characteristic TPR, true positive rate. 1 Bureau of Cardio-Respiratory Diseases and Diabetes, Labora- tory Centre for Disease Control, Health Canada, Ottawa, Ontario, Canada. 2 Department of Public Health Sciences, Faculty of Medicine, and Faculty of Dentistry, University of Toronto, Toronto, Ontario, Canada. 3 Department of Epidemiology and Community Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada. Reprint requests to Dr. Bernard Choi at the Laboratory Centre for Disease Control, Health Canada, PL #1918C3, Tunney's Pasture, Ottawa, Ontario, Canada K1A 0K9. ROC curve has not been clearly defined. Many authors refer to it as the "slope of the ROC curve" (3-6) or simply the "ROC curve slope" (7). Others have im- plied that the slope is the tangent of the curve at a particular point, speaking of, for example, the "slope of the curve at that point" (1) or the "slope at each point on the ROC curve" (8). (Some authors have defined the slope of an ROC curve as the slope of a straight line which is derived by plotting ROC points on binormal coordinate paper (9, 10) that definition is not relevant to this article.) Second, the likelihood ratio that is measured by the slope of the ROC curve has not been defined. Many authors have simply called it a "likelihood ratio" (1). Some authors have sug- gested that it is the positive likelihood ratio which is measured by the ROC curve slope (8). The definition of a likelihood ratio is further complicated when a test produces results that are not dichotomous, a condition that is required for generating an ROC curve. In such a case, more than one likelihood ratio can be defined. This paper has two purposes: first, to point out that there are three types of slopes which can be defined for an ROC curve, and that there are three types of like- lihood ratios which can be defined for tests that gen- erate results on a continuous scale (e.g., the serum 1127

1128 Choi creatinine kinase test for the diagnosis of myocardial infarction (11), the pulmonary function test for the diagnosis of asthma (12), or the serum glucose test for the diagnosis of diabetes mellitus (13)) and second, to illustrate mathematically the relations between these various slopes and likelihood ratios. An empirical ex- ample using real data is provided to demonstrate the relations. METHODS The ROC curve ROC methodology has recently been reviewed (2, 14, 15). The ROC curve is constructed by plotting the true positive rate (TPR) on the y-axis as a function of the false positive rate (FPR) on the x-axis, for all possible cutoff values of a diagnostic test. The TPR is the proportion of patients with a disease who have a positive test, and the FPR is the proportion of patients without a disease who have a positive test. The likelihood ratio The likelihood ratio (LR) is defined as the ratio between the probability of a defined test result given the presence of a disease and the probability of the same test result given the absence of a disease (16, 17): LR = probability of a test result among the diseased persons probability of the same test result among the nondiseased persons. The likelihood ratio is useful in clinical decision- making because it is also the ratio of the post-test odds of disease (odds of disease among persons with a given test result) to the pretest odds of disease (odds of disease among all persons) (16, 17). If a test generates results on a continuous scale, then a likelihood ratio can theoretically be defined for each test value x: probability of a test result x among the diseased persons probability of the same test result x among the nondiseased persons. This likelihood ratio is theoretically estimable by using an infinitesimally small interval for the test result based on two criteria���e.g., 99.99 and 100.01 for a test value of 100. In practice, the likelihood ratio for a single test value is very difficult to estimate, unless an investigator is dealing with an exceptionally large sample. Therefore, likelihood ratios are more often calculated for test results on one side of a par- ticular criterion (dichotomous test), or for results in a larger interval defined by two criteria (multiple-level test). If a test generates dichotomous results (i.e., positive or negative, by using a specific criterion for positivi- ty), then two likelihood ratios can be defined���i.e., the likelihood ratio for a positive test (LR+) and the likelihood ratio for a negative test (LR���) (16): LR+ = LR- = probability of a positive test among the diseased persons probability of a positive test among the nondiseased persons. probability of a negative test among the diseased persons probability of a negative test among the nondiseased persons. If a test has multiple levels of results rather than just two, by using defined intervals on a continuous scale, then a likelihood ratio can be calculated for each level bounded by criteria x and y (17): LR(x,y) = probability of a test result bounded by criteria x and y among the diseased persons probability of the same test result bounded by criteria x and y among the nondiseased persons. ROC curve slopes and likelihood ratios The slope of an ROC curve can be defined in three ways: first, as the tangent at a particular point on the ROC curve corresponding to a test value x (i.e., tangent(x)) second, as the slope between the origin o (i.e., point (0,0)) and the point on the ROC curve corresponding to a test value x (i.e., slope(ox)) and third, as the slope between two points on the ROC curve corresponding to the test values x and y (i.e., slope(x-y)). Three relations are observed, as follows (figure 1). First, the tangent at a point x on the ROC curve, i.e., tangent(x), corresponds to the likelihood ratio for a single test value corresponding to that point on the ROC curve for a continuous test, i.e., LR(x). The tangent at a point on an ROC curve represents the instantaneous change in the TPR per unit change in the FPR, and therefore it corresponds to the likelihood ratio at that particular point, which is estimable only for tests that produce results on a continuous scale. Second, the slope between the origin and the point on the ROC curve corresponding to a criterion x, i.e., Am J Epidemiol Vol. 148, No. 11, 1998