Decision criteria do not shift: Reply to Treisman

Abstract

Recently, we argued that the detection criterion representation of decision-making biases, embedded within the theory of signal detection, is empirically testable and has, in fact, been falsified by empirical results from visual discrimination experiments. Treisman (2002) attempts to show that there is an alternative interpretation of our results that could explain them without dropping the detection criterion construct. In lieu of attempting to fit the data with a model, however, he gives two kinds of theoretical examples, both involving manipulations of the spacing of criteria on a decision axis. The first example correctly predicts that the bias estimate we developed will be zero but does so by assuming zero spacing between some criteria (some rating responses are never used). We did not observe zero spacing between any criteria and did not perform any analyses on responses that never occurred. Moreover, this example does not explain why the upper-bound bias estimates that we obtained by combining results from two criteria placements were also trivially small. His second example predicts that the bias should have been detectable with sufficiently large sample sizes. In our experiments, the sample sizes were, in fact, quite large, large enough for the results to be consistent in 18 different experimental conditions. Finally, all of Treisman's criteriaplacement examples also fail to explain the pronounced effects of base rates on the shapes of the rating ROC curves, and his suggestion that there are problems of logical interpretation with our proposed distribution model ignores the predictions of large classes of alternatives to detection theory, including the dynamic models of perception.