Forced-choice staircases with fixed step sizes: asymptotic and small-sample properties

  • García-Pérez M
N/ACitations
Citations of this article
1Readers
Mendeley users who have this article in their library.
Get full text

Abstract

Departamento de Metodología, Facultad de Psicología, Universidad Complutense, Campus de Somosaguas, 28223 Madrid, Spain Visual detection and discrimination thresholds are often measured using adaptive staircases, and most studies use transformed (or weighted) up/down methods with fixed step sizes—in the spirit of Wetherill and Levitt (Br J Mathemat Statist Psychol 1965;18:1–10) or Kaernbach (Percept Psychophys 1991;49:227–229)—instead of changing step size at each trial in accordance with best-placement rules—in the spirit of Watson and Pelli (Percept Psychophys 1983;47:87–91). It is generally assumed that a fixed-step-size (FSS) staircase converges on the stimulus level at which a correct response occurs with the probabilities derived by Wetherill and Levitt or Kaernbach, but this has never been proved rigorously. This work used simulation techniques to determine the asymptotic and small-sample convergence of FSS staircases as a function of such parameters as the up/down rule, the size of the steps up or down, the starting stimulus level, or the spread of the psychometric function. The results showed that the asymptotic convergence of FSS staircases depends much more on the sizes of the steps than it does on the up/down rule. Yet, if the size Δ+ of a step up differs from the size Δ− of a step down in a way that the ratio Δ−/Δ+ is constant at a specific value that changes with up/down rule, then convergence percent-correct is unaffected by the absolute sizes of the steps. For use with the popular one-, two-, three- and four-down/one-up rules, these ratios must respectively be set at 0.2845, 0.5488, 0.7393 and 0.8415, rendering staircases that converge on the 77.85%-, 80.35%-, 83.15%- and 85.84%-correct points. Wetherill and Levitt's transformed up/down rules—which require Δ−/Δ+=1—and the general version of Kaernbach's weighted up/down rule—which allows any Δ−/Δ+ ratio—fail to reach their presumed targets. The small-sample study showed that, even with the optimal settings, short FSS staircases (up to 20 reversals in length) are subject to some bias, and their precision is less than reasonable, but their characteristics improve when the size Δ+ of a step up is larger than half the spread of the psychometric function. Practical recommendations are given for the design of efficient and trustworthy FSS staircases. Author Keywords: Forced choice; Adaptive staircases; Threshold estimates; Efficiency; Error; Simulation techniques The notion of efficient and criterion-free psychophysical procedures is intimately associated with 2AFC staircases. These are all variations of the up/down method designed by Dixon and Mood [14], and some of them implement sophisticated criteria for deciding the size of each step up or down. For instance, QUEST [90] is essentially a one-down/one-up method in which the size of each individual step is determined by the entire history of the staircase. Papers have recently been published that catalog and describe these and other psychophysical procedures [59, 82]. This paper is concerned with 2AFC staircases in which the sizes of the steps up and down are fixed, although they may be different from one another. An analysis of papers published in Vision Research and the Journal of the Optical Society of America A in 1994–1996 showed that psychophysicists prefer these fixed-step-size (FSS) staircases: out of 120 papers using 2AFC staircases, 82 (68%) used FSS staircases. Table 1 lists their characteristics, revealing that the description of staircases is often incomplete. In some papers, data are just said to have been obtained with “a standard forced-choice staircase”, indicating how secondary these procedures are considered. At the least, lack of specification of the details of a staircase hampers replicability. This will not be appreciated if one subscribes to the common belief that every staircase that implements a given up/down rule will always target the same percent-correct point, but this has never been proved rigorously. Wetherill and Levitt ([93]; see also Ref. [9]) applied some probabilistic considerations to speculating on the percent-correct points on which FSS staircases implementing some transformed up/down rules should converge, as did Kaernbach [32] for designing weighted up/down rules. The validity of these arguments is suspect, since the sizes of the steps relative to the spread of the psychometric function affect some staircase procedures more than others [25, 32]. Also, Stillman [77] used staircases that supposedly target the 71%- and 79%-correct points, but testing at the resulting thresholds showed that the stimuli were detected a significantly smaller percent of times. Then, the question of what percent-correct point each up/down rule targets is still unanswered, as is that of what factors affect it. Several studies have compared the small-sample bias, precision and efficiency of some FSS staircases for fixed number of trials (e.g. Refs. [24, 25, 32, 33, 73, 74]), but no work appears to have studied these characteristics in the more frequent case of fixed number of reversals. More important, these studies have all assumed Wetherill and Levitt's [93] or Kaernbach's [32] claims as to the percent-correct point on which up/down rules converge. If these claims prove wrong, apparent bias will occur as a result of convergence on a point other than the presumed one. The work described in this paper used simulation techniques to determine the asymptotic and small-sample properties of FSS staircases implementing some of Wetherill and Levitt's [93] transformed up/down rules, Kaernbach's [32] weighted up/down rule, and a natural extension of these that can be called transformed and weighted up/down rules (see Section 2). The conditions explored in the study are described in Section 3, and include the up/down rule, the sizes of the steps, and the starting point of the staircase. Asymptotic results are presented in Section 4, and can easily be summarized: in general, the percent-correct point targeted by an FSS staircase depends more on the sizes of the steps than on the up/down rule. Yet, if the size of a step down is a specific fraction of the size of a step up—a fraction that differs across rules—then the target is unaffected by the sizes of the steps. Section 5 presents results bearing on the small-sample characteristics of these optimal staircases, showing that they may fail to achieve their potential in the typical small-sample setting. Section 6 summarizes the results and gives recommendations for users of FSS staircases. Readers who are content with the practical consequences of this study may skip Section 4 and Section 5. The language of contrast detection tasks is used throughout this paper, but all conclusions must apply to any task where a 2AFC staircase is suitable. Along an FSS staircase, the stimulus level at any given trial depends on the subject's responses in one or more of the preceding trials. Let D be the set of events (sequences of responses over one or more trials) that trigger a step down, let U be the set of events that trigger a step up, and let Ψ be a monotonic increasing psychometric function. Then, Prob(D|x) and Prob(U|x), respectively, are the probabilities of a step down and a step up at stimulus level x, and there is some level x0 such that Prob(U|x0)=Prob(D|x0). Different procedures arise as a result of (i) how the sets D and U are defined and (ii) what the relative sizes of the steps up and down are. In the simplest case, every correct response (C) determines a step down of fixed size Δ and every wrong response (W) determines a step up of the same size. Thus, D={C}, U={W}, Prob(D|x)=Ψ(x) and Prob(U|x)=1−Ψ(x). Then, Ψ(x0)=1−Ψ(x0), so that Ψ(x0)=1/2. In other words, Prob(D|x)>Prob(U|x) if x>x0 and Prob(D|x)<Prob(U|x) if x<x0. This variant of the up/down method allows the sets D and U to include several sequences of responses over various numbers of consecutive trials, although the steps up and down continue to be identical in size. For some examples of how the sets D and U can be defined see Wetherill and Levitt ([93]; Table 1) or Brown ([9]; Table 1). In these circumstances, Prob(D|x) and Prob(U|x) must be derived as some transformation f of Ψ(x). Then Prob(D|x)=f(Ψ(x)) and Prob(U|x)=1−f(Ψ(x)), from where only Ψ(x0)=f−1(1/2) obtains. The only implication is again that Prob(D|x)>Prob(U|x) when x>x0 and Prob(D|x)<Prob(U|x) when x<x0. This is a different variant of the simple up/down method in which the sets D and U remain as described in Section 2.1, but the size of a step up, Δ+, differs from that of a step down, Δ−. Using the same notation as above, Kaernbach [32] claimed without apparent justification that1 A fourth alternative presents itself at this point, which results from combining non-unitary sets D and U (as in the transformed up/down methods) with unequal sizes for the steps up and down (as in the weighted up/down method). One might assume that the equation that applies here is The foregoing presentation of the four methods has made clear that the only true fact about them is that the specific stimulus level x0 satisfying Prob(U|x0)=Prob(D|x0) varies across methods, since Ψ(x0) must have different values for different methods (see Eq. 2 and Eq. 4 and similar equations in Section 2.1 and Section 2.2). For use with their up/down method, Dixon and Mood [14] derived maximum-likelihood estimators for x0 under several restrictive conditions, and they also proved that computationally simpler approximations work reasonably well under additional restrictions. Wetherill et al. [94] showed that still simpler estimators based on the average of the reversal values are just as good under the same conditions. Apparently, Wetherill and Levitt [93] and Kaernbach [32] simply assumed that the average of reversal values will also provide approximate estimates of x0 under their modifications of the simple up/down method. The general form of a psychometric function,

Cite

CITATION STYLE

APA

Garc&#xed;a-P&#xe9;rez, M. A. (1998). Forced-choice staircases with fixed step sizes: asymptotic and small-sample properties. Vision Research, 38(12), 1861–1881. https://doi.org/10.1016/S0042-6989(97)00340-4

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free