Testing the stability of utility parameters in repeated best, repeated best-worst and one-off best-worst studies

Abstract

Stated choice (SC) surveys are a key tool for studying travel behaviour and are used to inform policy decisions in many countries. Recently, the best-worst (BW) variant of SC has rapidly increased in popularity in fields as diverse as transport, marketing and health research. A key argument for its implementation has been that it is perceived to be easier for respondents to identify the best and the worst alternative in a choice set compared to identifying the second- or even third best. For elicitation formats asking respondents to consecutively identify the first, second and third best (etc.) alternative, labelled here as repeated best stages, it is well known that utility and scale parameters are generally not stable across the stages. Joint analysis of the responses to each stage may increase the efficiency of the utility and willingness-to-pay parameters (i.e. smaller standard errors), but incorrect inferences may be made if these parameters are not stable across the stages. This paper tests the stability of utility parameters for the repeated BW and one-off BW format. Using data from three different studies, we show that, regardless of the dataset and elicitation format used, the obtained utility parameters and willingness to pay estimates are not stable across stages. The results thereby question the use of BW data in applied work aimed at forecasting and understanding first (best) choices. Our findings thereby contradict recent discussions about potentially beneficial framing effects in BW surveys. The unique presence of corresponding data from a repeated best and repeated BW exercise in one survey highlights the observed rank-orders are highly consistent across the two elicitation formats and that any differences in marginal willingness to pay estimates can be attributed to the imposed econometric model rather than to differences in the behaviour of respondents.