Full throttle: Demonstrating the speed, accuracy, and validity of a new method for continuous two-dimensional self-report and annotation

Abstract

Research on fine-grained dynamic psychological processes has increasingly come to rely on continuous self-report measures. Recent studies have extended continuous self-report methods to simultaneously collecting ratings on two dimensions of an experience. For all the variety of approaches, several limitations are inherent to most of them. First, current methods are primarily suited for bipolar, as opposed to unipolar, constructs. Second, respondents report on two dimensions using one hand, which may produce method driven error, including spurious relationships between the two dimensions. Third, two-dimensional reports have primarily been validated for consistency between reporters, rather than the predictive validity of idiosyncratic responses. In a series of tasks, the study reported here addressed these limitations by comparing a previously used method to a newly developed two-handed method, and by explicitly testing the validity of continuous two-dimensional responses. Results show that our new method is easier to use, faster, more accurate, with reduced method-driven dependence between the two dimensions, and preferred by participants. The validity of two-dimensional responding was also demonstrated in comparison to one-dimensional reporting, and in relation to post hoc ratings. Together, these findings suggest that our two-handed method for two-dimensional continuous ratings is a powerful and reliable tool for future research.