Methods and designs

Abstract

Cognitive training research faces a number of methodological challenges. Some of these are general to evaluation studies of behavioral interventions, like selection effects that confound the comparison of treatment and control groups with preexisting differences in participants’ characteristics. Some challenges are also specific to cognitive training research, like the difficulty to tell improvements in general cognitive abilities from improvements in rather task-specific skills. Here, an overview of most important challenges is provided along an established typology of different kinds of validity (statistical conclusion, internal, external, and construct validity) that serve as central criteria for evaluating intervention studies. Besides standard approaches to ensure validity, like using randomized assignment to experimental conditions, emphasis is put on design elements that can help to raise the construct validity of the treatment (like adding active control groups) and of the outcome measures (like using latent factors based on measurement models). These considerations regarding study design are complemented with an overview of data-analytical approaches based on structural equation modeling, which have a number of advantages in comparison to the still predominant approaches based on analysis of variance.