Risks of drawing inferences about cognitive processes from model fits to individual versus average performance

Abstract

With the goal of drawing inferences about underlying processes from fits of theoretical models to cognitive data, we examined the tradeoff of risks of depending on model fits to individual performance versus risks of depending on fits to averaged data with respect to estimation of values of a model's parameters. Comparisons based on several models applied to experiments on recognition and categorization and to artificial, computer-generated data showed that results of using the two types of model fitting are strongly determined by two factors: model complexity and number of subjects. Reasonably accurate information about true parameter values was found only for model fits to individual performance and then only for some of the parameters of a complex model. Suggested guidelines are given for circumventing a variety of obstacles to successful recovery of useful estimates of a model's parameters from applications to cognitive data. Copyright 2005 Psychonomic Society, Inc.