Promoting Cumulation in models of the human mind

Abstract

Lee et al. (2019) address a critical issue in cognitive science—defining scientific practices that will promote rigor and confidence in our science by ensuring that our mechanisms, models, and theories are adequately described and validated to facilitate replication and to foster trust. They provide a number of concrete suggestions to advance our science along that path. The recommendations emphasize preregistration of models and predictions combined with more comprehensive model evaluation, including published descriptions of exploratory analyses, alternative mechanisms, and model assumptions. These are excellent recommendations, and general adoption of such practices will benefit model assessment and validation methodologies in cognitive science research while improving trust in published reports of computational and mathematical accounts of cognitive phenomena. However, it is unclear that these strategies alone will resolve many other important challenges faced in developing quantitative theories of human cognition and behavior. For example, addressing the crisis of confidence will not, by itself, move the science toward the broader goal of developing more comprehensive and cumulative theories of the nature of the human mind. Cognitive modeling is a critical methodology for achieving that goal. However, to realize the potential will require changes not only to how we evaluate our models, but also to how we measure progress and scientific contribution.