Beyond validation: Alternative uses and associated assessments of goodness for computational social models

Abstract

This discussion challenges classic notions of validation, suggesting that 'validity' is not just an attribute of a model. It is a function of the relationship of a particular characteristic of the model (the probability that the model will produce a data set that will match a set it has not yet seen) and a user. If we shift our focus from the model to the user, we can identify other ways in which models can be of use as well as start clarifying how we can identify their goodness in multiple use scenarios. In this discussion, we distinguish validation from calibration, and research or generalizable models from site-specific models. We use literature from fields as varied as physics, systems ecology and computational linguistics to characterize the process of validation. Finally, by extending the use space for models beyond prediction, we introduce the possibility of assessments of goodness other than validation. © 2012 Springer-Verlag.