A simulation-based method to determine model misspecification: Examples using natural mortality and population dynamics models

Abstract

Recent developments in the models used in wildlife and fisheries science have allowed the inclusion of a wider range of data than previously. However, the diagnostics of such complex models have not kept pace. We describe a new diagnostic technique based on simulation analysis. Model misspecification was identified through simulation methods that created a distribution of likely parameter values for a model that was correctly specified. If the actual estimate of that parameter is outside the bounds of the simulated distribution, then the model is probably misspecified. We tested the reliability of the new diagnostic by introducing known-model misspecification into complex fisheries stock assessment models. We then compared the results from this new diagnostic with those of a more tradition fisheries diagnostic, namely, retrospective analysis. The simulation-based diagnostic was shown to identify misspecification affecting the estimated dynamics more reliably than retrospective analysis. © American Fisheries Society 2011.