Procedures for Statistical Validation of Stochastic Simulation Models

Abstract

Proper validation of a stochastic simulation model requires that the predictions of the model be compared with real world data that are independent of the data that were used in the construction of the model. Valid comparisons of real data and model output require an understanding of the nature of the validation problem plus the availability of statistical procedures that are designed to fit the conditions of the problem. In many cases, decisions about the validity of a model are made by a cursory examination of the predicted values or by statistical procedures that may not be appropriate for the problem. This paper develops a framework for testing the validity of models in situations that are frequently encountered in renewable natural resource problems. The suitability of several common statistical tests is discussed and several additional parametric and nonparametric procedures are proposed for the validation problem. The proposed procedures are applied to the validation of a stochastic forest stand simulator using data from 63 loblolly pine plantation plots. The results of the analysis indicated that there were previously undetected deficiencies in the simulator's ability to predict certain aspects of the real system. Forest Sci. 27:349-364.