A survey of recent advances in forecast accuracy comparison testing, with an extension to stochastic dominance

Abstract

In recent years, an impressive body of research on predictive accuracy testing and model comparison has been published in the econometrics discipline. Key contributions to this literature include the paper by Diebold and Mariano (J Bus Econ Stat 13:253-263,1995) which sets the groundwork for much of the subsequent work in the area, West (Econometrica 64:1067-1084, 1996) who considers a variant of the DM test that allows for parameter estimation error in certain contexts, and White (Econometrica 68:1097-1126,2000) who develops testing methodology suitable for comparing many models. In this chapter, we begin by reviewing various key testing results in the extant literature, both under vanishing and non-vanishing parameter estimation error, with focus on the construction of valid bootstrap critical values in the case of non-vanishing parameter estimation error, under recursive estimation schemes, drawing on Corradi and Swanson (Int Econ Rev 48:67-109, 2007a). We then review recent extensions to the evaluation of multiple confidence intervals and predictive densities, for both the case of a known conditional distribution Corradi and Swanson (J Econ 135:187-228,2006a; Handbook of economic forecasting Elsevier, Amsterdam, pp 197-284) and of an unknown conditional distribution. Finally, we introduce a novel approach in which forecast combinations are evaluated via the examination of the quantiles of the expected loss distribution. More precisely, we compare models looking at cumulative distribution functions (CDFs) of prediction errors, for a given loss function, via the principle of stochastic dominance, and we choose the model whose CDF is stochastically dominated, over some given range of interest.