Comparing DIC and WAIC for multilevel models with missing data

Abstract

In Bayesian statistics, the most widely used criteria of Bayesian model assessment and comparison are Deviance Information Criterion (DIC) and Watanabe–Akaike Information Criterion (WAIC). We use a multilevel mediation model as an illustrative example to compare different types of DIC and WAIC. More specifically, we aim to compare the performance of conditional and marginal DICs and WAICs, and investigate their performance with missing data. We focus on two versions of DIC (DIC1 and DIC2) and one version of WAIC. In addition, we explore whether it is necessary to include the nuisance models of incomplete exogenous variables in likelihood. Based on the simulation results, whether DIC2 is better than DIC1 and WAIC and whether we should include the nuisance models of exogenous variables in likelihood functions depend on whether we use marginal or conditional likelihoods. Overall, we find that the marginal likelihood based-DIC2 that excludes the likelihood of covariate models generally had the highest true model selection rates.