Bayesian historical borrowing with longitudinal large-scale assessments

Abstract

The purpose of this paper is to extend and evaluate methods of Bayesian historical borrowing applied to longitudinal data with a focus on parameter recovery and predictive performance. Bayesian historical borrowing allows researchers to utilize information from previous data sources and to adjust the extent of borrowing based on the similarity of current data to historical data. We examine the utility of three static historical borrowing methods including complete pooling, Bayesian synthesis with aggregated data-dependent priors, traditional power priors, and two dynamic borrowing methods including Bayesian dynamic borrowing and commensurate priors. Using data from two administrations of the United States Early Childhood Longitudinal Study, we evaluate these methods in terms of in-sample simulation statistics, as well as pseudo out-of-sample measures of predictive performance. A case study examining growth in reading competency over time revealed that for one historical cycle, most methods of historical borrowing perform similarly with respect to predictive performance and parameter recovery. Pooling and power priors performed relatively poorly across the conditions in this study, particularly when the current data and the historical data were heterogeneous. Results from a comprehensive simulation study revealed that the advantages of different historical borrowing methods vary across different evaluation criteria. Overall, Bayesian dynamic borrowing and commensurate priors are no worse, and in some cases better, than other methods in terms of parameter recovery and predictive performance, and considering a previous paper by Kaplan et al. (Psychometrika, 10.1007/s11336-022-09869-3, 2022) found clear benefits of Bayesian dynamic borrowing over other methods of historical borrowing in the multilevel context using data from the Program for International Student Assessment (PISA) with five historical cycles, this paper argues that Bayesian dynamic borrowing or commensurate priors is a prudent choice for borrowing information from previous cycles of longitudinal data.