The Challenge of Prediction in Information Systems Research

Abstract

Empirical research in Information Systems (IS) is dominated by the use of explanatory statistical models for testing causal hypotheses, and by a focus on explanatory power. Predictive statistical models, which are aimed at predicting out-of-sample observations with high accuracy, are rare, and so is attention to predictive power. The distinction between explanatory and predictive statistical models is key, as both types of models play a different, yet essential, role in advancing scientific research. Similarly, explanatory power and predictive accuracy are two distinct qualities of a statistical model, and are measured in different ways. A literature review of MISQ and ISR shows that predictive goals, predictive claims, and predictive statistical models are scarce in mainstream empirical IS research. In addition, we find three questionable common practices: First, even when the stated goal of modeling is predictive, explanatory statistical modeling is often employed. Second, the predictive power of a model is often inferred from its explanatory power. And third, the vast majority of explanatory statistical models lack proper predictive assessment, which is a key scientific requirement. In light of the distinction between explanatory and predictive statistical modeling and power, and current practice in IS, we highlight the main differences between them, focusing on practical issues that confront an empirical researcher in the data analysis process.