Temporal dependence and the sensitivity of quantities of interest: A solution for a common problem

Abstract

Scholars of international relations increasingly use temporal dependence variables (polynomials or splines) to control for unmodeled duration dependence in nonlinear models (such as logit or probit) of events ranging from interstate conflict and civil war to sanctions imposition and trade agreements. I identify two inferential obstacles that are widespread to nonlinear models, and are exacerbated by the unique features of temporal dependence variables. First, compression causes the quantities of interest to be sensitive to the values in the counterfactual scenario (most notably, time). Second, presenting substantive effects calculated at one simulation scenario (such as an "average" scenario) grossly inflates the representativeness of that scenario and neglects the variability within the sample. The consequences of these problems range in severity from understating the magnitude of the substantive effects to deriving inferences that are wholly unrepresentative of the data. I offer a simple checklist. First, use the values observed in the data to generate in-sample quantities of interest. Second, plot those quantities of interest across the offending variable (for example, time) and interpret the relationship. Finally, provide a sense of the sample's variability in quantities of interest through simple summary statistics (such as mean, standard deviation, and range). These simple fixes provide much-needed transparency and act as a shield against scholars who might otherwise present misleading results.