When the Mean is Misleading: a Guide to Ordered Regression for Meaningfully Modeling Ordinal Outcomes

Abstract

Objectives: We demonstrate the utility of ordered regression models for analyzing ordinal outcomes frequently encountered in criminology, comparing their predictive accuracy and estimation against conventional linear models. Methods: The study employs simulation to illustrate how linear and ordered probit models represent ordinal data distributions. Subsequently, it analyzes national survey data (N = 1,150) on racial differences in fear of police from Pickett et al. (2022). Linear models (item-specific, multilevel, normed-scale) are compared to Bayesian cumulative probit models using cross-validation, visualizations (mosaic plots; predicted probabilities), and a comprehensive R tutorial. Results: Simulations confirm linear models poorly represent non-normal ordinal distributions, while ordered probit models accurately recover observed patterns. Cross-validation reveals dramatically superior predictive performance for ordinal models (e.g., ΔELPD ≈ 1,920, SE = 48.6). While both approaches yield convergent average effect estimates (d = 0.98 vs. d = 0.97), they generate substantially different category-specific predictions. Ordered probit models estimate 32.8% of Black participants report being “very afraid” of police killing compared to 7.6% of White participants, while linear models predict only 11.0% and 1.1%, respectively. Observed proportions were 32.1% and 8.6%. Conclusions: Ordered regression models provide superior predictive accuracy and precise quantification of distributional patterns while yielding equivalent average effects when linear models perform adequately. This demonstrates that methodological choice affects substantive understanding of policy-relevant phenomena. Although the paper limits its focus to cumulative probit models and a single substantive domain, the tutorial, supplementary material, and review of relevant literature are meant to encourage practical adoption and elaboration of ordinal methods in criminology.