From Prediction to Prevention: Identifying Actionable Crash Factors Through ML and Narrative-Based Sensitivity Testing

Abstract

Crashes on roadways continue to represent a major global public health concern due to high rates of death and injury, underscoring the need for predictive tools that can identify high-risk conditions and guide prevention strategies. This study develops a framework that combines structured crash records and road information with unstructured police narratives to predict injury severity using machine learning and natural language processing (NLP). The dataset is used to train, validate, and test nine models, combining three algorithms (Random Forest, AdaBoost, and XGBoost) with two NLP methods (TF-IDF and Word2Vec). Model performance is evaluated using macro-average F1-scores to address severe class imbalance. Results show that XGBoost with TF-IDF achieves the best performance (macro-F1 = 0.644), demonstrating measurable improvements from incorporating narrative features compared to structured data alone. Beyond prediction, a simulation-based sensitivity analysis is conducted on the top 100 features, identifying 11 variables with the greatest impact on severity outcomes in Kentucky. Seatbelt non-use, occupant entrapment, and impaired driver control emerge as the most influential factors, with simulated improvements leading to notable reductions in fatalities and major injuries. The study introduces a “prediction-to-prevention” framework that links injury severity prediction with simulation-based sensitivity analysis. By integrating structured and narrative crash data, the framework identifies how changes in key behavioral and roadway factors can shift injury outcomes toward less severe levels. These findings highlight the dual contribution of this study: improving predictive accuracy through narrative integration and offering actionable insights to support evidence-based traffic safety interventions.