From rules to forests: rule-based versus statistical models for jobseeker profiling

Abstract

Public employment services (PES) commonly apply profiling models to target labor market programs to jobseekers at risk of becoming long-term unemployed. Such allocation systems often codify institutional experiences in a set of profiling rules, whose predictive ability, however, is seldomly tested. We systematically compare the predictive performance of a rule-based profiling procedure currently used by the PES in Catalonia, Spain, with the performance of statistical models in predicting future long-term unemployment (LTU) spells. Using comprehensive administrative data, we develop logit and machine learning models and evaluate their performance with respect to both model discrimination and calibration. Compared to the rule-based model used in Catalonia, our machine learning models achieve greater discrimination ability and remarkable improvements in calibration. Particularly, our random forest model is able to accurately forecast LTU spells and outperforms the rule-based model by offering robust predictions that perform well under stress tests. This paper presents the first performance comparison between a complex, currently implemented, rule-based approach and complex statistical profiling models. Our work illustrates the importance of assessing the calibration of profiling models and the potential of statistical tools to assist public employment services.