Everything under control: comparing machine learning and classical econometric impact assessment methods using FADN data

Abstract

Machine learning (ML) methods have been proposed to improve the assessment of agricultural policies through enhanced causal inference. This study uses a simulation framework tailored to Farm Accountancy Data Network (FADN) data to scrutinize the performance of both ML and classical methods under diverse causal properties crucial for identification. Our findings reveal significant variations in performance across different treatment assignment rules, sample sizes and causal properties. Notably, the Causal Forest method consistently outperforms others in retrieving the causal effect and accurately characterizing its heterogeneity. However, the data-driven approach of ML methods proves ineffective in selecting the correct set of controls and addressing latent confounding.