A semi-permeable attention network for ESG score prediction

Abstract

Environmental, Social, and Governance (ESG) metrics have become critical indicators of corporate sustainability, ethical behavior, and long-term financial performance. However, accurately predicting ESG scores remains challenging due to the tabular nature of ESG datasets and their small size, which often limits the effectiveness of traditional deep learning approaches. In this study, we propose an attention-based deep learning model specifically designed for tabular ESG data. Our model leverages the semi-permeable attention mechanism in the ExcelFormer architecture to selectively regulate feature interactions based on their predictive importance. This design enables the model to mitigate noise from less informative features while preserving critical dependencies within structured data. We evaluate our method on a simulated dataset comprising ESG and financial performance data from 1, 000 companies across multiple industries and regions. The proposed model consistently outperforms traditional machine learning models and state-of-the-art tabular deep learning models, achieving the lowest errors across all ESG dimensions. Specifically, it attains a coefficient of determination of 0.7373 for overall ESG prediction, with a mean squared error of 0.0063. These results demonstrate the potential of attention-augmented tabular neural networks in advancing ESG forecasting, offering meaningful contributions to the field of sustainable finance.