Comparative Analysis of ESG Indicators in Enhancing Debt Default Prediction with Transformer-derived Attention Models

Abstract

This study explores the application of Transformer-derived algorithms, namely TabNet and Tab Transformer, in predicting corporate debt defaults and compares their performance with classical machine learning models. Concurrently, it assesses the efficacy of incorporating ESG indicators into the corporate debt default risk assessment framework. The research focuses on Chinese A-share listed companies from January 2018 to June 2023, comprising 23 companies with observed defaults and a control group of 846 companies with regular debt maturities. The findings indicate that although attention-based models like TabNet and Tab Transformer provide enhanced interpretability, their performance does not significantly surpass ensemble algorithms such as XGBoost. Attention-based models emphasize the importance of merging the advantages of deep learning with the interpretability of traditional algorithms, especially when dealing with vast, high-dimensional datasets. Additionally, the incorporation of ESG data did not yield a significant improvement in prediction outcomes. Potential reasons for the limited impact of ESG indicators on predictions, including data quality and the comprehensiveness of existing financial disclosures, are discussed. Given the limited sample size and constraints related to test data, future research directions suggest expanding the dataset and diversifying ESG data sources.