Comparative Study of Machine Learning Classification Models on Loan Approval Prediction

Abstract

As people’s demands for loans with various purposes have increased drastically over the years, the process for banks to determine whether applications should be approved or not has become increasingly time-consuming and complicated. Therefore, models that can automatically provide banks with initial decisions for reference will significantly improve the efficiency of the process. Reviewing the results provided by the model may also offer an insight into potential biases in their decision-making procedures. The study will focus on determining better models based on evaluations and further manipulations of 6 machine learning methods: Logistic Regression, Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Decision Tree, Random Forest, XGBoost, and Light-GBM. After comparisons among different classifiers established through different combinations of methods, such as oversampling and grid search, the study finally identifies two models that are most balanced on multiple metrics: the XGBoost model that undergoes oversampling and grid search, and a voting classifier, i.e., an ensemble of XGBoost, Random Forest, and Light-GBM models. In the comparison, it is found that, particularly when the original data set is imbalanced, the oversampling leads the models to make fewer conservative decisions on whether a loan application should be approved by decreasing the precision score while increasing the recall score.