A Study of a Prediction Framework Incorporating Random Forests and Gradient Boosting Trees

Abstract

This study constructs a hybrid prediction framework integrating Random Forest regression and gradient boosting tree. Firstly, the study uses Random Forest regression to mine the nonlinear associations between multidimensional features through self-sampling and feature random selection mechanisms to achieve the classification and regression tasks, and builds in feature importance analysis to assess feature contributions. Second, logistic regression and linear regression are used to provide statistical explanations for binary classification problems and multivariate interactions, respectively, to enhance the interpretability of the model. In addition, this study constructs a gradient boosting tree model and combines it with SHAP value analysis, iterative fitting of residuals to improve prediction accuracy, and hyperparameter tuning to further optimise model performance. This framework enhances the analysis and prediction ability of multivariate system data through multi-model collaboration, demonstrates good stability and generalisation ability, and provides a reusable technical paradigm for similar studies.