Titanic Survival Prediction with Enhanced Random Forests

Abstract

This paper proposes an enhanced random forest framework designed to address key challenges in the Titanic survival prediction task, including class imbalance, feature heterogeneity, and limited sample size. The method integrates an entropy-based adaptive feature weighting mechanism to amplify the influence of critical socio-demographic features—such as gender and passenger class—during decision tree splits, thereby improving split reliability and model interpretability. To mitigate bias arising from the underrepresentation of survivors (minority class), SMOTE is employed to synthetically balance the training data. Furthermore, Bayesian optimization is utilized for efficient and robust hyperparameter tuning, enhancing generalization performance. Extensive experiments on the Kaggle Titanic dataset demonstrate that the proposed approach consistently outperforms a range of baselines—including logistic regression, SVM, standard random forests, XGBoost, and MLP—in terms of accuracy, recall, F1-score, and AUC. Ablation studies confirm the complementary contributions of each component, while error analysis reveals systematic misclassifications in specific subgroups (e.g., male third-class passengers), offering insights into model behavior and limitations. The framework not only achieves superior predictive performance but also improves fairness and stability, presenting a principled and extensible solution for classification tasks on small, imbalanced, and heterogeneous tabular datasets.