Banking Study Combines ML, SHAP and LLMs
- •Study tests four models on 45,211 UCI Bank Marketing observations for customer response prediction
- •Random Forest posts 90.0% accuracy and 91.8% ROC-AUC; XGBoost leads recall at 78.0%
- •SHAP explanations and LLM-generated insights form a decision workflow requiring institution-specific validation
Researchers Md Abu Sufian Mozumder, Md. Salim Chowdhury and colleagues developed a predictive business intelligence framework for predicting banking customers’ responses. Their study used 45,211 observations from the UCI Bank Marketing dataset and compared Logistic Regression, Decision Tree, Random Forest and XGBoost using accuracy, precision, recall, F1-score and ROC-AUC. Random Forest had the strongest overall performance, with 90.0% accuracy, 56.0% precision, 53.0% recall, 55.0% F1-score and 91.8% ROC-AUC. XGBoost recorded the highest recall, 78.0%, alongside 86.0% accuracy, 45.0% precision, 57.0% F1-score and 91.4% ROC-AUC. Logistic Regression reached 82.0% accuracy and 90.3% ROC-AUC; Decision Tree reached 87.0% accuracy and 67.3% ROC-AUC.
The framework combines machine learning, explainable AI and large language models. SHAP (a method for attributing predictions to input factors) identifies factors contributing to model predictions, while an LLM turns structured predictions and explanations into natural-language business insights. The architecture links predictive modeling, model explainability, business intelligence visualization and managerial interpretation in one decision-support workflow. The researchers say Random Forest offers balanced performance on the benchmark task, while XGBoost is more sensitive to potential positive cases.
The study describes the framework as a research and deployment architecture, not evidence of performance in U.S. banking. Practical use requires validation for each institution: the benchmark data come from a Portuguese banking campaign, and some variables may introduce temporal leakage, where information from a later time affects prediction.