Krishnendu Biswas

Customer exit prediction: 8-classifier benchmark on 90,000 rows

Benchmarked 8 classifiers on 90,000 customer records from a financial institution. Highlighted why accuracy (0.8414) is flattering while F1 (0.6365) measures true minority churn capture.

Winning Model
Tuned Random Forest (RandomizedSearchCV)
Validation Metrics
F1 0.6365 · Accuracy 0.8414 · Recall 0.6561
Dataset Scale
90,000 Training Records · 10 Predictors
Missing Value Policy
IterativeImputer (Numerical) · Mode Imputation (Categorical)

Context and Objective

Predicting which customers of a financial institution will leave is hard because few of them do: the majority of customers remain active, so a naive baseline predicting zero churn achieves high accuracy while capturing none of the business risk. We benchmarked models using validation F1 and recall on the exit class.

Data Pipeline and Model Benchmarking

The dataset contained missing values across credit score, country, account balance, and product count. Numerical fields were imputed using an IterativeImputer (multivariate feature imputation), while categorical variables received mode imputation.

I compared 8 classification algorithms on validation F1 (AdaBoost: 0.4604 F1 is the eighth):

Hyperparameter Tuning and Final Outcome

The top 3 models were tuned with RandomizedSearchCV optimizing for F1. The tuned Random Forest achieved the highest validation F1 score of 0.6365 with an accuracy of 0.8414 and recall of 0.6561 (best CV F1 0.6456). Tuned LightGBM reached 0.6334 and tuned XGBoost reached 0.6309.

Customer exit prediction: classifier benchmark comparison

Validation F1 comparison of 8 classifiers on 90,000 records Horizontal bar chart showing validation F1 scores across untuned baselines and tuned models. AdaBoost, 0.4604 F1, is the lowest baseline and is not drawn as a bar. Tuned Random Forest 0.6365 F1 Tuned LightGBM 0.6334 F1 Tuned XGBoost 0.6309 F1 Baseline XGBoost 0.6277 F1 Baseline LightGBM 0.6277 F1 Gradient Boosting 0.6006 F1 Random Forest 0.5796 F1 Extra Trees 0.5736 F1 Logistic Regression 0.5509 F1 Decision Tree 0.5033 F1 0.0 Validation F1 Score 0.70 Validation F1 of 10 models on 90,000 records. AdaBoost, 0.4604 F1, is the lowest and is not drawn as a bar. Tuned Random Forest 0.6365 Tuned LightGBM 0.6334 Tuned XGBoost 0.6309 Baseline XGBoost 0.6277 Baseline LightGBM 0.6277 Gradient Boosting 0.6006 Random Forest 0.5796 Extra Trees 0.5736 Logistic Regression 0.5509 Decision Tree 0.5033

Validation F1, 7 untuned baselines plus 3 tuned models, 90,000 rows.

Stack

Python · scikit-learn · XGBoost · LightGBM · pandas · matplotlib