Bayesian-Optimized Explainable AI for CKD Risk Stratification: A Dual-Validated Framework

Abstract

Chronic kidney disease (CKD) impacts more than 850 million people globally, yet existing machine learning methodologies for risk stratification encounter substantial challenges: computationally intensive hyperparameter tuning, model opacity that conflicts with clinical interpretability standards, and class imbalance leading to systematic prediction bias. We constructed an integrated architecture that combines XGBoost with Optuna-driven Bayesian optimization, evaluated against 19 competing hyperparameter tuning approaches and tested on CKD patients using dual-paradigm statistical validation. The architecture delivered 93.43% accuracy, 93.13% F1-score, and 97.59% ROC-AUC—representing gains of 6.22 percentage points beyond conventional XGBoost and 7.0–26.8 percentage points compared to 20 baseline algorithms. Tree-structured Parzen Estimator optimization necessitated merely 50 trials compared to 540 for grid search and 1069 for FLAML, whereas Boruta feature selection accomplished 54.2% dimensionality reduction with no performance compromise. Over 30 independent replications, the model exhibited remarkable stability (cross-validation standard deviation: 0.0121, generalization gap: −1.13%) alongside convergent evidence between frequentist and Bayesian paradigms (all p < 0.001, mean CI-credible interval divergence < 0.001, effect sizes d = 0.665–5.433). Four separate explainability techniques (SHAP, LIME, accumulated local effects, Eli5) consistently identified CKD stage and albumin-creatinine ratio as principal predictors, aligning with KDIGO clinical guidelines. Clinical utility evaluation demonstrated 98.4% positive case detection at 50% screening threshold alongside near-optimal calibration (mean absolute error: 0.138), while structural equation modeling revealed hyperuricemia (β = −3.19, p < 0.01) as the most potent modifiable risk factor. This dual-validated architecture demonstrates that streamlined hyperparameter optimization combined with convergent multi-method interpretability enables precise CKD risk stratification with clinical guideline alignment, supporting evidence-informed screening protocols.