Optimasi Hyperparameter Ensemble Learning untuk Prediksi Perkembangan Penyakit Diabetes dengan Explainable AI

Abstract

This research focuses on optimizing and assessing ensemble learning models for predicting diabetes progression by combining hyperparameter tuning and explainable artificial intelligence techniques. Experiments were conducted using the scikit-learn diabetes dataset, which contains 442 samples with ten numerical features representing patients’ clinical conditions. The data were split into 80% for training and 20% for testing. Two ensemble methods were explored: Random Forest Regressor (bagging) and XGBoost Regressor (boosting). Hyperparameter optimization was carried out using RandomizedSearchCV and BayesianSearchCV under a five-fold cross-validation scheme. Model performance was evaluated using MAE, MSE, RMSE, and R² metrics, while interpretability was examined through SHAP summary plots. The results indicate that BayesianSearchCV consistently delivered superior performance gains compared to random search. In particular, the optimized XGBoost model achieved an R² score of 0.5018, improving by 19.8% over the baseline model (R² = 0.4188), and reduced RMSE from 55.49 to 51.37. SHAP analysis showed that serum triglycerides, body mass index, and blood pressure were the most influential features. Overall, the findings suggest that Bayesian-based hyperparameter optimization can effectively improve ensemble regression performance in medical prediction tasks involving limited datasets.