Investigating XGBoost Efficiency on Diverse Time-Series Data Through PSO Parameter Tuning

Abstract

This study investigates the effectiveness of Particle Swarm Optimization (PSO) in enhancing the performance of the XGBoost algorithm on three distinct types of time-series datasets: intermittent (Dataset-1), multi-seasonal (Dataset-2), and non-stationary (Dataset-3). Time-series data often exhibit complex patterns, making hyperparameter tuning crucial for improving model accuracy. In this research, two PSO variants, PSOv1 and PSOv2, were applied to optimize key hyperparameters, specifically the learning rate (LR) and Max Depth (MxD), in order to minimize the Root Mean Square Error (RMSE). Both PSOv1 and PSOv2 demonstrated significant improvements over the baseline XGBoost model, with PSOv2 showing faster and more stable convergence across all datasets. Notably, the PSO algorithms were particularly effective for multi-seasonal and non-stationary time-series, where the convergence rate was faster compared to intermittent data. These results highlight PSO’s ability to fine-tune hyperparameters for better generalization, reducing overfitting and improving performance on unseen data. This work underscores the importance of swarm intelligence algorithms in optimizing machine learning models for complex time-series forecasting tasks, paving the way for future research in hybrid optimization techniques and the exploration of additional hyperparameters.