Green AI Practices in Multi-objective Hyperparameter Optimization for Sustainable Machine Learning

Abstract

The hyperparameter tuning process is an essential step for ML model optimization, as it is necessary to improve model performance. However, this enhancement involves high computational resources and time costs. Model tuning can significantly raise energy consumption and consequently increase carbon emissions. Therefore, there is an essential need to construct a new framework for this challenge by adding carbon emissions as a vital consideration along with performance. The paper proposes a novel Sustainable Hyperparameter Optimization (SHPO) framework that uses an optimized multi-objective fitness approach. The framework focuses on ensemble classification models (ECMs) namely, Random Forest, ExtraTrees, XGBoost, and AdaBoost. All these models will be optimized using traditional and advanced techniques like Optuna, Hyperopt, and Grid Search. The proposed framework tracks carbon emissions during model hyperparameter tuning. The methodology uses the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) as a method of multi-criteria decision-making (MCDM). This TOPSIS method ranks the hyperparameter sets based on both accuracy and carbon emissions. The objective of the multi-objective fitness approach is to reach the best parameter set with high accuracy and low carbon emissions. It is observed from the experimental results that Optuna based Hyperparameter optimization consistently produced low carbon emissions and achieved high predictive accuracy across the majority of benchmark hyperparameter setups for the models.