Multi-Teacher Knowledge Distillation Framework for Lightweight Deep Learning-Based State-of-Health Estimation

Abstract

While deep learning-based approaches for state of health (SOH) estimation in lithium-ion batteries have been actively studied, most models face deployment constraints in on-device applications due to their high complexity and large number of parameters. Although previous studies have introduced knowledge distillation (KD) for model compression, single-teacher architectures exhibit limited performance improvement due to insufficient knowledge diversity. To resolve this issue, this study proposes a multi-teacher knowledge distillation (MTKD) framework to simultaneously achieve efficient SOH estimation and model compression. From raw charging data, a total of 18 health indicators (HIs) were obtained from diverse perspectives, including temporal information, statistical features, equivalent circuit model (ECM) parameters, and incremental calculation. Key features were selected through Pearson correlation analysis and the maximal information coefficient (MIC), and were utilized as inputs for the deep learning models. Subsequently, large-scale teacher models based on deep neural network (DNN), long short-term memory (LSTM), and one-dimensional convolution neural network (1D CNN) architectures were trained to capture various degradation characteristics, including nonlinear relationships, temporal dependencies, and local patterns. The lightweight student model was then trained using soft targets obtained from the teacher models along with ground truth labels. Experimental results demonstrate that the student model trained with the proposed MTKD achieved a 45.98% reduction in root mean square error (RMSE) and a 15.72% improvement in coefficient of determination (R2) compared to single-teacher KD (STKD). This study successfully extends KD research beyond traditional computer vision and image processing domains, demonstrating practical applicability in battery data-driven applications.