Deep knowledge distillation: A self-mutual learning framework for traffic prediction

Abstract

Traffic flow prediction in spatio-temporal networks is a crucial aspect of Intelligent Transportation Systems (ITS). Existing traffic flow forecasting methods, particularly those utilizing graph neural networks, encounter limitations. When processing large-scale graph data, the depth of these models can restrict their ability to effectively capture complex relationships and patterns. Additionally, these methods often focus mainly on local neighborhood information, which can limit their capability to recognize and analyze global relationships and patterns within the graph data. Therefore, we proposed a deep knowledge distillation model, tailored to effectively capture spatio-temporal patterns in traffic flow prediction. This model incorporates a bidirectional random walk process on a directed graph, enabling it to effectively capture both spatial and temporal dependencies. Utilizing a blend of mutual learning and self-distillation, our approach enhances the detection of spatio-temporal relationships within traffic data and improves the feature perception ability at both local and global levels. We tested our model on two real-world datasets, achieving notable improvements in prediction accuracy, especially for predictions within a one-hour timeframe. In comparison to the baseline model, our proposed model achieved accuracy improvements of 0.19 and 0.18 on the respective datasets. These results highlight the success of using mutual learning and self-distillation to transfer knowledge effectively within and between models and to improve the model's capability in identifying and extracting features.