Short-Term Electrical Load Forecasting Based on Time Augmented Transformer

Abstract

Electrical load forecasting is of vital importance in intelligent power management and has been a hot spot in industrial Internet application field. Due to the complex patterns and dynamics of the data, accurate short-term load forecasting is still a challenging task. Currently, many tasks use deep neural networks for power load forecasting, and most use recurrent neural network as the basic architecture, including Long Short-Term Memory (LSTM), Sequence to Sequence (Seq2Seq), etc. However, the performance of these models is not as good as expected due to the gradient vanishing problem in recurrent neural network. Transformer is a deep learning model initially designed for natural language processing, it calculates input–output representations and captures long dependencies entirely on attention mechanisms which has great performance for capturing the complex dynamic nonlinear sequence dependence on long sequence input. In this work, we proposed a model Time Augmented Transformer (TAT) for short-term electrical load forecasting. A temporal augmented module in TAT is designed to learn the temporal relationships representation between the input history series to adapt to the short-term power load forecasting task. We evaluate our approach on a real-word dataset for electrical load and extensively compared it to the performance of the existed electrical load forecasting model including statistical approach, traditional machine learning and deep learning methods, the experimental results show that the proposed TAT model results in higher precision and accuracy in short-term load forecasting.