Time Series Forecasting With Volatility Activation Function

Abstract

Time series forecasting is the method of predicting future values of a model by reviewing its past data. Various models like traditional approaches, statistical methods, moving average, ARIMA, RNN's, or XGBoost may also be applied. Activation functions are the primary most important hyper parameters or artificial neural networks that decide whether a neuron will be active or not. However, the most widely used sigmoid and ReLU activation functions include the problems of linearity, gradient protection, vanishing gradient and data convergence. Solution of these problems is significant for the development of activation functions which are one of the most important hyper parameters for artificial neural networks. A new activation function is suggested to generate a solution for the problems existing in the activation functions within the scope of this study. A hybrid model has been created from the RNN and LSTM algorithms and applied on different time series data in order to implement this suggested activation function called as the volatility activation function. The volatility activation function has been compared with the literature studies, the conditions for being a function have been proved and its applicability has been demonstrated by means of financial instrument and electrical transformer temperature data. As a result of the study, the characteristics of the volatility activation function have been presented; additionally its applicability, validity and proof has been performed. Furthermore, it has been proved that the problems found in the activation functions used in the literature are not existing in the volatility activation function. It has been verified that the volatility activation function is functional on time series. To demonstrate the feasibility of the volatility activation function, it has been applied to three different time series problems. As a result of the study conducted on the electrical transformer temperature estimation, MSE = 0.362 and MAE = 0.448 were obtained; namely these results are similar to the literature studies. In the test results with ounce gold data, the accuracy rate increased by 0,1. In the test results with Australian rain forecast data, the accuracy rate increased by 0,2. As a result, a new activation function is suggested for the deep learning.