Real-time deployment of artificial neural network forecasting models: Understanding the range of applicability

Abstract

When an operational artificial neural network (ANN) model is deployed, new input patterns are collected in order to make real-time forecasts. However, ANNs (like other empirical and statistical methods) are unable to reliably extrapolate beyond the calibration range. Consequently, when deployed in real-time operation there is a need to determine if new input patterns are representative of the data used in calibrating the model. To address this problem, a novel detection system for identifying uncharacteristic data patterns is presented. This approach combines a self-organizing map (SOM), to partition the data set, with nonparametric kernel density estimators to calculate local density estimates (LDE). The SOM-LDE method determines the degree to which a new input pattern can be considered to be contained within the domain of the calibration set. If a new pattern is found to be uncharacteristic, a warning can be issued with the forecast, and the ANN model retrained to include the new pattern. This approach of selectively retraining the model is compared to no retraining and the more computationally onerous case of retraining the model after each new sample. These three approaches are applied to forecast flow in the Kentucky River, USA, using multilayer perceptron (MLP) models. The results demonstrate that there is a significant advantage in retraining an ANN that has been deployed as a real-time, operational model, and that the SOM-LDE classifier is an effective approach for identifying the model's range of applicability and assessing the usefulness of the forecast.