Data anomaly detection with automatic feature selection and deep learning

Abstract

As increasing sensor-based structural health monitoring (SHM) systems are implemented on civil infrastructures, sensor data reliability plays a crucial role in the assessment of operational performance of bridges. Sensors are heavily exposed to harsh environmental conditions during their operations and inevitably lead to possible unstable performance or failure. Thus, to accurately identify faulty sensors is a prerequisite to processing and analyzing the collected data for assessment purpose. Recently, researchers adopted the convolutional neural network (CNN) approach to identify faulty sensors, focusing on image features. Such approach may overlook some important detailed signal features and the time series approach may still be needed. However, algorithms based on time series tend to be time consuming because of the lengthy and high dimensional dataset. This may be effectively resolved using an automatic feature selection technique, namely Tsfresh, as proposed in this paper to select highly relevant signal features based on statistical tests of significance. A deep learning technique based on fully convolutional network (FCN) can then be efficiently employed for anomaly classification. The algorithm is validated using a dataset collected from a real cable-stayed bridge and results show that the proposed method significantly reduces the training time for the neural network, albeit with high classification accuracy.