Big data storage and parallel analysis of grid equipment monitoring system

Abstract

With the analysis on data feature of grid equipment operation monitoring, this work focuses on discussing the big data storage scheme for grid equipment online monitoring data, and describes optimization measure of grid monitoring data analysis. Based on the characteristics of large data scale, multiple data types and low value density with the online monitoring data, we provide a big data storage scheme based on HDFS cloud platform using consistent hashing. Meanwhile, we also employ a multi-channel data acquisition system using multiscale multivariate entropy as the feature extraction algorithm of the multi-source power grid monitoring data. To validate the efficiency of the algorithm, we perform experiments using power grid equipment ledger data, chromatographic hydrocarbons data of transformer oil, microclimate data, and transformer vibration data for association analysis. The big data storage scheme and the feature extraction algorithm proved that it could reduce the communication overhead between storage nodes, efficiently improve system performance, and is suitable for the actual application of power grid monitoring system.