Study on data cleansing algorithms for outliers in water supply system

Abstract

There has been systematically sharing data between government agencies, local governments and public organizations and more public organizations are encouraged to open public data in Korea. In the water industry, research on artificial intelligence using big data, a technology related to the fourth industrial revolution, is currently being carried out actively. As a result, quality control of acquired data is necessary to secure the reliability of data by developing algorithms of data cleansing to minimize outliers. In this paper, LSTM (Long Short Term Memory) for cleansing outliers and missing data is proposed to improve data quality management.