AI-based validation of wastewater treatment plant sensor data using an open data exchange architecture

Abstract

Typically, within the context of treatment plant-wide data, the quality of data can be impacted by sensor faults, sensor calibration issues, fouling of and obstruction to the sensors and connectivity problems between sensors, actuators and the data management system, therefore hampering advanced data driven monitoring and control of (critical) water operations. Here, a smart data validation scheme is proposed that validates sensor data from a wastewater treatment plant and is tightly integrated with the open-source data exchange system called FIWARE, an EU supported framework. The data validation application and FIWARE setup are integrated, tested and deployed at the water utility, Waternet. The validation scheme is based on an anomaly detector using (statistical) threshold techniques and a data reconciliation part that aggregates deep learning based autoencoder model predictions whenever an anomaly is detected. The autoencoder models proved to have a high accuracy and good reconciliation performance considering the variability of the signal. Furthermore, (near) real-time validated and raw data signals are relayed towards a dashboard. Finally, the validated data can be used as a screening for data ingested by another AI-based model that enables monitoring and smart control of the wastewater treatment plant in order to minimise greenhouse gas emissions and energy consumption while meeting effluent water quality standards.