A Framework to Quality Control Oceanographic Data

Abstract

The ocean is an intrinsically challenging environment to collect data, which makes spurious measurements inevitable. Thus, the quality of oceanographic datasets is highly dependent on the ability to identify and remove bad samples. Quality control (QC) of oceanographic data has mostly relied on manual QC by experts, which, despite resulting in the best data quality, is not scalable and becomes impractical to handle large datasets or real-time data streams. To address this issue, automatic QC procedures have been proposed and widely used for decades (e.g., IOC/IODE, 1993; DATA–MEQ working group, 2010; GTSPP Real–Time Quality Control Manual, 2010; Morello et al., 2014; QARTOD group, 2016; Wong, Keeley, Carval, & Argo Data Management Team, 2015); however, these procedures are seldom organized and distributed as packages, so it is still common for new users to have to implement them from scratch. Additionally, different applications of the same dataset may require different QC procedures. For example, a particular user faced with the QC of a small dataset might be willing to apply a less conservative QC in order to preserve a larger number of data points, paying the price of having some false positives. CoTeDe is an Open Source Python package that provides a flexible way to automatic QC oceanographic data by combining multiple QC standards while allowing the users to fully control and tune the parameters according to their own needs.