Efficient quality assurance and quality control for passive acoustic monitoring data: reducing and documenting false-positive and false-negative errors

Abstract

Autonomous Recording Units (ARUs) are widely used to survey for a variety of taxa. This survey method allows for high spatial and temporal coverage but will typically include identification errors that can bias estimates of occupancy. In some instances, verifying all individual detections is prohibitive. To direct verification effort, we developed a model to estimate the probability that transcribers would agree on an identification. Agreement probability was positively influenced by transcriber skill, identification confidence, species commonness and some song types. In contrast, agreement probability was lower when an acoustic signal was classified as a trill. We evaluated our model on independent data where all species detections were verified, and verification effort (time) was quantified. Our model performed well at predicting transcriber agreement on independent data (AUC = 0.71). We applied the model to randomised subsets of the independent data to compare the cost benefit of three approaches to verification under varying effort. We show how modelling probability of transcriber agreement can be used to more efficiently direct verification of species acoustic tags. Our approach could be adapted elsewhere to quantify and reduce species misidentifications in unverified passive acoustic monitoring data for either manual processing or detections from automated classifiers.