Humans outperform Merlin Sound ID in field-based point-count surveys

Abstract

Point counts are integral to bird population monitoring, and multiple methods have been developed to improve their reliability. These methods include multiple-observer protocols, double-sampling, and surveys using autonomous recording units. However, these improved protocols are time-consuming, expensive, and often require hiring surveyors with inconsistent abilities. Merlin Sound ID, an automated bird-sound classifier that can run in the field on mobile devices, could be a part of the solution to the problems associated with point counts by functioning as a no-cost observer with consistent performance. Here, we compare the accuracy of Merlin and human observers in 144 paired field-based point-counts. Our goals were to evaluate how the species identified, number of individual detections, and precision for humans and Merlin compare. We also examined the consistency of Merlin across different devices. Human observers reported 382 individual detections over the course of the study, 72% more than the 222 detections reported by Merlin. Precision was similar with 92% and 86% for humans and Merlin, respectively. Merlin detected 4 species missed by humans but also detected 12 false positive species. Merlin disagreed with itself on 57% of point counts when running on 2 different devices at the same time. We propose Merlin not as a replacement for human point counters, but as an aid to increase detection probability of point-count surveys as secondary observers, with humans acting as reviewers and arbitrating disagreements between devices when needed. With further research into this methodology, technology like Merlin could help address some of the challenges associated with point-count-based research.