What’s That Frog? Evaluating a Crowdsourced Approach to Species Identification From Audio Recordings

Abstract

Citizen science is rapidly advancing as a field, engaging the public in scientific research and outcomes, while obtaining large quantities of data. Within the natural sciences, identification of organisms by citizen scientists is an important and increasing source of biodiversity records. However, to ensure the reliability and quality of citizen science data, including species identifications, various strategies have been employed. Crowdsourcing is commonly used to increase confidence in data quality, where results are an aggregate of multiple responses. Projects utilising crowdsourcing techniques have focused on image-based datasets, and are taxonomically biased towards large and charismatic taxa. We explored the utility of crowdsourcing species identification from audio files of calling frogs. We developed a series of audio analysis trials, inviting members of the public to identify frog species from audio recordings submitted to the FrogID project. By comparing the identifications made by citizen scientists with those made by experts in frog identification, we evaluated the suitability of using crowdsourcing to identify frog species calling in 20–60-second audio files. Citizen scientists identified frog species with 57% accuracy, although this varied depending on species, task complexity, and the individual volunteer. By aggregating responses, the accuracy of citizen scientist responses increased to up to 66% when we required a consensus of two volunteers from five responses. Our results indicate that crowdsourcing the identification of multiple frog species from audio recordings is challenging, requiring further exploration into the benefits of training volunteers, providing detailed feedback, and weighting volunteer responses according to individual skill.