Scene-specific convolutional neural networks for video-based biodiversity detection

Abstract

Finding, counting and identifying animals is a central challenge in ecology. Most studies are limited by the time and cost of fieldwork by human observers. To increase the spatial and temporal breadth of sampling, ecologists are adopting passive image-based monitoring approaches. While passive monitoring can expand data collection, a remaining obstacle is finding the small proportion of images containing ecological objects among the majority of frames containing only background scenes. I proposed a scene-specific convolutional neural network for detecting animals of interest within long duration time-lapse videos. Convolutional neural networks are a type of deep learning algorithm that have recently made significant advances in image classification. The approach was tested on videos of floral visitation by hummingbirds. Despite low frame rates, poor image quality, and complex video conditions, the model correctly classified over 90% of frames containing hummingbirds. Combining motion detection and image classification can substantially reduce the time investment in scoring images from passive monitoring studies. These results underscore the promise of deep learning to lead ecology into greater automation using passive image analysis. To help facilitate future applications, I created a desktop executable that can be used to apply pre-trained models to videos, as well as reproducible scripts for training new models on local and cloud environments.