A machine learning assisted method for rapidly annotating benthic megafauna in large volumes of marine imagery

Abstract

Recent technological advancements have rapidly expanded our capacity for collecting image data in the marine environment, but processing images into meaningful ecological metrics remains a manual, time-consuming, and biased process. This is particularly challenging with electro-optical cabled imaging systems which generate images at a rate that makes manual identification impractical. To address this challenge, we have developed a machine learning-assisted method for annotating images. Our approach leverages a pre-trained model based on marine-specific imagery from the FathomNet database (499 classes; some classes to species level). We demonstrated the application of this method on a 1-yr time series of images collected at Southern Hydrate Ridge by a digital still camera on the NSF Ocean Observatories Initiative Regional Cabled Array, which resulted in 92,153 benthic megafaunal (organisms > 2 cm) annotations across 10 morphotaxa classes in 50,840 images. This method annotated the full dataset in 6 weeks, compared to an estimated ~ 5.9 yr required for fully manual annotation, representing approximately a 50-fold increase in efficiency. This process also produced a computer vision model with a precision of 0.75, recall of 0.80, mAP50 of 0.84, and mAP50-95 of 0.65. Our method combines machine learning efficiency with human expertise to create high-quality, verified datasets. The output of this methodology is key to achieving the full potential of sustained ecosystem monitoring via cabled observing systems that can capture both short-term and long-term ecological and environmental dynamics.