Fish Species Identification Using a CNN-based Multimodal Learning Method

Abstract

In this study, we propose an advanced fish species identification method that uses fish images and their meristic characters. Conventionally, fish species are only identified using feature values obtained from images. Because fish of the same species can have different colors or look very similar to other species, it is difficult to identify fish species based only on an image. We developed a multi-input model that features images and, additionally, trait data, aimed at becoming an advanced fish species identification system. We constructed a learning model using fish images and their meristic characters obtained by web scraping and compared its accuracy with a case that only used images. As a result, it was clarified that using two or more kinds of meristic characters provided higher accuracy than using only images.