Large Scale Fish Images Classification and Localization using Transfer Learning and Localization Aware CNN Architecture

Abstract

Building an automatic fish recognition and detection system for large-scale fish classes is helpful for marine researchers and marine scientists because there are large numbers of fish species. However, it is quite difficult to build such systems owing to the lack of data imbalance problems and large number of classes. To solve these issues, we propose a transfer learning-based technique in which we use Efficient-Net, which is pre-trained on ImageNet dataset and fine-tuned on QuT Fish Database, which is a large scale dataset. Furthermore, prior to the activation layer, we use Global Average Pooling (GAP) instead of dense layer with the aim of averaging the results of predictions along with having more information compared to the dense layer. To check the validity of our model, we validate our model on the validation set which achieves satisfactory results. Also, for the localization task, we propose an architecture that consists of localization aware block, which captures localization information for better prediction and residual connections to handle the over-fitting problem. Actually, the residual connections help the layer to combine missing information with the relevant one. In addition, we use class weights and Focal Loss (FL) to handle class imbalance problems along with reducing false predictions. Actually, class weights assign less weights to classes having fewer instances and large weights to classes having more number of instances. During the localization, the qualitative assessment shows that we achieve 57% Mean Intersection Over Union (IoU) on testing data, and the classification results show 75% precision, 70% recall, 78% accuracy and 74% F1-Score for 468 fish species.