A deep feature fusion based method for bird sound recognition and its interpretability analysis

Abstract

Background: Bird sound recognition is a crucial tool for ecological monitoring. However, current research still faces the challenges of achieving low recognition rates in complex datasets and a lack of robustness. Moreover, there is a noticeable absence of interpretability analysis for deep learning model in the existing research. Methods: Firstly, we utilized a deep feature extraction network to extract features from the l ogarithmic Mel spectrogram of bird sound and the deep features o f the supplementary feature set. These two types of deep features were then fused and fed into a light gradient boosting machine (lightGBM) classifier for classification. Class activation maps were applied to perform interpretability analysis on deep learn ing models to understand how the models recognize bird sound. Results: The experimental results demonstrate d that the proposed method in this paper achieve d state of the art results on the Beijing Bird Dataset, with an average accuracy of 98.70% and an ave rage F 1 score of 98.84%. Compared to traditional methods, the deep fusion features show a significant improvement in accuracy for bird sound recognition, with an increase of at least 5.62%. Additionally, the introduction of the lightGBM classifier contribu te d to a 3.02% improvement in classification accuracy. Furthermore, the proposed method exhibit ed outstanding performance on the CLO 43SD and BirdCLEF2022 competition datasets, achieving average accuracies of 98.32% and 91.12%, respectively. The result of the class activation maps revealed that the disparities in attentional regions within the neural network for each specific bird sound type. Conclusion: The method proposed in this paper effectively improves the accuracy of bird sound recognition and demons trates excellent performance on three datasets, offering strong technical support for ecological monitoring based on bird sound recognition. This analysis serves as a theoretical foundation for subsequent endeavors in feature selection and model optimizati on.