Early fusion of multi-level saliency descriptors for image classification

Abstract

In this paper, we propose a method that improves the classification of images. Considering saliency maps as if they were topographic maps and filtering the characteristics of the image’s background, the Bag of Visual Words (BoVW) coding is improved. First, we evaluated six known algorithms to generate saliency maps and we selected GBVS and SIM because they are the ones that retain most of the information of the object. Next, we eliminated the extracted SIFT descriptors belonging to the background by filtering features based on binary images obtained at various levels of the selected saliency maps. We filtered the descriptors by obtaining layers at various levels of the saliency maps, and we evaluated the early fusion of the SIFT descriptors contained in these layers into five different datasets. The results obtained indicate that the proposed method always improves the reference method when combining the first two layers of GBVS or SIM and the dataset contains images with a single object.