Using coarse label constraint for fine-grained visual classification

Abstract

Recognizing fine-grained categories (e.g., dog species) relies on part localization and fine-grained feature learning. However, these classification methods use fine labels and ignore the structural information between different classes. In contrast, we take into account the structural information and use it to improve fine-grained visual classification performance. In this paper, we propose a novel coarse label representation and the corresponding cost function. The new coarse label representation idea comes from the category representation in the multi-label classification. This kind of coarse label representation can well express the structural information embedded in the class hierarchy, and the coarse labels are only obtained from suffix names of different category names, or given in advance like CIFAR100 dataset. A new cost function is proposed to guide the fine label convergence with the constraint of coarse labels, so we can make full use of this kind of coarse label supervised information to improve fine-grained visual classification. Our method can be generalized to any fine-tuning task; it does not increase the size of the original model; and adds no overhead to the training time. We conduct comprehensive experiments and show that using coarse label constraint improves major fine-grained classification datasets.