Multi-label Pattern Image Retrieval via Attention Mechanism Driven Graph Convolutional Network

Abstract

Pattern images are artificially designed images which are discriminative in aspects of elements, styles, arrangements and so on. Pattern images are widely used in fields like textile, clothing, art, fashion and graphic design. With the growth of image numbers, pattern image retrieval has great potential in commercial applications and industrial production. However, most of existing content-based image retrieval works mainly focus on describing simple attributes with clear conceptual boundaries, which are not suitable for pattern image retrieval. It is difficult to accurately represent and retrieve pattern images which include complex details and multiple elements. Therefore, in this paper, we collect a new pattern image dataset with multiple labels per image for the pattern image retrieval task. To extract discriminative semantic features of multi-label pattern images and construct high-level topology relationships between features, we further propose an Attention Mechanism Driven Graph Convolutional Network (AMD-GCN). Different layers of the multi-semantic attention module activate regions of interest corresponding to multiple labels, respectively. By embedding the learned labels from attention module into the graph convolutional network, which can capture the dependency of labels on the graph manifold, the AMD-GCN builds an end-to-end framework to extract high-level semantic features with label semantics and inner relationships for retrieval. Experiments on the pattern image dataset show that the proposed method highlights the relevant semantic regions of multiple labels, and achieves higher accuracy than state-of-the-art image retrieval methods.