Exploring Category Attention for Open Set Domain Adaptation

Abstract

Great success has been achieved in the area of unsupervised domain adaptation, which learns to generalize from the labeled source domain to the unlabeled target domain. However, most of the existing techniques can only handle the closed-set scenario, which requires both the source domain and the target domain to have a shared category label set. In this work, we propose a two-stage method to deal with the more challenging task of open set domain adaptation, where the target domain contains categories unseen to the source domain. Our first stage formulates the alignment of two domains as a semi-supervised clustering problem, and initially associates each target-domain sample x^{t}\in \mathcal {X}^{t} with a source-domain category label \ell ^{s} \in \mathcal {L}^{s}. To this end, we use the self-training strategy to learn a teacher network and a student network, both of which adopt the self-attention mechanism. Our second stage refines the resulting clusters by identifying the negative associations (x^{t}, \ell ^{s}) and labeling the involved x^{t} as unknown. For this purpose, we investigate the compatibility of each association by replacing the self-attention maps in the last convolutional layers with the newly proposed category attention maps (CAMs), which locate the informative feature pixels for a given category. Experimental results on three public datasets show the effectiveness and robustness our method in adaptation across various domain pairs.