Semi-supervised semantic segmentation with multi-reliability and multi-level feature augmentation

Abstract

Semi-supervised semantic segmentation aims to classify the pixels with both labeled and unlabeled images. How to utilize unlabeled images is a key part of semi-supervised learning. The existing methods tend to exploit the reliable pixels which can be classified to a specific category with high reliability, ignoring the unreliable pixels which cannot be identified into a special category with low reliability, resulting in information loss. Moreover, uneven number of pixels per category often leads to misclassification of small-scale category pixels. In this paper, we propose a multi-reliability and multi-level feature augmentation framework. Multi-reliability feature augmentation dedicates to extract unreliable/reliable pixel features to take full advantage of pixel information. Multi-level feature augmentation dedicates to extract category/image level semantic features to avoid the sparse semantic information provided by category with a small number of pixels, which can lead to misclassification. Furthermore, a variety of data perturbations are employed to enhance the robustness of our model. Experimental results on the PASCAL VOC 2012, Cityscapes datasets verify the effectiveness of our method. Our code is available at https://github.com/JianJianYin/MMFA.