Context-aware encoding for clothing parsing

Abstract

Clothing parsing is a special type of semantic segmentation in which each pixel is assigned with clothing labels. Unlike general scene semantic segmentation, stylish match (e.g. skirts + blouse, jeans + T-shirt) is an important cue for recognising fine-grained categories in clothing parsing. In this Letter, the authors propose a context-aware outfit encoder (COE), as a side branch, that drives the convolutional neural network to take the stylish match into account for clothing parsing. The proposed COE provides information on matching clothes that can be utilised to improve the prediction accuracy of the base network significantly. Experimental results show that fully convolutional network and MobileNet with the COE improve the mean intersection of the union of those without the COE by 2.5 and 2.8%, respectively, on CFPD dataset.