Part-level sketch segmentation and labeling using dual-CNN

Abstract

Part-level sketch segmentation and labeling refers to segment an object sketch to semantic component parts. It is a hard task since sketches carry much fewer features than natural images. Inspired by the neural networks used in sketch classification, which shows the performance of the network is significantly affected by the kernel size, we propose a dual-convolutional neural network (CNN) method to tackle automatic sketch segmentation and labeling. The dual-CNN model contains two CNNs, one with large-size convolutional kernels to process long sketches, the other with small-size kernels to work on short ones. Both CNNs have three convolutional layers and three fully connection layers. Except for the first convolutional layer, the rest configurations of these two CNNs are same. To further enhance the performance of the method, we model position and orientation as a triple-channel input of our networks by fusing the minimal oriented rectangle bounding boxes (MORBB) of stroke and its host sketch as masks. Extensive experimental results verify our method and demonstrate that our approach outperforms state of the art.