A Simple Convolutional Transfer Neural Networks in Vision Tasks

Abstract

Convolutional neural networks (ConvNets) is multi-stages trainable architecture that can learn invariant features in many vision tasks. Real-world applications of ConvNets are always limited by strong requirements of expensive and time-consuming labels generating in each specified task, so the challenges can be summarized as that labeled data is scarce while unlabeled data is abundant. The traditional ConvNets does not consider any information hidden in the large-scale unlabeled data. In this work, a very simple convolutional transfer neural networks (CTNN) has been proposed to address the challenges by introducing the idea of unsupervised transfer learning to ConvNets. We propose our model with LeNet5, one of the simplest model in ConvNets, where an efficient unsupervised reconstruction based pre-training strategy has been introduced to kernel training from both labeled and unlabeled data, or from both training and testing data. The contribution of the proposed model is that it can fully use all the data, including training and testing simultaneously, thus the performances can be improved when the labeled training data is insufficient. Widely used hand-written dataset MNIST, together with two retinal vessel datasets, DRIVE and STARE, are employed to validate the proposed work. The classification experiments results have demonstrated that the proposed CTNN is able to reduce the requirement of sufficient labeled training samples in real-world applications.