Deep Twin Support Vector Networks

Abstract

Twin support vector machine (TSVM) is a successful improvement for traditional support vector machine (SVM) for binary classification. However, it is still a shallow model and has many limitations on prediction performance and computational efficiency. In this paper, we propose deep twin support vector networks (DTSVN) which could enhance its performances in all aspects. Specifically, we put forward two version of DTSVN, for binary classification and multi classification, respectively. DTSVN improves the abilities of feature extraction and classification performance with neural networks instead of a manually selected kernel function. Besides, in order to break the bottleneck that the original model cannot directly handle multi classification tasks, multiclass deep twin support vector networks (MDTSVN) is further raised, which could avoid the inefficient one-vs-rest or one-vs-one strategy. In the numerical experiments, our proposed DTSVN and MDTSVN are compared with the other four methods on MNIST, FASHION MNIST and CIFAR10 datasets. The results demonstrate that our DTSVN achieves the best prediction accuracy for the binary problem, and our MDTSVN significantly outperforms other existing shallow and deep methods for the multi classification problem.