Stochastic conjugate gradient descent twin support vector machine for large scale pattern classification

Abstract

With the advent of technology, the amount of data available for learning is increasing day by day. However, machine learning algorithms such as Support Vector Machines (SVMs) are effective but slow in dealing with this huge inflow of information. Recent researches have largely focussed on increasing the scalability of machine learning algorithms including by using algorithmic level speed-ups such as TWSVM [10], LS-SVM [18] and training level speed-ups such as using Newton-Armijo method [12], Coordinate Descent Method [8] etc. Among these, recently proposed stochastic gradient based methods have attracted significant attention. However, these methods suffer from the inherent problems of stochastic gradient methodology such as ill-conditioning, slow convergence near minima etc. In this paper, we propose a Stochastic Conjugate Gradient Descent method based Twin Support Vector Machine (SCG-TWSVM) which improves upon the limitations of Stochastic Gradient Descent Support Vector Machine (SG-SVM) and Stochastic Gradient Twin Support Vector Machine (SG-TWSVM) and leads to a more robust, effective and generalizable classifier. We also extend our proposed classifier to non-linear case by using Kernel trick. We perform extensive experiments on a variety of machine learning benchmark datasets as well as real-world machine learning datasets which prove the efficacy of our proposed approach compared to related methods on large scale problems.