A new cache replacement algorithm in SMO

Abstract

In the methods for training Support Vector Machines (SVM), precomputed elements in the Hessian matrix are usually cached in order to avoid recomputation. However, the least-recent-used replacement algorithm that is widely used is not suitable since the elements are requested in an irregular way. A new cache replacement algorithm applied in Sequential Minimal Optimization (SMO) is presented in the paper. The item corresponding to the component with minimal violation of the Karush-Kuhn-Tucher (KKT) condition is deleted to make room for new one when the cache is full. It is shown in the experiments that the hit ratio of the cache is improved compared with LRU cache while the training time can be reduced in the tasks where the computation of elements in Hessian matrix is very time-consuming.