Improving support vector machines performance using local search

Abstract

In this paper, we propose a method for optimization of the parameters of a Support Vector Machine which is more accurate than the usually applied grid search method. The method is based on Iterated Local Search, a classic metaheuristic that performs multiple local searches in different parts of the space domain. When the local search arrives at a local optimum, a perturbation step is performed to calculate the starting point of a new local search based on the previously found local optimum. In this way, exploration of the space domain is balanced against wasting time in areas that are not giving good results. We show a preliminary evaluation of our method on a radial-basis kernel and some sample data, showing that it is more accurate than an application of grid search on the same problem. The method is applicable to other kernels and future work should demonstrate to what extent our Iterated Local Search based method outperforms the standard grid search method over other heterogeneous datasets from different domains.