On The Algorithmic Implementation of Multiclass Kernel-based Vector Machines

Abstract

In this paper we describe the algorithmic implementation of multiclass kernel-based vector machines. Our starting point is a generalized notion of the marginto multiclass problems. Using this notion we cast multiclass categorizationproblems as a constrained optimization problem with a quadratic objectivefunction. Unlike most of previous approaches which typically decompose amulticlass problem into multiple independent binary classication tasks, ournotion of margin yields a direct method for training multiclass predictors. Byusing the dual of the optimization problem we are able to incorporate kernelswith a compact set of constraints and decompose the dual problem into multipleoptimization problems of reduced size. We describe an ecient xed-point algorithmfor solving the reduced optimization problems and prove its convergence. We thendiscuss technical details that yield signicant running time improvements forlarge datasets. Finally, we describe various experiments with our approachcomparing it to previously studied kernel-based methods. Our experimentsindicate that for multiclass problems we attain state-of-the-art accuracy.