A Comparative Study of L1 and L2 Norms in Support Vector Data Descriptions

Abstract

The Support Vector Data Description (L1 SVDD) is a non-parametric one-class classification algorithm that utilizes the L1 norm in its objective function. An alternative formulation of SVDD, called L2 SVDD, uses a L2 norm in its objective function and has not been extensively studied. L1 SVDD and L2 SVDD are formulated as distinct quadratic programming (QP) problems and can be solved with a QP-solver. The L2 SVDD and L1 SVDD’s ability to detect small and large shifts in data generated from multivariate normal, multivariate t, and multivariate Laplace distributions is evaluated. Similar comparisons are made using real-world datasets taken from various applications including oncology, activity recognition, marine biology, and agriculture. In both the simulated and real-world examples, L2 SVDD and L1 SVDD perform similarly, though, in some cases, one outperforms the other. We propose an extension of the SMO algorithm for L2 SVDD, and we compare the runtimes of four algorithms: L2 SVDD (SMO), L2 SVDD (QP), L1 SVDD (SMO), and L1 SVDD (QP). The runtimes favor L1 SVDD (QP) versus L2 SVDD (QP), sometimes substantially; however using SMO reduces the difference in runtimes considerably, making L2 SVDD (SMO) feasible for practical applications. We also present gradient descent and stochastic gradient descent algorithms for linear versions of both the L1 SVDD and L2 SVDD. Examples using simulated and real-world data show that both methods perform similarly. Finally, we apply the L1 SVDD and L2 SVDD to a real-world dataset that involves monitoring machine failures in a manufacturing process.