Let F be a set of M classification procedures with values in [-1,1]. Given a loss function, we want to construct a procedure which mimics at the best possible rate the best procedure in F. This fastest rate is called optimal rate of aggregation. Considering a continuous scale of loss functions with various types of convexity, we prove that optimal rates of aggregation can be either ((log M)/n)1/2 or (log M)/n. We prove that, if all the M classifiers are binary, the (penalized) Empirical Risk Minimization procedures are suboptimal (even under the margin/low noise condition) when the loss function is somewhat more than convex, whereas, in that case, aggregation procedures with exponential weights achieve the optimal rate of aggregation. © Springer-Verlag Berlin Heidelberg 2007.
CITATION STYLE
Lecué, G. (2007). Suboptimality of penalized empirical risk minimization in classification. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 4539 LNAI, pp. 142–156). Springer Verlag. https://doi.org/10.1007/978-3-540-72927-3_12
Mendeley helps you to discover research relevant for your work.