Self-concordant analysis for logistic regression

Abstract

Most of the non-asymptotic theoretical work in regression is carried out for the square loss, where estimators can be obtained through closed-form expressions. In this paper, we use and extend tools from the convex optimization literature, namely self-concordant functions, to provide simple extensions of theoretical results for the square loss to the logistic loss. We apply the extension techniques to logistic regression with regularization by the l2-norm and regularization by the l1-norm, showing that new results for binary classification through logistic regression can be easily derived from corresponding results for least-squares regression. © 2010, Institute of Mathematical Statistics. All rights reserved.