Convergence of alternating optimization.

Abstract

Summary: Let f: {\germ R}^s\mapsto{\germ R} be a real-valued function,and let x= (x_1,\dots, x_s)^T\in{\germ R}^s be partitioned intot subsets of non-overlapping variables as x= (X_1,\dots, X_t)^T,with X_i\in{\germ R}^{p_i} for i= 1,\dots, t, \sum^t_{i=1} p_i=s. Alternating optimization (AO) is an iterative procedure for minimizingf(x)= f(X_1,X_2,\dots, X_t) jointly over all variables by alternatingrestricted minimizations over the individual subsets of variablesX_1,\dots, X_t. Alternating optimization has been (more or less)studied and used in a wide variety of areas. Here a self-containedand general convergence theory is presented that is applicable toall partitionings of x. Under reasonable assumptions, the generalAO approach is shown to be locally, q-linearly convergent, andto also exhibit a type of global convergence.