Invexity Preserving Transformations for Projection Free Optimization with Sparsity Inducing Non-convex Constraints

Abstract

Forward stagewise and Frank Wolfe are popular gradient based projection free optimization algorithms which both require convex constraints. We propose a method to extend the applicability of these algorithms to problems of the form (Formula Presented) where f(x) is an invex (Invexity is a generalization of convexity and ensures that all local optima are also global optima.) objective function and g(x) is a non-convex constraint. We provide a theorem which defines a class of monotone component-wise transformation functions (Formula Presented). These transformations lead to a convex constraint function (Formula Presented). Assuming invexity of the original function f(x) that same transformation (Formula Presented) will lead to a transformed objective function (Formula Presented) which is also invex. For algorithms that rely on a non-zero gradient (Formula Presented) to produce new update steps invexity ensures that these algorithms will move forward as long as a descent direction exists.