A Necessary Condition for the Strong Oracle Property

Abstract

A problem of using a non-convex penalty for sparse regression is that there are multiple local minima of the penalized sum of squared residuals, and it is not known which one is a good estimator. The aim of this paper is to give a guide to design a non-convex penalty that has the strong oracle property. Here, the strong oracle property means that the oracle estimator is the unique local minimum of the objective function. We summarize three definitions of the oracle property - the global, weak and strong oracle properties. Then, we give sufficient conditions for the weak oracle property, which means that the oracle estimator becomes a local minimum. We give an example of non-convex penalties that possess the weak oracle property but not the strong oracle property. Finally, we give a necessary condition for the strong oracle property.