Inverse problem approach to regularized regression models with application to predicting recovery after stroke

Abstract

Regression modelling is a powerful statistical tool often used in biomedical and clinical research. It could be formulated as an inverse problem that measures the discrepancy between the target outcome and the data produced by representation of the modelled predictors. This approach could simultaneously perform variable selection and coefficient estimation. We focus particularly on a linear regression issue, (Formula presented.), where (Formula presented.) is the parameter of interest and its components are the regression coefficients. The inverse problem finds an estimate for the parameter (Formula presented.), which is mapped by the linear operator (Formula presented.) to the observed outcome data (Formula presented.). This problem could be conveyed by finding a solution in the affine subspace (Formula presented.). However, in the presence of collinearity, high-dimensional data and high conditioning number of the related covariance matrix, the solution may not be unique, so the introduction of prior information to reduce the subset (Formula presented.) and regularize the inverse problem is needed. Informed by Huber's robust statistics framework, we propose an optimal regularizer to the regression problem. We compare results of the proposed method and other penalized regression regularization methods: ridge, lasso, adaptive-lasso and elastic-net under different strong hypothesis such as high conditioning number of the covariance matrix and high error amplitude, on both simulated and real data from the South London Stroke Register. The proposed approach can be extended to mixed regression models. Our inverse problem framework coupled with robust statistics methodology offer new insights in statistical regression and learning. It could open a new research development for model fitting and learning.