Predictor versus response permutation for significance testing in weighted regression and redundancy analysis

Abstract

In testing an overall null hypothesis, it does not matter whether to permute the response variables (Y) while keeping the predictors fixed or to permute the predictor variables (X) while keeping the response variables fixed. However, in weighted (univariate and multivariate) regression and in partial tests these options yield different results. This paper defines and evaluates by simulation ordinary and standardized versions of X- and Y-permutation for tests which encompass the existing Double Semi-Partialling, here called Collins-Dekker, and Freedman-Lane permutation methods. When the error variance is inversely proportional to the weights, the standardized permutation methods (which use standardized residuals) are most powerful, as expected, but otherwise they can be extremely liberal. In contrast, ordinary X-permutation (which permutes the residuals of X given any covariates ‘as is’) is by far the most robust against variability in the weights and against the error variance-weight relationship and is thus recommended for general use.