The effects of range restriction on estimates of criterion interrater reliability: Implications for validation research

Abstract

This paper explores the implications of the fact that range restriction affects reliability coefficients as well as validity coefficients. Of concern was the possibility that, if range restriction produced an underestimate of criterion reliability, the use of such an estimate in correcting observed validity coefficients for attenuation could result in overestimating population validity. To address this issue, 3 scenarios were considered: truncation on the predictor, truncation on an unobserved variable, and truncation on the criterion. The 3 scenarios were found to have substantial differences in their consequences for reliability and validity estimation. Although range restriction did result in an underestimate of criterion reliability, it is generally not the case that these underestimates will result in overestimates of validity if researchers make effective use of available information about the presence and degree of range restriction.