Doubts on the validity of correlation as a validation tool in second language testing research: the case of cloze testing

Abstract

The statistical analysis technique of correlation has extensively been used in second/foreign language testing research for a variety of purposes. Most commonly, the technique has been used to validate newly constructed language tests (particularly cloze tests) against previous supposedly valid measures. Tests such validated are meant to substitute the older ones, and based on these newly validated tests, important decisions are made on the candidates’ suitability for certain careers, entry into universities, and so on. All such decisions can be appropriate and justified only when the validating technique is the right one. Doubts are cast in this paper on such widespread and unquestioned use of the technique of correlation for substitution purposes. After a short introduction, the meaning of correlation and cloze test is clarified. Then, a few studies in which the technique of correlation has been misapplied are reviewed. Finally, the argument against correlational validation is presented, and suggestions as to alternative validation techniques are offered.