AN INVESTIGATION OF THE IMPACT OF COMPOSITION MEDIUM ON THE QUALITY OF TOEFL WRITING SCORES

Abstract

This study examined scores from 133,906 operationally scored Test of English as a Foreign Language™ (TOEFL®) essays to determine whether the choice of composition medium has any impact on score quality for subgroups of test-takers. Results of analyses demonstrate that (a) scores assigned to word-processed essays are slightly more reliable than scores assigned to handwritten essays and exhibit higher correlations with TOEFL multiple-choice subscores; (b) female test-takers, examinees whose native language is not based on a Roman/Cyrillic alphabet, and examinees with lower English proficiency are more likely to choose the handwriting medium; (c) the probability of choosing handwriting as the composition medium increases with age for Asian examinees, but decreases with age for most European examinees; and (d) examinees with lower TOEFL multiple-choice scores tend to have higher handwritten than word-processed essay scores, while examinees with higher TOEFL multiple-choice scores tend to have similar scores in either medium.