Psychological assessments of paired-comparison and Likert formats: Comparing their reliability, validity, and ease of answering questions

Abstract

Despite the widespread use of Likert scales for psychological assessments, problems of response biases —such as response style and social desirability biases —have been associated with this format. Questionnaires employing the forced-choice format, of which paired comparison is one of the most popular methods, have been drawing researchers’ attention as an alternative that is resistant to response biases. In the current study, we compared the assessments of the paired comparison and Likert formats in terms of their reliability, validity, and ease in answering questions using two psychological questionnaires. The results revealed that the Likert format was superior in terms of reliability and ease in answering, while the paired-comparison format exhibited better factorial validity. The questionnaire that contained a few reverse items underperformed in the paired-comparison, which requires due attention during scale development.