Content validity uses Rasch model on computerized testlet instrument to measure chemical literacy capabilities

Abstract

Content validity is useful for testing the suitability of test items with the materials and capabilities to be measured. This validity is done by testing the feasibility of the instrument through panel discussions by experts who are competent in their fields. In this study, content validity was applied to the computerized testlet instrument. Computerized testlet is a group of multiple-choice items to reveal the same information developed in a computerized system. The computerized testlet instrument in this study consists of sevens stem (the subject matter), each of with consists of three interrelated multiple-choice questions. This computerized testlet instrument was validated by nine raters consisting of three chemistry education experts, three chemistry teachers, and three chemistry education students. The validation was done by referring to the assessment of five aspects including the aspects of scientific competence, science subject knowledge, question context, spelling and language, and clarity of the question command. Assessment was done by giving a score of 1-4 in each aspect that refers to the assessment rubric. This research was a quantitative study using Rasch Model analysis. The result of the analysis shows that the exact agreement is 56.2% and the expected agreement is 58.2%. The most difficult aspects to be achieved in the sequential assessment of the questions are the aspects of spelling and language, clarity of the question command, science subject knowledge, and question context, while the easiest aspects to be achieved are the aspects of scientific competence.