Developing IRT-based physics critical thinking skill test: A CAT to answer 21st century challenge

Abstract

The objective of this study was to develop Physics critical thinking skill test using computerized adaptive test (CAT) based on item response theory (IRT). This research was a development research using 4-D (define, design, develop, and disseminate). The content validity of the items was proven using Aiken's V. The test trial involved 252 students in Yogyakarta, Indonesia. The data were analysed according to partial credit model (PCM). The test reliability was estimated using PCM based on information function (IF) and standard error measurement (SEM), whereas the empirical validity was proven through INFIT MNSQ. The results showed that all items were valid with Aiken's V spread from 0.67 to 1.00 and INFIT MNSQ values from 0.86 to 1.20. The item difficulty index ranges from -0.75 to 1.30, which was a good item difficulty index. The IF and SEM showed that PhysTCriTS was suitable to measure students' critical thinking abilities from -1.80 to 1.50, with the reliability score reaches 0.75. The results of this study have an implication to reduce cheating in a test because each student gets a different item in accordance to the student's ability. The test using CAT may accurately and effectively measure physics critical thinking skill of students.