Higher-order Thinking Test of Science for College Students Using Multidimensional Item Response Theory Analysis

Abstract

The purpose of this study was to construct a higher-order thinking test of science for pre-service elementary school teachers. The test was created using the ADDIE model. The analysis stage was carried out by identifying the needs and baseline of higher-order thinking skills of students from the department of primary School Teacher education in Yogyakarta. The design stage involved the creation of test blueprints and questions cards. The development stage involved validating the test’s content and construct validity. The content validity test was conducted using the Delphi technique with seven validators, whilst the construct validity test was conducted using item response theory and EFA. This study developed 77 questions, 73 multiple choice questions, and four essay questions, all of which were determined to be valid in terms of content and constructions. The HOTS test’s content validity test resulted in a V-value of 0.879 (valid with high criteria) based on the average Aiken’s V index. Meanwhile, reliability analysis using the Cronbach’s Alpha coefficient revealed a score of 0.907 for the 77 test items based on the construct validity test. The discriminatory index (di) classified all items as good, whereas the difficulty index (bi) classified 63 items as good and 10 as poor. The ten items were revised, despite their high index of difference. All of the test questions are appropriate for students whose ability score (θ) ranged from -2.85 to 2.15