Validity of UKM2 intelligence test using Rasch analysis

Abstract

UKM2 is an intelligence test developed to screen gifted students in Malaysia. This study aims at identifying the reliability and validity of the UKM2 using the item analysis method. 497 undergraduate students randomly selected from a Malaysian local university, were chosen as respondents. The test was administered via on-line. The Items Response Theory (IRT) is used with the application of Rasch model of measurement for the analysis procedure. The scoring of UKM2 are dichotomus and partial credit models based. The result of study shows that all the subtests have the reliability values of Cronbach's alpha>0.80 (0.86-0.99) and separation indexes>2 (2.51-11.94) with the number of stratas 3-16. All the subtests are unidimensionals except for Block Design and Comprehension subtests. The study confirms that UKM2 is a multidimensional test with four correlated dimensions which are Verbal Comprehension Index (VCI), Working Memory Index (WMI), Perceptual Reasoning Index (PRI) and Processing Speed Index (PSI). Research shows that 221 are fit items (MNSQ 0.5-1.5), 69 items are overfit and 80 items are misfits, using the Infit and Outfit statistics. Most of the misfit items are from the Symbol Search subtests, while most of the overfit items are from Arithmetic, Letter-Number Sequencing and Picture Completion subtests. The study suggests for the elimination of the overfit items and for the revision or removal of the misfit items in the UKM2 intelligence test. The good psychometrics properties of the items from the analysis of the study could be used for the banking purposes in the adult version of UKM2.