Development of a PISA-based mathematical literacy instrument for Indonesian students using item response theory

Abstract

This study presents the development of a mathematical literacy assessment instrument for Indonesian students, grounded in the Programme for International Student Assessment (PISA) framework and constructed using Item Response Theory (IRT). Designed to measure students' ability to apply mathematical reasoning in authentic contexts, the instrument integrates four core dimensions: context, content, cognitive processes, and levels of reasoning. A research and development approach was employed, involving expert validation, pilot testing, and psychometric analysis. The final instrument comprises 32 items with strong content validity (Aiken's V = 0.80-1.00) and satisfactory reliability (Cronbach's α = 0.873). Rasch modeling confirmed unidimensionality, local independence, and parameter invariance, supporting the instrument's robustness across diverse ability levels. However, item difficulty distribution revealed a concentration of low-to-moderate tasks, with limited representation of higher-order competencies. The Test Information Function peaked near the population mean (θ ≈ -0.28), indicating optimal precision for average-ability students, while measurement accuracy declined at the extremes. These findings underscore the instrument's suitability for formative assessment and curriculum alignment, while highlighting the need for future item development targeting advanced mathematical literacy. Overall, the instrument offers a valid, reliable, and contextually responsive tool for evaluating Indonesian students' mathematical competencies in accordance with international standards.