Changing the order of factors does not change the product but does affect students’ answers, especially girls’ answers

Abstract

This study is aimed at exploring how different formulations of the same mathematical item may influence students’ answers, and whether or not boys and girls are equally affected by differences in presentation. An experimental design was employed: the same stem-items (i.e., items with the same mathematical content and question intent) were formulated differently and administered to a probability sample of 1647 students (grade 8). All the achievement tests were anchored via a set of common items. Students’ answers, equated and then analysed using the Rasch model, confirmed that different formulations affect students’ performances and thus the psychometric functionality of items, with discernible differences according to gender. In particular, we explored students’ sensitivity to the effect of a typical misconception about multiplication with decimal numbers (often called “multiplication makes bigger”) and tested the hypothesis that girls are more prone than boys to be negatively affected by misconception.