Using competencies to explain mathematical item demand: A work in progress

Abstract

This chapter describes theoretical and practical issues associated with the development and use of a rating scheme for the purpose of analysing mathematical problems–specifically, to assess the extent to which solving those problems calls for the activation of a particular set of mathematical competencies. The competencies targeted through the scheme are based on the mathematical competencies that have underpinned each of the PISA Mathematics Frameworks. The scheme consists of operational definitions of the six competencies (labelled as communication; devising strategies; mathematisation; representation; using symbols, operations and formal language; and reasoning and argument), descriptions of four levels of activation of each competency, and examples of the ratings given to particular items together with commentary that explains how each proposed rating is justified in relation to the competency definition and level descriptions. The mathematical problems used so far to investigate the action of those competencies are questions developed for use in the PISA survey instruments from 2000 through to 2012. Ratings according to the scheme predict a large proportion of the variation in difficulty across items, providing evidence that these competencies are important elements of students’ problem solving capabilities. The appendix gives definitions of each competence and the specification of each of four levels for each.