Standard Setting in a Formative Assessment of Digital Responsibility Among Norwegian Eighth Graders

Abstract

Students’ digital responsibility is an important topic in a digital society. Since 2016, a learning supportive 50-item test in digital responsibility is available for Norwegian eighth-graders. Rasch model was used to select tasks to the test. Our study addresses experiences from using two standard setting methods, Angoff and bookmark, to set the cut-off scores and to define the proficiency levels of digital responsibility. In this process, as this is a learning supportive assessment, the decision makers had to take into consideration both the implied expectations from teachers who would use the assessment in their classroom practices and the results of psychometric analyses. A sample test with 1026 students from 26 schools was used to define three proficiency levels. The use of two standard setting methods, Angoff and bookmark, gave different results, and this difference addresses uncertainty about where to set the cut-off score. What are the validity arguments for considering different expectations when setting cut scores?