Changing the quality of teachers' written tests by implementing an authentic assessment teachers' training program

Abstract

This case study aimed to change the construction of teachers' written tests so that items were designed to assess competencies in an authentic and challenging way. A small group of five psychology teachers participated in 10 sessions of an authentic assessment faculty-training program, to learn to assess problem-solving competencies for situations typically faced by professionals in the workplace. The authentic assessment training emphasized the incorporation of three main characteristics: (1) inclusion of a realistic context, (2) measurement of higher order thinking skills, and (3) development of evaluative judgment, concerning the quality of their own performance. Post-training the itemś construction was analysed, according to their type and authenticity. Mixed effects logistic regression showed a statistically significant increase in open-response items, and two-way ANOVA indicated that cognitive challenge improved. The results showed written tests had: A) more open-response items of higher cognitive complexity, b) fewer items requiring closed, memorized responses, c) more use of realistic contexts to measure knowledge in a situated way, and d) improvement in curriculum alignment of tests and greater consistency in measuring competencies. These advances validate the authentic-assessment training program for a better written assessment design.