Assuring quality in competence assessments: The value added of applying different assessment approaches to professional education

Abstract

Due to their multidimensional nature and their development in group settings, the assessment of competencies acquired in professional education goes beyond typical methodological difficulties related to any assessment of behaviour, personality or capability. Competencies are complex constructs for which it is challenging to develop reliable and valid assessments because classical assumptions such as unidimensionality, which means that components of an assessment should reflect only one underlying dimension, and independence of the individuals assessed are violated. The present paper discusses these methodological challenges and presents ways how to deal with them based on examples from different studies carried out on competence-based professional education at the secondary and post-secondary educational level. Classical test theory provides a useful tool for examining the reliability of competence assessments and examining the quality of an assessment as a whole (instead of examining single items). But since quality assurance in competence assessments is difficult, drawing on a range of other methodological approaches including generalizability theory (GT) and item response theory (IRT) is important. The recognition of these methodological challenges has strengthened the sensitivity to subject-specific and methodological problems of measurements during the past decade – not least because of public debates about the outcomes of international large-scale assessments of student achievement which have been scrutinized in the public media but also in academia. Collaboration of methodological and subject-matter experts as well as utilizing technological progress has strengthened the quality of competence assessments which is again demonstrated based on examples from professional education.