The Development and Validation of an Instrument for Assessing Teachers' Competence to Deal With Digital Problems Situations in the Classroom

Abstract

Teachers are increasingly called upon to skillfully integrate digital media into the teaching-learning processes. However, there is a lack of objective procedures for measuring digital skills among this target group. This paper presents the development and validation of a Situational Judgement Test to assess the handling of digital problem situations in the classroom. Employing a multistep development process, we developed a set of problems based on a preliminary survey (n = 72) and optimized them using a Loud Thinking Study (n = 20). We determined content validity and scoring key through an expert survey (n = 8). Two pretests with teachers-in-training (n = 83 and n = 58) established a final itemset based on item analyses. A validation study with teachers (n = 238) confirmed the postulated factor structure utilizing confirmatory factor analysis. The criterion validity was demonstrated through expected correlations with external criteria. We confirmed the prognostic validity with a structural equation model by significantly predicting the frequency of problems occurring when teachers use digital media.