Are fourth grade students able to rate instructional quality validly? Results from German Language classes

Abstract

Recent studies on teaching quality in different subjects often focus on classroom management, constructive support and cognitive activation and their role in fostering successful learning processes. In large scale assessments teaching characteristics are usually assessed by student ratings. The validity of these ratings, however, is often discussed critically. With this study, we aim to contribute to the discussion about the validity of student ratings: First, we tested the ability of fourth graders to differentiate between the three dimensions. Secondly, we examined connections between each dimension and reading achievement as an indicator for predictive validity. In PIRLS 2016 (Progress in International Reading Literacy Study), fourth grade students in Germany (3797 students from 192 classes) were asked to rate the teaching quality in their German language classes according to the three dimensions. Using that data and the PIRLS reading achievement test we specified multilevel confirmatory factor analyses. A model with three factors on the individual and the classroom level fitted the data acceptably. In order to analyse predictive validity, doubly-latent multilevel models were also specified. The ratings of classroom management were positively related to reading achievement on both levels. Ratings of cognitive activation were positively associated with reading achievement on the classroom-level, while student support only showed a significant connection to reading achievement on the individual level. When all three dimensions were entered into the model together, no relation could be identified on the classroom level, whereas the connections on the individual level remained stable.