An alternative assessment instrument for science teachers’ ability in designing and implementing assessments

Abstract

Designing and implementing assessments are essential abilities for teachers. Both abilities must be assessed and trained. An alternative assessment can be one solution that can be used to assess and train both abilities. Thus, this study aims to evaluate the quality of the developed alternative assessment instrument on the basis of content validity and reliability. The method for testing content validity uses Aiken and extended Gregory’s formula, in which the instrument is focused on content validity. For reliability testing, this instrument is evaluated on the basis of internal reliability, rater reliability, and generalizability theory. The alternative assessment instrument as a project-based assessment consisted of three parts, namely, planning, implementation, and reflection and follow-up. The instrument is evaluated by three experts, and the results are processed via statistical analysis. The Aiken formula yields average validity index results for the planning, implementation, reflection and follow-up parts, with values of 0.87, 0.83, and 0.81, respectively. Using the Gregory formula, the validation index of the instrument is 0.90. The Cronbach’s a is 0.939, and McDonald’s w is 0.958, indicating that the instrument has a very high level of reliability. The value of Fleiss’ kappa is 0.742. The two values of the ICC are 0.798 (consistency) and 0.806 (agreement). Using generalizability theory, values of 0.9220 (generalizability coefficient, G) and 0.9220 (dependability coefficient (F) are obtained. The findings indicate that the instrument can be classified as valid or reliable on the basis of content validity, internal reliability, and rater reliability. The novelty of the instrument is that it not only assesses teachers’ ability but also provides stages that guide teachers in designing, implementing, and following up on the shortcomings and advantages of implementing the assessments.