Identifying levers related to student performance on high-stakes science exams: Examining school, teaching, teacher, and professional development characteristics

Abstract

Background: Many students enter into postsecondary education without the preparation to face the demands of postsecondary coursework in science. Increasingly, policymakers and educational researchers are responding to calls for reforming secondary education to provide more opportunity for all students to receive high-quality education and to become career and college ready. Purpose: This study attempts to identify levers to increase student learning in secondary education. In particular, it examines relationships between school, teaching, teacher, and teacher professional development characteristics and student scores on high-stakes Advanced Placement (AP) examinations in the sciences. Setting: This study is situated in the context of the large-scale, top-down, nationwide AP curriculum and examination reform in the sciences (biology, chemistry, physics) in the United States. This is an unprecedented opportunity to analyze changing educational landscapes in the United States with large-scale national student-, teacher-, school-, and district-level datasets across multiple science disciplines and different stages of the curriculum reform implementation connected to a standardized and high-stakes student outcome measure. Population: This study analyzes nationwide data samples of the AP Biology, AP Chemistry, and AP Physics population during the first, second, and third year of the curriculum reform implementation. Across disciplines and years, the analytical samples include a total of 113,603 students and 6,046 teachers. Research design: This empirical quantitative study uses data from web-based surveys sent to all AP science teachers. Additionally, the College Board provided student- and school-level data for all students taking AP examinations. Data preparation methods included exploratory and confirmatory factor analysis. Associations with student achievement were analyzed through a multilevel ordered logistic regression analysis, separately by science discipline and year of the curriculum reform implementation. Afterwards, results were aggregated through a meta-analysis. Findings: Even after controlling for student background variables, roughly 60% of the AP score variance could be explained at the teacher and school levels. In particular, teachers' perceived administrative support, self-efficacy, teaching experience, and elements of classroom instruction were related to student performance. Notably, teachers' professional development participation-which has been a major focus of interventions-has a small, mixed impact on student achievement. Conclusion: The identified levers for improving student achievement provide a strong rationale for the continued efforts of policymakers to improve school environments and to support science teachers, with the ultimate goal of improving student learning to help all students to be prepared for college and ready for their future careers.