What matters for student learning outcomes? A systematic review of studies exploring system-level factors of educational effectiveness

Abstract

Meta-analysis comprises a powerful tool for synthesising prior research and empirically validating theoretical frameworks. Using this tool and two recent multilevel models of educational effectiveness as guiding frameworks, this paper synthesises the results of 195 studies investigating the association between system-level characteristics and student learning outcomes. Results show a broad range of system-level factors studied in the international literature, which could be grouped into the three categories used in the integrated multilevel model of education: antecedents determined by the larger societal context, including factors such as level of development, inequality and societal values; system ecology and structural reform, covering factors such as decentralisation, accountability and stratification; and direct educational policies, including financial resources, time resources and variables related to teacher training and qualifications. Results highlight the importance of the larger context in which educational systems operate, as well as the need for further research looking at actual educational policies. Further analyses provide support for a generic effect of system-level factors, regardless of the educational level or subject domain assessed. However, results show variation in terms of methodological choices, such as the number of levels used in the analysis. Based on these results, implications for theory, research, and policy are drawn. Context and implications Rationale for this study We present a meta-analysis of the international literature on system-level factors associated with student learning outcomes to contribute to the improvement of quality in education. Why the new findings matter A synthesis of the literature is important to identify what kind of factors have been studied so far, their consistency and significance, and to identify future research needs. Implications for theory, research, and policy Socioeconomic factors, such as affluence and inequality, and other characteristics of the larger societal context are important system-level conditions. Conducting the meta-analysis allowed to identify areas where more research is needed, such as on the impact of national educational policies, and on the impact of system factors on non-cognitive learning outcomes. Further studies are also needed with a longitudinal component and focusing on the indirect effects of system-level factors. The results of the meta-analysis give further support to exerting caution when it comes to educational reforms and their implementation across different contexts, particularly because of the role that system inequality plays for educational outcomes.