Meta-analysis and meta-regression: An alternative to multilevel analysis when the number of countries is small

Abstract

Hierarchically nested data structures are often analyzed by means of multilevel techniques. A common situation in cross-national comparative research is data on two levels, with information on individuals at level 1 and on countries at level 2. However, when dealing with few level-2 units (e.g. countries), results from multilevel models may be unreliable due to estimation bias (e.g. underestimated standard errors, unreliable country-level variance estimates). This chapter provides a discussion on multilevel modeling inaccuracies when using a small level-2 sample size, as well as a list of available alternative analytic tools for analyzing such data. However, as in practice many of these alternatives remain unfeasible in testing hypotheses central to cross-national comparative research, the aim of this chapter is to propose and illustrate a new technique - the 2-step meta-analytic approach - reliable in the analysis of nested data with few level-2 units. In addition, this method is highly infographic and accessible to the average social scientist (not skilled in advanced simulation techniques).