Optimal designs for multilevel studies

Abstract

The analysis of multilevel data with individuals nested within clusters is complicated by the correlation between outcomes of individuals within the same cluster. Ignoring this correlation and the use of traditional analysis methods, like ordinary least squares regression, may sometimes lead to biased parameter estimates and will generally lead to incorrect standard errors and, consequently, to incorrect tests and conclusions on effect sizes. The presence of an intraclass correlation also complicates the design of multilevel studies. Optimal designs calculated from standard formulae for non-nested data [5] may be far from optimal for multilevel data. Moreover, these formulae only specify the total number of individuals needed to gain a certain power on statistical tests, and cannot specify the number of clusters and the number of individuals per cluster. Experiments and observational studies in the social and medical sciences often involve large amounts of time, money, and labor. These efforts could be somewhat wasted if the study was not designed optimally. Therefore, guidelines for the optimal design of multilevel studies are asked for. During the last two decades, a number of papers on the design of multilevel studies has been published. Most have focussed on the optimal sample sizes for cluster randomized trials [9, 10, 12, 18, 23, 25, 29, 34, 35, 36, 39, 42, 49, 55], and multisite randomized trials where randomization to treatment conditions is done at the patient level and treatment by site interaction may be present [50]. A comparison of cluster randomized trials and multisite trials with person randomization shows that the latter are more efficient [30, 34, 35, 36]. [57] derive sample size formulae for two-level designs with any number of explanatory variables at each level. Cohen [7] derives optimal sample size formulae for surveys based on several optimality criteria for the fixed and random part. Afshartous [1] and Mok [41] compare designs with different sample sizes at both levels by means of simulation studies. For multilevel experiments, four design issues may arise. The first three that are listed may also arise for surveys with nested data. The first design issue concerns the optimal allocation of units, or, in other words, the optimal sample sizes at each level of the multilevel data structure. The optimal sample sizes are restricted by the actual sample sizes in the study population, since the number of clusters that are enrolled in the study cannot be larger than the number of clusters that are available for the study. Likewise, the number of individuals per cluster in the study cannot be larger than the actual cluster size. Sampling individuals within an already selected cluster may be less expensive than sampling in a new cluster. This can be expressed by a cost function that is used as a precondition in the derivation of the optimal sample sizes. The second design issue concerns the required budget to obtain a specified power on the test of a certain parameter given the true value of that parameter and a type I error rate. As we will see in the next section, the power of the test of a certain parameter is inversely related to the variance of that parameter, which depends on the sample sizes at each level of the multilevel data structure. Thus, the second design issue is closely related to the first one. The third design issue concerns the robustness of optimal designs. A prior specification of the values of the model parameters, in particular the intraclass correlation coefficient, must be given to calculate optimal sample sizes, and one may wonder if the optimal design is robust against incorrect prior specifications. A fourth design issue that may be considered is the efficiency of cluster randomization versus randomization at the individual level. Although individual-level randomization gives a higher power on statistical tests of a treatment effect, randomization is often done in practice at the cluster level and one may wonder what the loss in efficiency for this level of randomization is. Reasons to favor a cluster randomized trial are often of an ethical, practical, logistical, or administrative nature. Examples are the need to reduce costs and the need to avoid control group contamination, which occurs when information leaks from the intervention to the control group. In this chapter we will give some guidelines for designing multilevel experiments and surveys (observational studies). The contents of this chapter are as follows. The next section focuses on optimality criteria and power calculation. Section 4.3 deals with the optimal design of multilevel experiments. Thereafter we focus on optimal experimental designs for models with covariates (Section 4.4), and for multilevel logistic models (Section 4.5). Section 4.6 gives However, control group contamination may destroy this advantage of person randomization and call for cluster randomization [32]. Snijders and Bosker results for optimal experimental designs with longitudinal data. Sections 4.7 and 4.8 deal with optimal designs for surveys and variance parameters, respectively. In Section 4.9 the robustness of optimal designs against an incorrect prior specification of the values of the model parameters is dealt with. This chapter concludes with some remarks on the use of the optimal designs in practice. Optimal designs will be derived for two levels of nesting; optimal designs for three levels of nesting can be found elsewhere [30, 34]. For the sake of concreteness, units at level 1 and 2 are called pupils and schools in this chapter, but, of course, any other terminology may be substituted. We will focus on optimal designs that minimize one optimality criterion at a time; multiple-objective optimal designs are presented elsewhere [40]. © 2008 Springer Science+Business Media, LLC.