Randomized control trials in an imperfect world

Abstract

Pros "Partial identification" methods are easy to construct and straightforward to implement in order to estimate causal effects in RCTs with imperfect compliance. Of particular policy interest is that partial identification methods provide a measure of treatment effectiveness for the entire population. Partial identification methods require less stringent statistical assumptions than alternative approaches. eLeVaTOr PiTCh Randomized control trials (RCTs) have become increasingly important as an evidence-based method to evaluate interventions such as government programs and policy initiatives. Frequently, however, RCTs are characterized by "imperfect compliance," in that not all the subjects who are randomly assigned to take a treatment choose to do so. This could result in a failure to identify the treatment effect, or the impact of the treatment on the population. However, useful information on treatment effectiveness can still be recovered by estimating "bounds," or a range of values in which treatment effectiveness can lie. auThOr'S MaiN MeSSaGe RCTs randomly allocate individuals to treatment or control groups and have become the gold standard of policy evaluation. However, non-perfect compliance in many trials means that comparing the average outcomes for the treated and control populations fails to provide an unbiased estimate of the effect of the policy that is of interest. Instead, only a "partial identification" of possible values of the effect of the policy can be recovered. Even so, this still provides policy-relevant information. Research shows that the methods are simple to implement and are based more on what is known rather than what is assumed. Cons Partial identification does not provide a single prediction but rather a range of values that treatment effectiveness can take. The range of possible values that treatment effectiveness can take can be sufficiently wide that it is no longer possible to determine whether the treatment has a positive or negative causal effect. randomized control trials in an imperfect world How can we assess the policy effectiveness of randomized control trials when people don't comply?