Noninferiority margins in trials of thrombectomy devices for acute ischemic stroke is the bar being set too low?

Abstract

Background and Purpose-Novel endovascular thrombectomy (EVT) devices for acute ischemic stroke are often cleared byregulatory agencies on the basis of noninferiority trials. The relation between the noninferiority margins used in trials andthe minimal clinically important differences (MCIDs) determined by experts have not been systematically investigated.Methods-Systematic searches were performed to identify (1) all noninferiority design or noninferiority-presentedstroke-EVT trials for acute ischemic stroke, (2) all studies determining the MCIDs for the same outcomes, and (3) allnoninferiority coronary revascularization trials. Stroke-EVT trial results were reanalyzed using the broad noninferioritymargins originally used and narrower noninferiority margins derived from formal MCID studies.Results-We identified 7 noninferiority-designed or noninferiority-interpreted stroke-EVT controlled trials, enrolling 1766patients, variously comparing coil retrievers, first- and second-generation stent retrievers, and aspiration devices. In 6 trials,the primary outcome was achievement of reperfusion, using noninferiority margins of 15% (3 trials), 10% (2 trials), and8% (1 trial). In contrast, a stroke expert survey identified the MCID for reperfusion as 3.1% to 5%, and cardiac trials usednoninferiority margins of 3.5% to 4.4%. In one stroke-EVT trial, the primary outcome was functional independence, usinga noninferiority margin of 15%. However, 2 stroke expert survey studies identified MCIDs for functional independenceas having lower values, 5% and 1% to 1.5%. For both reperfusion and functional independence outcomes, all 7 trialsdemonstrated noninferiority with the broadest noninferiority margin, but only 4 and 3 trials demonstrated noninferioritywith actual expert-derived margins for reperfusion and functional independence, respectively.Conclusions-Noninferiority margins used in EVT device trials have regularly exceeded the MCIDs determined by strokeexperts, as well as margins used for cardiac devices. New approaches, such as the use of reasonably adequate performancemargins, rather than noninferiority margins, are needed to optimize stroke-EVT trial design integrity and trial performancefeasibility.