Interactive martingale tests for the global null

Abstract

Global null testing is a classical problem going back about a century to Fisher’s and Stouffer’s combination tests. In this work, we present simple martingale analogs of these classical tests, which are ap-plicable in two distinct settings: (a) the online setting in which there is a possibly infinite sequence of p-values, and (b) the batch setting, where one uses prior knowledge to preorder the hypotheses. Through theory and sim-ulations, we demonstrate that our martingale variants have higher power than their classical counterparts even when the preordering is only weakly informative. Finally, using a recent idea of “masking” p-values, we develop a novel interactive test for the global null that can take advantage of co-variates and repeated user guidance to create a data-adaptive ordering that achieves higher detection power against structured alternatives.