Exact level and power of permutation, bootstrap, and asymptotic tests of trend

Abstract

We develop computational tools that can evaluate the exact size and power of three tests of trend (e.g., permutation, bootstrap and asymptotic) without resorting to large-sample theory or simulations. We then use these tools to compare the operating characteristics of the three tests. It is seen that the bootstrap test is ultra-conservative relative to the other two tests and as a result suffers from a severe deterioration in power. The power of the asymptotic test is uniformly larger than that of the other two tests, but it fails to preserve the Type I error for most of the range of the baseline response probability. The permutation test, being exact, is guaranteed to preserve the Type I error throughout the range of the baseline response probability. The price paid for this guarantee is a loss of power relative to the asymptotic test. The power loss is, however, small in most situations. Copyright © 2002 JMASM, Inc.