When are random data not random, or is the PTP test useful?

Abstract

Recently, empirical evidence was presented that the permutation tail probability (PTP) test has extremely low discriminatory power when assessing character covariance in phylogenetic data based on bootstrap measures of confidence. Here we are concerned with the problem of using one statistical approach, especially when applied to empirical data, to judge the performance of another. Applying an appropriate statistical approach, we statistically demonstrated that the PTP test is extremely weak in detecting the absence of character covariation. In addition, we show that PTP is highly dependent on the number of terminals and the proportion of character states in phylogenetic matrices. In conclusion, we advocate the use of simulation studies when testing the performance of statistical tools applied to phylogenetic data. © 2000 The Willi Hennig Society.