Permutation tests for multi-factorial analysis of variance

by Dapeng P. Dp Zhang, D. Carbajulca, L. Ojeda, Genoveva Rossel, S. Milla, C. Herrera, Marc Ghislain, Benard Yada, Phinehas Tukamuhabwa, Bramwell Wanjala, Dong Jin Kim, Robert a. Skilton, Agnes Alajo, Robert O.M. M Mwanga, Zhangying Wang, Jun Li, Zhongxia Luo, Lifei Huang, Xinliang Chen, Boping Fang, Yujun Li, Jingyi Chen, Xiongjian Zhang, Elizabeth Ann Veasey, Aline Borges, Mariana Silva Rosa, Jurema R. Queiroz-Silva, E. De a Bressan, Nivaldo Peroni, Abdelhameed Elameen, Siri Fjellheim, Arild Larsen, Odd Arne Rognli, Leif Sundheim, Susan Msolla, Esther Masumba, Kiddo Mtunda, Sonja S. Klemsdal, Silver Tumwegamire, P. R. Rubaihayo, Don R. LaBonte, F. Diaz, R. Kapinga, Robert O.M. M Mwanga, W. J. Grüneberg, Roland Schafleitner, Luz R Tincopa, Omar Palomino, Genoveva Rossel, Ronald F Robles, Rocio Alagon, Carlos Rivera, Cynthia Quispe, Luis Rojas, Jaime a Pacheco, Julio Solis, Diogenes Cerna, Ji Young Kim, Jack Hou, Reinhard Simon, Sun-Hyung Kim, Tatsuro Hamada, Jingjie Hu, Makoto Nakatani, Antonio Garcia Lalusin, Toshikazu Kuranouchi, Tatsuhito Fujimura, Scovia Adikini, Settumba B. Mukasa, Robert O.M. M Mwanga, Richard W. Gibson, Simon Templar Gichuki, Maria Berenyi, Dapeng P. Dp Zhang, Michael Hermann, Josef Schmidt, Josef Glössl, Komel Burg, G. S. Varadarajan, C. S. Prakash, Douglas W. Miano, Don R. LaBonte, Christopher a. Clark, Janice Bohac, Marti Anderson, Cajo Ter Braak, Maniyam Nedunchezhiyan, Gangadharan Byju, Susantha K Jata, H W Karuri, E M Ateka, R Amata, a B Nyende, a W T Muigai, J. C. Cervantes-Flores, B. Sosinski, K. V. Pecota, Robert O.M. M Mwanga, G. L. Catignani, V. D. Truong, R. H. Watkins, M. R. Ulmer, G. C. Yencho, Wenzhi Zhou, Jun Yang, Yan Hong, Guiling Liu, Jianli Zheng, Zhengbiao Gu, Peng Zhang, Dapeng P. Dp Zhang, Jim Cervantes, Zosimo Huam, Edward Carey, Marc Ghislain, Genoveva Rossel, Albert Kriegner, Dapeng P. Dp Zhang, Jie Qin, Li Hua, Zhai Hong, Wang Yu-ping, Li Qiang, Lifei Huang, Huifang Jiang, Xiaoping Ren, Yuning Chen, Yingjie Xiao, Xinyan Zhao, Mei Tang, Jiaquan Huang, Hari D. Upadhyaya, Boshou Liao, Marti Anderson, Cajo Ter Braak, Silver Tumwegamire, Jonathan Marchini, Lon R Cardon, Michael S Phillips, Peter Donnelly, Jesse W. Breinholt, Renee Van Buren, Olga R. Kopp, Catherine L. Stephen, Gao Feng, Gong Yifu, Zhang Pinbo, Dapeng P. Dp Zhang, Genoveva Rossel, Albert Kriegner, Robert Hijmans, Shih Y. Hwang, Yu T. Tseng, Hsiao F. Lo show all authors
Genetic Resources and Crop Evolution ()
Get full text at journal

Abstract

Several permutation strategies are often possible for tests of individual terms in analysis-of-variance (ANOVA) designs. These include restricted permutations, permutation of whole groups of units, permutation of some form of residuals or some combination of these. It is unclear, especially for complex designs involving random factors, mixed models or nested hierarchies, just which permutation strategy should be used for any particular test. The purpose of this paper is two-fold: (i) we provide a guideline for constructing an exact permutation strategy, where possible, for any individual term in any ANOVA design; and (ii) we provide results of Monte Carlo simulations to compare the level accuracy and power of different permutation strategies in two-way ANOVA, including random and mixed models, nested hierarchies and tests of interaction terms. Simulation results showed that permutation of residuals under a reduced model generally had greater power than the exact test or alternative approximate permutation methods (such as permutation of raw data). In several cases, restricted permutations, in particular, suffered more than other procedures, in terms of loss of power, challenging the conventional wisdom of using this approach. Our simulations also demonstrated that the choice of correct exchangeable units under the null hypothesis, in accordance with the guideline we provide, is essential for any permutation test, whether it be an exact test or an approximate test. For reference, we also provide appropriate permutation strategies for individual terms in any two-way or three-way ANOVA for the exact test (where possible) and for the approximate test using permutation of residuals. Several permutation strategies are often possible for tests of individual terms in analysis-of-variance (ANOVA) designs. These include restricted permutations, permutation of whole groups of units, permutation of some form of residuals or some combination of these. It is unclear, especially for complex designs involving random factors, mixed models or nested hierarchies, just which permutation strategy should be used for any particular test. The purpose of this paper is two-fold: (i) we provide a guideline for constructing an exact permutation strategy, where possible, for any individual term in any ANOVA design; and (ii) we provide results of Monte Carlo simulations to compare the level accuracy and power of different permutation strategies in two-way ANOVA, including random and mixed models, nested hierarchies and tests of interaction terms. Simulation results showed that permutation of residuals under a reduced model generally had greater power than the exact test or alternative approximate permutation methods (such as permutation of raw data). In several cases, restricted permutations, in particular, suffered more than other procedures, in terms of loss of power, challenging the conventional wisdom of using this approach. Our simulations also demonstrated that the choice of correct exchangeable units under the null hypothesis, in accordance with the guideline we provide, is essential for any permutation test, whether it be an exact test or an approximate test. For reference, we also provide appropriate permutation strategies for individual terms in any two-way or three-way ANOVA for the exact test (where possible) and for the approximate test using permutation of residuals.

Author-supplied keywords

Cite this document (BETA)

Readership Statistics

3 Readers on Mendeley
by Discipline
 
by Academic Status
 
33% Researcher (at an Academic Institution)
 
33% Ph.D. Student
 
33% Assistant Professor

Sign up today - FREE

Mendeley saves you time finding and organizing research. Learn more

  • All your research in one place
  • Add and import papers easily
  • Access it anywhere, anytime

Start using Mendeley in seconds!

Sign up & Download

Already have an account? Sign in