Avoiding pitfalls in estimating heritability with the common options approach

Abstract

In many circumstances, heritability estimates are subject to two potentially interacting pitfalls: the spatial and the regression to the mean (RTM) fallacies. The spatial fallacy occurs when the set of potential movement options differs among individuals according to where individuals depart. The RTM fallacy occurs when extreme measurements are followed by measurements that are closer to the mean. We simulated data from the largest published heritability study of a behavioural trait, colony size choice, to examine the operation of the two fallacies. We found that spurious heritabilities are generated under a wide range of conditions both in experimental and correlative estimates of heritability. Classically designed cross-foster experiments can actually increase the frequency of spurious heritabilities. Simulations showed that experiments providing all individuals with the identical set of options, such as by fostering all offspring in the same breeding location, are immune to the two pitfalls.