Generalization of the QST framework in hierarchically structured populations: Impacts of inbreeding and dominance

Abstract

QST is a differentiation parameter based on the decomposition of the genetic variance of a trait. In the case of additive inheritance and absence of selection, it is analogous to the genic differentiation measured on individual loci, FST. Thus, QST−FST comparison is used to infer selection: selective divergence when QST > FST, or convergence when QST < FST. The definition of Q-statistics was extended to two-level hierarchical population structures with Hardy–Weinberg equilibrium. Here, we generalize the Q-statistics framework to any hierarchical population structure. First, we developed the analytical definition of hierarchical Q-statistics for populations not at Hardy–Weinberg equilibrium. We show that the Q-statistics values obtained with the Hardy–Weinberg definition are lower than their corresponding F-statistics when FIS > 0 (higher when FIS < 0). Then, we used an island model simulation approach to investigate the impact of inbreeding and dominance on the QST−FST framework in a hierarchical population structure. We show that, while differentiation at the lower hierarchical level (QSR) is a monotonic function of migration, differentiation at the upper level (QRT) is not. In the case of additive inheritance, we show that inbreeding inflates the variance of QRT, which can increase the frequency of QRT > FRT cases. We also show that dominance drastically reduces Q-statistics below F-statistics for any level of the hierarchy. Therefore, high values of Q-statistics are good indicators of selection, but low values are not in the case of dominance.