Allele frequency difference AFD-an intuitive alternative to FST for quantifying genetic population differentiation

Abstract

Measuring the magnitude of differentiation between populations based on genetic markers is commonplace in ecology, evolution, and conservation biology. The predominant differentiation metric used for this purpose is FST. Based on a qualitative survey, numerical analyses, simulations, and empirical data, I here argue that FST does not express the relationship to allele frequency differentiation between populations generally considered interpretable and desirable by researchers. In particular, FST (1) has low sensitivity when population differentiation is weak, (2) is contingent on the minor allele frequency across the populations, (3) can be strongly affected by asymmetry in sample sizes, and (4) can differ greatly among the available estimators. Together, these features can complicate pattern recognition and interpretation in population genetic and genomic analysis, as illustrated by empirical examples, and overall compromise the comparability of population differentiation among markers and study systems. I argue that a simple differentiation metric displaying intuitive properties, the absolute allele frequency difference AFD, provides a valuable alternative to FST. I provide a general definition of AFD applicable to both bi- and multi-allelic markers and conclude by making recommendations on the sample sizes needed to achieve robust differentiation estimates using AFD.