Constraints on the F ST -Heterozygosity Outlier Approach

73Citations
Citations of this article
125Readers
Mendeley users who have this article in their library.

This article is free to access.

Abstract

The F ST -heterozygosity outlier approach has been a popular method for identifying loci under balancing and positive selection since Beaumont and Nichols first proposed it in 1996 and recommended its use for studies sampling a large number of independent populations (at least 10). Since then, their program FDIST2 and a user-friendly program optimized for large datasets, LOSITAN, have been used widely in the population genetics literature, often without the requisite number of samples. We observed empirical datasets whose distributions could not be reconciled with the confidence intervals generated by the null coalescent island model. Here, we use forward-in-time simulations to investigate circumstances under which the F ST -heterozygosity outlier approach performs poorly for next-generation single nucleotide polymorphism (SNP) datasets. Our results show that samples involving few independent populations, particularly when migration rates are low, result in distributions of the F ST -heterozygosity relationship that are not described by the null model implemented in LOSITAN. In addition, even under favorable conditions LOSITAN rarely provides confidence intervals that precisely fit SNP data, making the associated P-values only roughly valid at best. We present an alternative method, implemented in a new R package named fsthet, which uses the raw empirical data to generate smoothed outlier plots for the F ST -heterozygosity relationship.

Cite

CITATION STYLE

APA

Flanagan, S. P., & Jones, A. G. (2017). Constraints on the F ST -Heterozygosity Outlier Approach. Journal of Heredity, 108(5), 561–573. https://doi.org/10.1093/jhered/esx048

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free