Considering genomic scans for selection as coalescent model choice

Abstract

First inspired by the seminal work of Lewontin and Krakauer (1973. Distribution of gene frequency as a test of the theory of the selective neutrality of polymorphisms. Genetics 74(1):175-195.) andMaynard Smith andHaigh (1974. The hitch-hiking effect of a favourable gene. Genet Res. 23(1):23-35.), genomic scans for positive selection remain awidely utilized tool inmodern population genomic analysis. Yet, the relative frequency and genomic impact of selective sweeps have remained a contentious point in the field for decades, largelyowing to an inability to accurately identify their presence and quantify their effects-with currentmethodologies generally being characterized by low true-positive rates and/or high false-positive rates under many realistic demographic models. Most of these approaches are based on Wright-Fisher assumptions and the Kingman coalescent and generally rely on detecting outlier regions which do not conform to these neutral expectations. However, previous theoretical results have demonstrated that selective sweeps are well characterized by an alternative class ofmodel known as the multiple-merger coalescent. Taken together, this suggests the possibility of not simply identifying regionswhich reject the Kingman, but rather explicitly testing the relative fit of a genomicwindow to themultiple-merger coalescent.We describe the advantages of such an approach,whichowe to the branching structure differentiating selective and neutral models, and demonstrate improved power under certain demographic scenarios relative to a commonly used approach.However, regions of the demographic parameter space continue to exist inwhich neither this approach nor existing methodologies have sufficient power to detect selective sweeps.