Asymptotics of the allele frequency spectrum associated with the Bolthausen-Sznitman coalescents

Abstract

We consider a coalescent process as a model for the genealogy of a sample from a population. The population is subject to neutral mutation at constant rate ρ per individual and every mutation gives rise to a completely new type. The allelic partition is obtained by tracing back to the most recent mutation for each individual and grouping together indi- viduals whose most recent mutations are the same. The allele frequency spectrum is the sequence (N1(n), N2, … Nn(n)), where Nk(n) is number of blocks of size k in the allelic partition with sample size n. In this paper, we prove law of large numbers-type results for the allele frequency spectrum when the coalescent process is taken to be the Bolthausen- Sznitman coalescent. In particular, we show that n− 1(log n)N1(n)→p ρ and, k ≣ 2, n− 1 (log n)2 Nk(n)→pρ/k(k - 1))as n → ∞. Our method of proof involves tracking the formation of the allelic partition using a certain Markov process, for which we prove a fluid limit. © 2008 Applied Probability Trust.