Splitting trees with neutral Poissonian mutations II: Largest and Oldest families

International audienceWe consider a supercritical branching population, where individuals have i.i.d. lifetime durations (which are not necessarily exponentially distributed) and give birth (singly) at constant rate. We assume that individuals independently experience neutral mutations, at constant rate $\theta$ during their lifetimes, under the infinite-alleles assumption: each mutation instantaneously confers a brand new type, called allele or haplotype, to its carrier. The type carried by a mother at the time when she gives birth is transmitted to the newborn. We are interested in the sizes and ages at time $t$ of the clonal families carrying the most abundant alleles or the oldest ones, as $t\to\infty$, on the survival event. Intuitively, the results must depend on how the mutation rate $\theta$ and the Malthusian parameter $\alpha>0$ compare. Hereafter, $N\equiv N_t$ is the population size at time $t$, constants $a,c$ are scaling constants, whereas $k,k'$ are explicit positive constants which depend on the parameters of the model. When $\alpha>\theta$, the most abundant families are also the oldest ones, they have size $cN^{1-\theta/\alpha}$ and age $t-a$. When $\alpha<\theta$, the oldest families have age $(\alpha /\theta)t+a$ and tight sizes; the most abundant families have sizes $k\log(N)-k'\log\log(N)+c$ and all have age $(\theta-\alpha)^{-1}\log(t)$. When $\alpha=\theta$, the oldest families have age $kt-k'\log(t)+a$ and tight sizes; the most abundant families have sizes $(k\log(N)-k'\log\log(N)+c)^2$ and all have age $t/2$. Those informal results can be stated rigorously in expectation. Relying heavily on the theory of coalescent point processes, we are also able, when $\alpha\leq\theta$, to show convergence in distribution of the joint, properly scaled ages and sizes of the most abundant/oldest families and to specify the limits as some explicit Cox processes