Fixation probabilities in populations under demographic fluctuations

We study the fixation probability of a mutant type when introduced into a resident population. As opposed to the usual assumption of constant population size, we allow for stochastically varying population sizes. This is implemented by a stochastic competitive Lotka-Volterra model. The competition coefficients are interpreted in terms of inverse payoffs emerging from an evolutionary game. Since our study focuses on the impact of the competition values, we assume the same birth and death rates for both types. In this general framework, we derive an approximate formula for the fixation probability ϕ of the mutant type under weak selection. The qualitative behavior of ϕ when compared to the neutral scenario is governed by the invasion dynamics of an initially rare type. Higher payoffs when competing with the resident type yield higher values of ϕ. Additionally, we investigate the influence of the remaining parameters and find an explicit dependence of ϕ on the mixed equilibrium value of the corresponding deterministic system (given that the parameter values allow for its existence)