Computing the stationary distribution for infinite Markov chains

AbstractIn a situation where the unique stationary distribution vector of an infinite irreducible positive-recurrent stochastic matrix P is not analytically determinable, numerical approximations are needed. This paper partially synthesizes and extends work on finite-vector approximative solutions obtained from nxn northwest corner truncations (n)P of P, from the standpoints of (pointwise convergence) algorithms as n→∞, and the manner of their computer implementation with a view to numerical stability and conditioning. The problem for finite n is connected with that of finding the unique stationary distribution of the finite stochastic matrix (n)P̃ obtained from (n)P by augmenting a column