Running Head: Numerical Approximations for Games

The Markov chain approximation method is a widely used and efficient family of methods for the numerical solution of many types of stochas-tic control problems in continuous time, for reflected-jump-diffusion-type models. It converges under broad conditions. It has been extended to zero-sum stochastic differential games. We apply the method to a class of non-zero stochastic differential games with a diffusion system model where the controls for the two players are separated in the dynamics and cost function. There have been successful applications of the algorithms, but convergence proofs have been lacking. It is shown that equilibrium values for the approximating chain converge to equilibrium values for the orig-inal process and that any equilibrium value for the original process can be approximated by an -equilibrium for the chain for arbitrarily small > 0. The numerical method solves a stochastic game for a finite-state Markov chain