Stationary Markov Perfect Equilibria in Discounted Stochastic Games

The existence of stationary Markov perfect equilibria in stochastic games is shown in several contexts under a general condition called "coarser transition kernels". These results include various earlier existence results on correlated equilibria, noisy stochastic games, stochastic games with mixtures of constant transition kernels as special cases. The minimality of the condition is illustrated. The results here also shed some new light on a recent example on the nonexistence of stationary equilibrium. The proofs are remarkably simple via establishing a new connection between stochastic games and conditional expectations of correspondences