Fixed point theorems for discounted finite markov decision processes

We establish the existence of a solution to the optimality equation for discounted finite Markov decision processes by means of Birkhoff's fixed point theorem. The proof yields the well-known linear programming formulation for the optimal value function while its dual characterizes the optimal value function as the maximum over all value functions