Markov Decision Processes with Applications to Finance Outline I Markov Decision Processes with Finite Time Horizon I Definition I Basic Results I Financial Applications I Markov Decision Processes with Infinite Time Horizon I Definition I Basic Results I

Let (Xn) be a Markov process (in discrete time) with I state space E, I transition kernel Qn(·|x). Let (Xn) be a controlled Markov process with I state space E, action space A, I admissible state-action pairs Dn ⊂ E × A, I transition kernel Qn(·|x,a). A decision An at time n is in general σ(X1,...,Xn)-measurable. However, Markovian structure implies An = fn(Xn) is sufficient. Markov Decision Processes with Applications to Finance MDPs with Finite Time Horizo