Dynamic Programming and Its Application to Households'

1. The problem of dynamic programming that appears often in economics takes the following form: Find fxtg and fvtg so as to maximise TÄ1X t=0 ãtf(xt; vt; t) + S(xT) subject to xt+1 = g(xt; vt; t); t = 0; 1; 2;ÅÅÅ; T Ä 1 given f, g, S, fãtg and x0. 2. For such a problem, Bellman's principle of optimality holds: The principle of optimality: An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the årst decision. [ Bellman (1957), p.83.] This principle justiåes solving the problem in the reverse order, starting from the end and going back to the beginning of the planning period. 3. The maximised present value of the objective is a function of x representing the initial state. Name it the value function. The value function of the last period problem is V (x; T) = S(x) This maximisation problem is trivial, nothing being left unknown given the initial condition, x. 4. The second from the last period problem turns out to be to ånd xT and vTÄ1 to maximise ãTÄ1f(xTÄ1; vTÄ1; T Ä 1) + V (xT; T) subject to xT = g(xTÄ1; vTÄ1; T Ä 1