Non-Homogeneous Markov Decision Processes with Applications to R and D Planning (Forecast Horizons, Infinite Optimization, Research, Development).

This dissertation develops conditions for the existence of forecast horizons in non-homogeneous Markov Decision Processes (MDP's). Forecast horizons provide a useful means for finding the optimal first decision in non-homogeneous MDP's, where the functional equation approaches generally used for homogeneous MDP's are not applicable. In discounted problems, forecast horizons are shown to exist as a result of the diminishing present value of future rewards. In undiscounted MDP's the diminishing influence of the first decision on the probability of reaching future states is shown to act analogously to discounting and hence provides the basis for existence of forecast horizons in these problems. The theoretical forecast horizon results are then applied to the analysis of a model of sequential interfirm R and D rivalries. This model differs from existing R and D models in that it considers temporal interdependencies between R and D projects. Successful development of an innovation is assumed to offer a firm an advantage in the competition to develop a related follow-on innovation. As posed in this model, the R and D investment problem is a non-homogeneous MDP with a very special structure. This special structure is exploited to develop practical solution techniques. Economic and policy implications of the model are examined and comparisons with existing models are made.Ph.D.Operations researchUniversity of Michiganhttp://deepblue.lib.umich.edu/bitstream/2027.42/159949/1/8412160.pd