Markov reward models and hyperbolic systems

This thesis attempts to bring together two different approaches to the modeling of event driven systems based on continuous-time Markov reward models and discrete-time Markov reward models.^ In the continuous-time approach, we will show that the distribution of the accumulated reward over a specified time interval, termed performability, is the solution of a system of either forward or adjoint linear hyperbolic partial differential equations. We also show that the moments of performability satisfy a recursive set of ordinary differential equations. Our approach provides a unified framework to interpret and extend existing numerical and analytical solutions to the distribution of cumulative operational time and performability, as well as as a vehicle to derive asymptotic results.^ In the discrete-time case, this work extends the results of continuous-time approach to discrete-time and generalizes the existing literature on discrete-time Markov models. Recursions for the computation of density and moments of the cumulative reward function are derived, and their asymptotic properties are studied. Three different reward structures are considered: (i) deterministic rewards, (ii) random rewards, and (iii) multiple or vector rewards. For all these cases, manufacturing examples are presented with numerical results as well as closed form expressions in several cases.