Transient Analysis of some rewarded Markov Models using Randomization with Quasistationarity Detection

Abstract—Rewarded homogeneous continuous-time Markov chain (CTMC) models can be used to analyze performance, dependability and performability attributes of computer and telecommunication systems. In this paper, we consider rewarded CTMC models with a reward structure including reward rates associated with states and two measures summarizing the behavior in time of the resulting reward rate random variable: the expected transient reward rate at time t and the expected averaged reward rate in the time interval 0; t. Computation of those measures can be performed using the randomization method, which is numerically stable and has good error control. However, for large stiff models, the method is very expensive. Exploiting the existence of a quasistationary distribution in the subset of transient states of discrete-time Markov chains with a certain structure, we develop a new variant of the (standard) randomization method, randomization with quasistationarity detection, covering finite CTMC models with state space S [ ff1; f2;...; fAg, A 1, where all states in S are transient and reachable among them and the states fi are absorbing. The method has the same good properties as the standard randomization method and can be much more efficient. We also compare the performance of the method with that of regenerative randomization. Index Terms—Rewarded continuous-time Markov chains, transient analysis, randomization, quasistationary distribution