Stat205B: Probability Theory (Spring 2003) Lecture: 6 Finite State Continuous Time Markov Chain

See Durrett Sec 5.6 for the theory of discrete time recurrent Markov Chains with uncountable state space, as developed following Harris. The general idea is to recognize a suitable regenerative struc-ture, like what happens to a discrete time, discrete space Markov chain each time it comes back to a point. Then decompose the path into blocks which are i.i.d. This idea can also be applied to continuous time, discrete space chains. We now discuss a continuous time, discrete space Markov Chain, with time-homogeneous transi-tion probabilities. Let S = state space. The theory is easiest if S is nite. Some aspects can be extended to S countable. The book of Karlin and Taylor [5], provides details for most of the fol-lowing discussion. See also [3, 4] for further developments. General theory of countable state space, continuous time chains is very tricky: see Chung [1] and Freedman [2]. Suppose (Xt; t 0) is a process dened on (;F; P), with values in S which is nite. For each t, ! ! Xt(!) is a measurable map from (;F) ! S. Look at the path: t! Xt(!) for xed!. In the nite state space case, we expect this path to be almost surely a step function, with only a nite number of jumps in any nite interval of time. Make the convention that the path is right-continuous with left limits. Such a process has the time-homogeneous Markov property if conditionally given Xt, the processs (Xs; 0 s t) and (Xu; t u < 1) are independent; (Xt+v; v 0jXt = i) is distributed like (Xv; v 0)jX0 = i). Introduce the transition matrices Pt = kPt(i; j)ki;j, where Pt(i; j) = P (Xs+t = jjXs = i); s 0; i; j 2 S The denition of Pt and the time-homogeneous Markov property yield: Pt(i; j)