82.ON AN APPLICATION OF MARKOV CHAIN TO NUMERICAL ANALYSIS

We treat a special type of Markov chain with a finite state space. This type of Markov chain often appears in traffic theory, especially in the theoretical model of the multi-channel telecommunication via some electronic device, say satellite, for which there is an upper limit of the amount of information transmitted per unit time. In such an application we are interested in its stationary state, that is the invariant measure of the Markov chain. We propose an iteration procedure to compute the invariant measure numerically by a digital computer. The original Markov chain has a continuous time parameter. Our recipe is to introduce another Markov chain having the same invariant measure but with a discrete time parameter. It can be known that for any initial distribution the distribution of the latter chain converges to its invariant measure, therefore iteration of multiplying the transition probability matrix to some initial distribution vector is enough to get the invariant measure. This can be easily undergone numerically by a digital computer. Compared with the conventional method to solve a system of linear equations, our method uses far less working area of the computer memory and this is a great advantage, especially in the case when the number of the states is very large