An Embedded Link MDP Model with Application to Call Admission Control

Abstract. In this paper we study the call admission control (CAC) problem for a single link in multi-service loss networks. Each call is described by a reward parameter representing the expected reward for carrying this call. The control objective is to maximize the reward from carried calls. The behavior of the link is modeled as a Markov Decision Process (MDP). The standard link MDP model assumes a Poisson call arrival process and exponentially distributed call holding times. How-ever, some services on the Internet, such as the World Wide Web ser-vice, have self-similar call arrival processes and heavy-tailed holding time distributions. In this paper we assume a simple call traffic model consisting of a single call category with general renewal arrivals and ex-ponential holding time distribution. We present an extended link MDP model based on an embedded controlled Markov chain for this traffic model. The numerical results show that the proposed link MDP model provides accurate estimates of the reward loss.