Opportunistic bandwidth sharing through reinforcement learning

Graduation date: 2011The enormous success of wireless technology has recently led to an explosive demand for, and hence a shortage of, bandwidth resources. This expected shortage problem is reported to be primarily due to the inefficient, static nature of current spectrum allocation methods. As an initial step towards solving this shortage problem, Federal Communications Commission (FCC) opens up for the so-called opportunistic spectrum access (OSA), which allows unlicensed users to exploit unused licensed spectrum, but in a manner that limits interference to licensed users. Fortunately, technological advances enabled cognitive radios, which are viewed as intelligent communication systems that can self-learn from their surrounding environment, and auto-adapt their internal operating parameters in real-time to improve spectrum efficiency. Cognitive radios have recently been recognized as the key enabling technology for realizing OSA. In this work, we propose a machine learning based scheme that exploits the cognitive radios’ capabilities to enable effective OSA, thus improving the efficiency of spectrum utilization. Specifically, we formulate the OSA problem as a finite Markov Decision Process (MDP), and use reinforcement learning (RL) to locate and exploit bandwidth opportunities effectively. Simulation results show that our scheme achieves high throughput performance without requiring any prior knowledge of the environment’s characteristics and dynamics