Green transmit power assignment for cognitive radio networks by applying multi-agent Q-learning approach

As the scarce spectrum resource is becoming overcrowded, cognitive wireless mesh networks (CogMesh) indicate great flexibility to improve the spectrum utilization by opportunistically accessing the authorized frequency bands. In this paper, we consider non-cooperative green power assignment in CogMesh with the consideration of energy efficiency. The problem is modeled as a stochastic learning process. We extend the single-agent Q-learning to a multi-user context, and propose a conjecture based multi-agent Q-learning scheme to obtain the optimal strategies with private and incomplete information. A learning secondary user performs Q-function updates based on the conjecture about other secondary users' behaviors. Simulations are used to verify the performance of our algorithm and demonstrate its effectiveness of improving the energy efficiency.