A reinforcement learning-based energy-efficient spectrum-aware clustering algorithm for cognitive radio wireless sensor network

Energy efficiency and spectrum efficiency are two main challenges in the realization of Cognitive Radio-Wireless Sensor Network (CR-WSN). Clustering is a well-known technique that could be used to achieve energy efficient communication and to enhance dynamic channel access in cognitive radio through cooperative sensing. While the energy efficiency issue has been well investigated in conventional wireless sensor networks, the latter has not been extensively explored. In this thesis, a Reinforcement Learning (RL) based clustering algorithm is proposed to address energy and Primary Users (PUs) detection challenges in CR-WSN. The scheme minimizes network energy consumption, improves channel utilization and enhances PUs detection performance from three different perspectives. Firstly, a RL based spectrum-aware clustering scheme in which a cluster member node learns energy and cooperative sensing costs for neighbouring clusters through exploration and imposes pairwise constraint to select optimal cluster. The optimal cluster minimizes network energy consumption and enhances channel sensing performance. Secondly, a weighted hard combining scheme that combines features of both quantized and hard combining schemes to minimize energy cost for reporting sensing result and improve PU detection performance. Thirdly, a RL based cooperative channel sensing scheme where a clusterhead learns channels dynamic behaviours in terms of channel availability, channel sensing energy cost and channel impairment to achieve optimal sensing sequence and optimal set of channels. Simulation results show convergence, learning and adaptability of the RL based algorithms to dynamic environment toward achieving the optimal solutions. Performance comparisons of the RL based clustering scheme with Groupwise spectrum-aware clustering scheme show that an energy savings of9% and PU detection performance improvement of 11.6% can be achieved. Similarly, the results indicate that the proposed fusion scheme minimizes reporting energy cost by 70% and improves detection performance by 5.6\% compared to the quantized 3-bits scheme. Furthermore, the results show that with the RL based channel sensing scheme, a sensing energy cost savings of 15.14% per channel sensing cycle can be achieved while improving PU detection accuracy and channel utilization compared to the Greedy search approach. The overall result indicates viability and improved performance from the RL based scheme over the other bench mark schemes in terms of energy efficiency and PU detection performance which are vital to resource constraint devices