Deep reinforcement learning for D2D transmission in unlicensed bands

In this paper, a reinforcement learning based approach is proposed to realize the distributed power and spectrum allocation for the Device-to-Device (D2D) communications in unlicensed bands, named as D2D-U. To guarantee the harmonious coexistence with the WiFi networks, the conventional duty-cycle muting (DCM) is employed by the D2D-U links. With the proposed learning approach, D2D-U links can optimally select the time fraction on unlicensed channels without knowing the accurate WiFi traffic in a dynamic WiFi working environment. To address the state space explosion during the learning process, the Deep Q-learning network (DQN) is adopted by combining a deep neural network (DNN) with the traditional Q-learning mechanism. After obtaining the available time fraction on unlicensed channels, the spectrum and power allocation on licensed and unlicensed bands can be optimized jointly via the classic convex optimization methods at each D2D-U link. Numerical results are demonstrated to verify the effectiveness of the proposed approach.