Stateless Reinforcement Learning for Multi-Agent Systems: the Case of Spectrum Allocation in Dynamic Channel Bonding WLANs

Spectrum allocation in the form of primary channel and bandwidth selection is a key factor for dynamic channel bonding (DCB) wireless local area networks (WLANs). To cope with varying environments, where networks change their configurations on their own, the wireless community is looking towards solutions aided by machine learning (ML), and especially reinforcement learning (RL) given its trial-and-error approach. However, strong assumptions are normally made to let complex RL models converge to near-optimal solutions. Our goal with this paper is two-fold: justify in a comprehensible way why RL should be the approach for wireless networks problems like decentralized spectrum allocation, and call into question whether the use of complex RL algorithms helps the quest of rapid learning in realistic scenarios. We derive that stateless RL in the form of lightweight multi-armed-bandits (MABs) is an efficient solution for rapid adaptation avoiding the definition of extensive or meaningless RL states.The work of Sergio Barrachina-Munoz and Boris-Bellalta was supported in part by Cisco, WINDMAL under Grant PGC2018099959-B-I00 (MCIU/AEI/FEDER,UE) and Grant SGR-2017-1188. Alessandro Chiumento is partially funded by the InSecTT project (https://www.insectt.eu/) which has received funding from the ECSEL Joint Undertaking (JU) under grant agreement No 876038