Distributed dynamic resource management and pricing in the IoT systems with blockchain-as-a-service and UAV-enabled mobile edge computing

In this article, we study the pricing and resource management in the Internet of Things (IoT) system with blockchain-as-a-service (BaaS) and mobile-edge computing (MEC). The BaaS model includes the cloud-based server to perform blockchain tasks and the set of peers to collect data from local IoT devices. The MEC model consists of the set of terrestrial and aerial base stations (BSs), i.e., unmanned aerial vehicles (UAVs), to forward the tasks of peers to the BaaS server. Each BS is also equipped with an MEC server to run some blockchain tasks. As the BSs can be privately owned or controlled by different operators, there is no information exchange among them. We show that the resource management and pricing in the BaaS-MEC system are modeled as a stochastic Stackelberg game with multiple leaders and incomplete information about actions of leaders/BSs and followers/peers. We formulate a novel hierarchical reinforcement learning (RL) algorithm for the decision makings of BSs and peers. We also develop an unsupervised hierarchical deep learning (HDL) algorithm that combines deep Q -learning (DQL) for BSs with the Bayesian deep learning (BDL) for peers. We prove that the proposed algorithms converge to stable states in which the peers' actions are the best responses to optimal actions of BSs.Agency for Science, Technology and Research (A*STAR)Ministry of Education (MOE)National Research Foundation (NRF)This work was supported in part by the National Natural Science Foundation of China under Project 61950410603; in part by the Singapore NRF National Satellite of Excellence, Design Science and Technology for Secure Critical Infra-Structure NSoE under Grant DeST-SCI2019-0007; in part by the A*STAR-NTUSUTD Joint Research Grant Call on Artificial Intelligence for the Future of Manufacturing RGANS 1906, WASP/NTU M4082187 (4080); in part by the Singapore MOE Tier 1 under Grant 2017-T1-002-007 RG122/17 and MOE Tier 2 under Grant MOE2014-T2-2-015 ARC4/15; in part by the Singapore NRF under Grant 2015-NRF-ISF001-2277; and in part by the Singapore EMA Energy Resilience under Grant NRF2017 EWT-EP003-041