Separation of Learning and Control for Cyber-Physical Systems

Most cyber-physical systems (CPS) encounter a large volume of data with a dynamic nature which is added to the system gradually in real time and not altogether in advance. Therefore, neither traditional supervised (or unsupervised) learning nor typical model-based control approaches can effectively facilitate optimal solutions with performance guarantees. In this paper, we provide a theoretical framework that yields optimal control strategies at the intersection of control theory and learning. In the proposed framework, we use the actual CPS, i.e., the "true" CPS that we seek to optimally control online, in parallel with a model of the CPS that we have available. We then institute a sufficient information state for the system that takes values in a time-invariant space, and use this information state to derive separated control strategies. By establishing separated control strategies, we can derive offline the optimal control strategy of the system with respect to the information state, which might not be precisely known due to model uncertainties or complexity of the system, and then use standard learning approaches to learn the information state online while data are added gradually to the system in real time. We show that after the information state becomes known, the separated control strategy of the model derived offline is optimal for the actual CPS. We illustrate the proposed framework in a dynamic system consisting of two subsystems with a delayed sharing information structure.Comment: 19 pages, 2 figures. arXiv admin note: text overlap with arXiv:2101.1099