Combining Learning and Control in Cyber-Physical Systems

Presented on January 25, 2023 at 12:15 p.m. in the Marcus Nanotechnology Building, Room 1116.Dr. Andreas Malikopoulos is the Terri Connor Kelly and John Kelly Career Development Associate Professor in the Department of Mechanical Engineering and the Director of the Sociotechnical Systems Center at the University of Delaware (UD). His research interests span several fields, including analysis, optimization, and control of cyber-physical systems; decentralized stochastic systems; stochastic scheduling and resource allocation; and learning in complex systems.Runtime: 56:17 minutesCyber-physical systems (CPS), in most instances, represent systems of subsystems with an informationally decentralized structure. To derive optimal control strategies for such systems, we typically assume an ideal model, e.g., controlled transition kernel. Such model-based control approaches cannot effectively facilitate optimal solutions with performance guarantees due to the discrepancy between the model and the actual CPS. On the other hand, in most CPS there is a large volume of data of a dynamic nature which is added to the system gradually in real time and not altogether in advance. Thus, traditional supervised learning approaches cannot always facilitate robust solutions using data derived offline. By contrast, applying reinforcement learning approaches directly to the actual CPS might impose significant implications on safety and robust operation of the system. In this talk, I will present a theoretical framework founded at the intersection of control theory and learning that circumvents these challenges in deriving optimal strategies for CPS. In this framework, we aim at identifying a sufficient information state for the CPS that takes values in a time-invariant space, and use this information state to derive separated control strategies. Separated control strategies are related to the concept of separation between the estimation of the information state and control of the system. By establishing separated control strategies, we can derive offline the optimal control strategy of the system with respect to the information state, which might not be precisely known due to model uncertainties or complexity of the system, and then use learning methods to learn the information state online while data are added gradually to the system in real time. This approach could effectively facilitate optimal solutions with performance guarantees in a wide range of CPS applications such as emerging mobility systems, networked control systems, smart power grids, cooperative cyber-physical networks, cooperation of robots, and internet of things