A Novel Safe Deep Reinforcement Learning Approach for Optimal Dispatch of Energy Hubs with Compressed Air Energy Storage

The development of renewable energy and energy storage technologies has resulted in the emergence of Energy Hubs (EHs) in recent years. Due to the uncertainty associated with energy supply and load, scheduling EH presents a challenging task. Current model-based optimization approaches have limitations in terms of solution accuracy and computational efficiency, which hamper their application. Deep Reinforcement Learning (DRL) is a model-free approach that has demonstrated superior performance over model-based approaches. The current DRL algorithms, however, perform poorly in terms of constraint handling and global optimality. The purpose of this study is to propose a model-free, safe deep reinforcement learning approach, combining primal-dual optimization and imitation learning, for the optimal scheduling of an EH with an Advanced Adiabatic Compressed Air Energy Storage (AA-CAES). First, the operation of an AA-CAES under off-design conditions is modeled and linearized using Mixed Integer Linear Programming (MILP). Then, a safe DRL approach is proposed with training and testing steps considering a case study. The performance of the proposed approach in reducing operational cost and satisfying constraints is compared to state-of-the-art DRL algorithms as well as a deterministic MILP approach. Additionally, a test set is used to examine the generalizability of the proposed approach. Finally, the effect of off-design conditions of a tri-generative AA-CAES on the optimal dispatch strategy is investigated. Furthermore, a sensitivity analysis indicates that the proposed approach is reproducible and reliable. The results indicate that the proposed approach can effectively reduce the operational cost and satisfy the operational constraints