Empowering peer-to-peer energy trading in smart grid via deep reinforcement learning

Electricity is traditionally generated in large, centralised power plants, resulting in high transmission costs and high emissions. Recently, small-scale renewable generation has become more and more popular due to the low carbon energy policy. Microgrids (MG) have been employed to address the challenges arising from the presence of a high share of distributed energy resources in local regions of modern energy systems. Furthermore, the peer-to-peer (P2P) energy trading paradigm with its improved system efficiency and reduced greenhouse gas emissions benefits the MGs more than traditional energy trading strategies. Considering the intermittent nature of renewable generation and hard-predicted local consumption patterns, a P2P energy trading system must cope with uncertainty, scale, and reliability issues in real-time. In order to ensure a fast and optimised energy trading operation and settlement, an automated decision-making system is necessary. However, traditional optimisation methodologies may not be able to produce results in a real-time manner when dealing with large data sets and an increased level of uncertainty in P2P energy trading schemes. Recent energy coupling technologies can be integrated by different power carriers to form a multi-energy microgrid (MEMG), resulting in economic and environmental benefits. A MEMG consists of DERs, energy coupling technologies, local active loads and energy storage systems (ESSs). By connecting multiple MEMGs, the distribution network can be made more efficient and reliable. In addition to the challenges posed by the intermittent nature of DERs, there are also additional obstacles related to the stability and operational safety of the network of multiple MEMGs that vary according to the deployment, including the size and type of DERs. To address the above challenges, this thesis utilises Deep reinforcement learning (DRL), as a decision-making learning algorithm, to automatically derive optimal P2P energy trading policies for MGs participating in a local energy trading market. Furthermore, this thesis investigates the external P2P energy trading problem and internal energy conversion problem within interconnected residential, commercial and industrial MEMGs. The problem is solved by a novel multi-agent deep reinforcement learning (MADRL) method. Finally, this thesis studies a P2P energy trading and energy conversion framework based on the highly efficient double auction (DA) market. A novel DA-MADRL method is proposed, which not only inherits the ability of MADRL to perform well in a multi-agent environment with various uncertainties and also addresses privacy concerns of the MEMGs