Studying Dynamic Pricing in Electrical Power Markets with Distributed Generation: Agent-Based Modeling and Reinforcement-Learning Approach

Distributed generation (DG) refers to small-scale generation resources that are located at or near end-consumers, such as photovoltaic (PV) solar systems. DG systems have become increasingly popular in recent years owing to their economic efficiency, reliability, and sustainability. However, the increasing adoption of DG is creating new obstacles for system operators due to the uncertainty in forecasting future demand. One concern is the possibility of facing a utility death spiral as a feedback loop between, on the one hand, the increasing adoption of DG and increasing electricity rates to cover generation and transmission overheads with, on the other hand, reduced demand from the grid. The goal of this study was to investigate the effects of the penetration of DG on the power infrastructure and wholesale power markets considering the dynamic pricing of electric power. To achieve that goal, a complex system-of-systems (SoS) simulation for wholesale power markets and infrastructure is developed using agent-based modeling (ABM), Optimal Power Flow (OPF), and reinforcement learning (RL). The simulation framework is enabled by (1) consumer behavior that compares the benefits of installing DG versus the costs of conventional power from the grid; and (2) the dynamic pricing of electric power. Several RL algorithms are compared, including basic RL, multiplicative RL, Roth-Erev RL, a modified Roth-Erev, and a variation of the aforementioned algorithms using a Gibbs-Boltzmann cooling factor by means of a grid of learning parameters relevant to each technique. The results show that (1) low-cost generators, such as nuclear power plants, are the least affected by the penetration of DG, while high-cost generators are the most affected, (2) a utility death spiral is unlikely to occur, and (3) a modified Roth-Erev RL algorithm can be used by generating companies to maximize their gross profits by optimizing their supply curves considering the feedback effect on DG adoption rates. Overall, the proposed simulation framework can assist policymakers and future researchers in studying the interaction between dynamic pricing in wholesale power markets and the adoption of DG. By simulating the long-term effect of DG on the electric power infrastructure and market, policymakers can introduce regulations and incentives to strategically influence the rate of adoption of DG