Electric Power Market Modeling with Multi-Agent Reinforcement Learning

Agent-based modeling (ABM) is a relatively new tool for use in electric power market research. At heart are software agents representing real-world stakeholders in the industry: utilities, power producers, system operators, and regulators. Agents interact in an environment modeled after the real-world market and underlying physical infrastructure of modern power systems. Robust simulation laboratories will allow interested parties to stress test regulatory changes with agents motivated and able to exploit any weaknesses, before making these changes in the real world. Eventually ABM may help develop better understandings of electric market economic dynamics, clarifying both delineations and practical implications of market power. The research presented here builds upon work done in collateral fields of machine learning and computational economics, as well as academic and industry literature on electric power systems. We build a simplified transmission model with agents having learning capabilities, in order to explore agent performance under several plausible scenarios. The model omits significant features of modern electric power markets, but is able to demonstrate successful convergence to stable profit-maximizing equilibria of adaptive agents competing in a quantity-based, available capacity model