Stochastic Control of Energy Systems: A Statistical Learning Framework

The overarching theme of this dissertation is to develop algorithms to efficiently solve finite horizon stochastic optimal control (SOC) problems. These problems naturally arise in the context of microgrid management where a controller is trying to optimally dispatch diesel generator or battery storage to maintain reliable supply of power. A popular approach is to formulate the microgrid management as a deterministic optimal control (DOC) and solve it using mixed integer linear program. However, this formulation fails to incorporate the stochasticity in the models and crucially relies on linearization of the objective/constraints. As a result, we formulate it as a SOC and consider two variants of it, first with explicit constraints on no-blackouts and second with implicit constraints on the probability of blackouts.We investigate Regression Monte Carlo (RMC), a simulation based approach to recursively solve the Bellman's dynamic programming equation (DPE) for SOC. The proposed algorithms convert the SOC into a recursive sequence of statistical learning tasks. In addition to estimating the conditional expectation encapsulated in the Bellman's DPE, they also find the set of admissible controls for probability constrained SOC. One of our main contributions is the bridge between statistical learning and numerical methods for SOC. The algorithms presented in this dissertation also generalize existing approaches within the RMC paradigm, provide additional features for efficient implementation and extends RMC to include probabilistic constraints. Besides microgrid management, we also benchmark the performance of the algorithms for the valuation of natural gas storage. In the final part of this dissertation we study the link between electricity tariffs and reliability of the distribution network. We assume the consumers at each node in the distribution network invest in behind-the-meter resources such as photovoltaic (PV) system and electrical storage. An industry model, Distributed Energy Resources-Customer Adoption Model (DER-CAM), based on DOC is used to compute the optimal size of investments and dispatch of PV and storage. We use PG&E 69-bus distribution network to assess several different aspects of electricity tariffs that can impact the reliability; such as homothetic change in the electricity purchase rate, change in the magnitude of the peak purchase rate, and the time-of-day of peak purchase rate. The work provides a new tool to the regulators for improving reliability of the distribution network