Participation of distributed loads in power markets that co-optimize energy and reserves

Thesis (Ph.D.)--Boston UniversityAs the integration of Renewable Generation into today's Power Systems is progressing rapidly, capacity reserve requirements needed to compensate for the intermittency of renewable generation is increasing equally rapidly. A major objective of this thesis is to promote the affordability of incremental reserves by enabling loads to provide them through demand response. Regulation Service (RS) reserves, a critical type of bi-directional Capacity Reserves, are provided today by expensive and environmentally unfriendly centralized fossil fuel generators. In contrast, we investigate the provision of low-cost RS reserves by the demand-side. This is a challenging undertaking since loads must first promise reserves in the Hour Ahead Markets, and then be capable of responding to the dynamic ISO signals by adjusting their consumption effectively and efficiently. To this end, we use Stochastic Control, Optimization Theory, and Approximate Dynamic Programming to develop a decision support framework that assists Smart Neighborhood Operators or Smart Building Operators (SNOs/SBOs) to become demand-side-providers of RS reserve. We first address the SNO/SBO short time scale operational task of responding to the Independent System Operator's (ISO) dynamic RS requests. We start by developing a model-based Markovian decision problem that trades off ISO RS tracking against demand response related utility loss. Starting with a model based approach we obtain near optimal operational policies through a novel approximate policy iteration technique and an actor critic approach which is robust to partial knowledge of the underlying system dynamics. We then abandon the model based terrain and solve the dynamic operational problem through reinforcement learning that is capable of modeling a population of duty cycle appliances with realistic thermodynamics. We finally propose a smart thermostat design and develop an adaptive control policy that can drive the smart thermostat effectively. The latter approach is particularly suited for systems whose dynamics and dynamically changing consumer preferences are not known or observed beyond the total power consumption. We then address the SNO/SBO task of bidding RS reserves to the hour ahead market. This task determines the maximal RS reserves that the SNO/SBO can promise based on information available at the beginning of an hour, so as to maximize the associated hour-ahead revenues minus the expected average operating cost that will be incurred during the operational task to follow. To accomplish this task, we (i) develop probabilistic constraints that model the feasible maximum reserves which can be offered to the market without exceeding the SNO/SBO's ability to later track the unanticipated dynamic ISO RS signal, and (ii) calibrate a describing function that approximates the average operational cost as a function of the maximal reserves that can be feasibly offered in the day ahead market. The above is made possible by statistical analysis of the controlled system's stochastic dynamics and properties of the optimal dynamic policies that we derive. The contribution of the thesis is twofold: The solution of a difficult stochastic control problem that is crucial for effective demand-response-based provision of regulation service, and, the characterization of key properties of the stochastic control problem solution, which allow its integration into the hour-ahead market bidding problem