Data-driven Modelling and Control of Energy Flexible Residential Loads

Burning fossil fuels to meet humanity's insatiable energy demand is one of the leading drivers for anthropogenic climate change. An important means of mitigating the worst impact of climate change is therefore improving efficiency of current energy use and transitioning to a cleaner energy mix to meet the residual demand. This requires a fundamental rethink of how the energy value-chain is organized at present. A key technology to enable the sustainability transition is through employing demand side management to leverage flexibility at multiple scales. Demand side management encompasses both demand reduction and demand response of controllable or flexible loads via automatic control or user engagement. Notable examples of flexible loads at the residential building level - which forms the focus of this thesis - include electric batteries, and thermostatically controllable loads such as hot water systems. Unlike electric batteries, modelling and controlling thermostatically controllable loads (such as electric heaters, heat pumps etc.) poses a more challenging problem. However, they make up for a substantial fraction of energy demand in many European countries, and can provide energy flexibility at a far cheaper price point than electric storage. Controlling thermostatically controllable loads is hard because of the general non-existence of a model which can be used to describe their behaviour. This model is necessary to control the load in a way which is beneficial for both the user and the grid. In its absence, naive rule-based controllers are frequently employed which perform demonstrably sub-optimally. Historically, this modelling step has been carried out by human domain experts creating models using first principles, or fitting reduced order models to data observed during offline testing. However, these approaches only work in practice for large scale installations; for residential loads, it is neither feasible nor cost-effective to create individualized models. A recent solution to the control problem has been to adopt model-free reinforcement learning which does away with the need for a model by translating observations directly to (optimal) control actions. However, at its heart, this approach also requires extensive state estimation, besides being relatively non-interpretible, data-intensive and incompatible with most legacy optimization code. This thesis presents a transfer learning based framework which allows modelling of thermostatically controllable loads in a completely data-driven manner. Furthermore, by transferring knowledge across multiple domains and tasks, it substantially accelerates the learning process. The models learned this way are not only interpretible, they are also data-efficient and compatible with many existing optimizers. Through trials conducted on both simulated and real world hot water systems, this thesis shows that (observation) time is largely interchangable with agency. In other words, by increasing the number of thermostatically controllable systems under consideration, it is possible to vastly improve the modelling accuracy while reducing the time period required to achieve it. This effect extends to both homogeneous and heterogeneous systems, although the optimal mechanism to achieve it is different in either case. The thesis also shows that models learned using transfer learning can be used to provide a number of other services, unlike with model-free reinforcement learning. Foremost amongst these is the possibility to optimize the operation of the hot water system (the form of thermostatically controllable loads considered in this thesis) according to a number of different objectives. This thesis focuses on optimizing the energy efficiency of hot water systems and demonstrates energy savings of around 20%, or over 200 kWh per household annually, for 53 Dutch households in a trial that lasted an entire year. These results were obtained by combining the learned model with an extremely simple controller. By making use of more sophisticated controllers, it is possible to further increase these savings to almost 300 kWh per household annually, while also making it possible to achieve other objectives, such as maximal consumption of local solar generation or cost optimization etc. It is also possible to make use of the developed models for diagnostic purposes. An example is to estimate the flexibility inherent to the hot water systems, and their possible impact on the future electricity grid. In doing so, it becomes possible to not only quantify the adverse effect of large scale heat electrification, but also to what degree the installed thermostatically controllable loads can be part of the solution. Finally, the model also enables other services, such as engaging users to further reduce their energy demand and diagnosing operational faults. Developing an interpretible, data-driven model also paves the way to close the loop between operational and design optimization. The thesis concludes by taking a step back from demonstrating demand reduction and response in countries with reliable grids, and considers the case of developing countries where daily electricity outages are symptomatic of an unreliable electric grid today. Unlike in developing countries, these outages have led to mass adoption of (inefficient) electric storage. By considering Pakistan as an exemplar for such developing countries, the thesis presents an estimate of the flexibility inherent in existing electric storage, and discusses potential methods to utilize it optimally by contrasting the value creation ecosystem with developed countries. At its heart, the thesis shows that it is possible to achieve demand reduction and response even in extremely efficient modern buildings. The efficiency gains achieved in this way are enabled by the underlying energy flexibility, which can also be used to provide other services. Such a data-driven approach not only holds the potential to substantially reduce the carbon footprint of the building stock, it also paves the way for future research into developing synergies between various facets of design and operational optimization in buildings.status: publishe