Smart Charging of Electric Vehicles: an Autonomous Driving Perspective

EVA Deliverable No. 4.1 - Advanced charge-scheduling algorithms for EVs and EVAs and forecastingElectrification of the transportation sector has a range of implications related to the environment, climate change, business, and economy. While there is the political will to support the growth of electric vehicles (EV), the charging process of large populations of EVs is challenging because the existing electrical infrastructure, and in particular power distribution grids, might not be capable of meeting increased levels of power demand.In the existing literature, smart charging algorithms have been widely investigated to resolve the issues related to the simultaneous charge of many EVs. This report takes the perspective of autonomous driving, which is expected to revolutionize road transport. In particular, this report explores how autonomous EVs (AEVs) could facilitate better management of the charging process, thanks to the innovations in data-driven control methods for making smart charging decisions. We present two architectures for smart charging for AEVs, a model-based controller and a model-free stochastic reinforcement learning (RL) controller, enabling grid-friendly and optimal integration of EVs and AEVs in a smart grid.The model-based approach leverages an optimal power flow (OPF) to schedule the charging process of AEVs. Compared to OPF-based smart charging algorithms for standard EVs where the charging location is totally determined by the location where the vehicles have been parked by their drivers, AEVs can change location autonomously and choose a more suitable charging spot for the power grid.It is shown how this problem can be formulated as a mixed-integer linear problem (MILP), levering linearized load flow models that can be solved with off-the-shelf optimization libraries. It is shown that the proposed scheduler for AEVs can AEVs can achieve significantly better congestion managementthan traditional EVs, postponing grid reinforcement.The second proposed approach uses a stochastic RL controller. The foremost opportunity to use stochastic RL is that a smart-grid environment is typically partially observable. That is to say, due to technical and investment limitations, it is challenging to observe the state of the smart grid in full for control purposes. Moreover, the uncertainties in a smart grid environment affect the controller’s optimality.Designing a stochastic RL controller is a challenging task that requires careful feature engineering, state abstractions, reward engineering, and hyperparameter tuning. However, we can easily roll out the trained stochastic RL agent for optimal charging management in a decentralized control architecture after training. As we show in the case studies, stochastic RL agents effectively find the optimal control policy based on the trade-offs in the reward function under stochastic conditions and imperfect information. Furthermore, the actor-critic architecture presented in the report has the advantage of scalability. For example, the second case study uses an architecture with one critic and many actors.It is also possible to build architectures with many critics and many actors for cases even larger. On the other hand, stochastic RL agents have difficulty handling hard constraints, which are essential for safety-critical applications. However, through proper training methods, it is possible to minimize the chance of constraint violations. Finally, we see that both model-based and model-free controllers have their advantages and unique place in a smart grid for optimal charging management. From a futuristic viewpoint, it is vital to understand the unique advantages and disadvantages of model-based and model-free controllers, their unique use cases, and even opportunities for hybrid mechanisms that build upon the strengths of both strategies