Formal Language Constraints in Deep Reinforcement Learning for Self-Driving Vehicles

In recent years, self-driving vehicles have become a holy grail technology that, once fully developed, could radically change the daily behaviors of people and enhance safety. The complexities of controlling a car in a constantly changing environment are too immense to directly program how the vehicle should behave in each specific scenario. Thus, a common technique when developing autonomous vehicles is to use reinforcement learning, where vehicles can be trained in simulated and real-world environments to make proper decisions in a wide variety of scenarios. Reinforcement learning models, however, have uncertainties in how the vehicle acts, especially in a previously unseen situation that can lead to dangerous situations with humans onboard or nearby. To improve the safety of the agent, we propose formal language constraints that augment a standard reinforcement learning agent while being trained in a simulated self-driving environment. The constraints help the vehicle navigate turns and other situations by penalizing the agent when an action is chosen that could lead to a dangerous situation such as a collision. Empirically, we show that the agent, with these constraints, has a slight performance improvement as well as a significant decrease in collisions. Future work can expand upon the current constraints and evaluate using different reinforcement learning algorithms with constraints for training the self-driving agent. Adviser: Stephen Scot