Multi-agent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers (MARLIN-ATSC)

The population is steadily increasing worldwide resulting in intractable traffic congestion in dense urban areas. Adaptive Traffic Signal Control (ATSC) has shown strong potential to effectively alleviate urban traffic congestion by adjusting the signal timing plans in real-time in response to traffic fluctuations to achieve the desired objectives (e.g., minimizing delay). Efficient and robust ATSC can be designed using a multi-agent reinforcement learning (MARL) approach in which each controller (agent) is responsible for the control of traffic lights around a single traffic junction. Applying MARL approaches to ATSC problem is associated with a few challenges as agents typically react to changes in the environment at the individual level but the overall behaviour of all agents may not be optimal. This dissertation presents the development and evaluation of a novel system of Multi-Agent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers (MARLIN-ATSC). The MARLIN-ATSC control system is developed to provide a self-learning ATSC using a synergetic combination of reinforcement learning approaches and game theory. MARLIN-ATSC operates in two modes: (1) independent mode, i.e. each intersection controller operates independently of other agents; and (2) integrated mode, where each controller coordinates the signal control actions with the neighbouring intersections. The system was tested on three networks (i.e., small, medium, large-scale) to ensure seamless transferability of the system design and results. The large-scale application was conducted on a computerized testbed network of 60 intersections in the lower downtown core of the City of Toronto for the morning rush hour handling 25,000 trips. The results show unprecedented reduction in the average intersection delay ranging from 27% in mode 1 to 39% in mode 2 at the network level; and travel time savings of 15% in mode 1 and 26% in mode 2, along the busiest routes in downtown Toronto. The thesis shows how mathematical modelling of the traffic control problem as a stochastic control problem, combined with the utilisation of artificial intelligence techniques such as reinforcement learning in a game-theory setup, can provide highly useful and economically inexpensive solutions to real-life problems such as urban traffic congestion.Ph