Adaptive Traffic Signals: Reinforcement Learning Meets North Broadway
Azizul Hakim, Wenlu Du
Motivation
Why Use AI to control Traffic?
Traditional traffic signal systems, like those on North Broadway Street, Saratoga Springs, rely on fixed timings, struggling with dynamic traffic scenarios. Reinforcement Learning (RL) adapts in real-time, optimizing flow by learning from traffic patterns. Our RL-based approach compares adaptive strategies, targeting reduced queue lengths, resulting in decreased congestion and enhanced travel efficiency
MARKOV DECISION PROCESS
ENVIRONMENT SETUP
state
The state is a list of all cars inside the lane detectors + the current traffic light phase
In this example our state is (2, 2, 2, 1, 0, 0, 2)
Here each of the first three lane detectors could fit upto two cars and the current traffic signal phase is phase 2
action
The slideshow above shows the available six traffic light phases (click to view).
Action is either 0 or 1. Action 1 switches it to the next phase.
reward
Reward = -(queue length)
The negative queue length penalizes the agent as it learns over the training period
Algorithms Used
Q-Learning
tabular Q-learning, manual table updates
PPO
on-policy, policy gradient method
DQN
target network, experience replay buffer
Fixed Time
data gathered from the traffic intersection itself
Simulation
Results
RL Creates Difference
Model training period demonstrate decreasing queue lengths showcasing benefits of adaptive learning
Compared to the static baseline Fixed-Time, which remains unchanged, PPO, Q-Learning, and DQN achieve reductions, with PPO leading
RL-based methods improve queue lengths under light and idle demand scenarios, but under heavy congestion, they fail to outperform the Fixed-Time baseline
Model training period demonstrate decreasing queue lengths showcasing benefits of adaptive learning. Compared to the static baseline Fixed-Time, which remains unchanged, PPO, Q-Learning, and DQN achieve reductions, with PPO leading
PROJECT IMPROVEMENT
FUTURE WORK
Hyperparameter Tuning to get the best possible model for each of the Deep RL algorithms.
MDP Engineering to optimize state, action and reward functions properly. Current MDP model is the simplest version.
Demand Engineering to make traffic simulation more realistic among different vehicle types. Network Engineering to add complicated turns and manage traffic light phases as done in the real world scenario.
Use SUMO additional elements most effectively (lane area detectors, bus stop waiting times and more components)
Unity and SUMO integration for realistic view of traffic flow in 3D
Train multiple agents at the same time for 1 traffic light
Train multiple agents at the same time for a network of traffic lights
Thank You
Prof. Du, my family, my friends, the computer science department
