Optimization of Traffic Signal Control Using Reinforcement Learning: A SUMO-Based Simulation Study on a Real-World Example

Abstract

Within this project, a system to train a model with reinforcement learning to control traffic lights was implemented. Two different learning algorithms have been evaluated and compared with a vehicle-actuated logic. The investigation area is an actual intersection in the city of Münster, and it was possible to use the currently used logic and recorded traffic demand as a basis for the simulation, as well as for the final comparison. Experiments performed with both used learning methods, DQN and PPO, show the effect of different initialisations and result in a final configuration used for a comparison of the methodologies and the vehicle-actuated baselines logic. For DQN, a hyperparameter configuration to train on the basis of one hour of traffic data was found and achieved an average advantage in waiting time of 48.64% when applying it to the traffic data of 24 hours. A model trained with PPO on the identical segment of traffic data achieved an average advantage in waiting time of 57.73% when applying it to the full day. And after training the model with PPO with the whole simulation, an average advantage of 65.15% was achieved compared to the waiting time of the vehicle-actuated logic. Besides the achieved improvements through training traffic controllers with reinforcement learning instead of using a vehicle-actuated logic, there is potential for further improvement left for future work.