Multi-Agent Soft Actor-Critic with Graph Attention Networks for Adaptive Traffic Signal Optimization (MASAC-GAT)
DOI:
https://doi.org/10.4108/eetiot.10486Keywords:
Traffic Signal Opmisation, Reinforcement Learning, Intelligent Transportation Systems, Sustainable Urban Mobility, Graph neural Network, AttentionAbstract
INTRODUCTION: Adaptive Traffic Signal Optimisation (ATSO) is a challenging problem for urban traffic networks, having important implications for congestion reduction, traffic efficiency, and environmental conservation. Conventional traffic signal control techniques, i.e., fixed-time and rule-based control, fail to respond to dynamic traffic behaviour efficiently.
OBJECTIVES: Recent developments in Reinforcement Learning (RL) have been promising for ATSO but are plagued by poor scalability, lack of coordination in multi-intersection networks, and inefficiency in dealing with continuous action spaces.
METHODS: Furthermore, most RL-based solutions are based on simplistic state representation and fail to incorporate complex interdependencies between traffic signals. Considering these limitations, this paper introduces a new framework, Multi-Agent Soft Actor-Critic with Graph Attention Networks (MASAC-GAT), which unites the sample efficiency and stability of Soft Actor-Critic (SAC) with the relational modelling ability of Graph Attention Networks (GATs).
RESULTS: The proposed method exhibited significant performance gains on three important traffic metrics: Signal Adjustment Efficiency (92%), Average Waiting Time (20–35 seconds), and Congestion Prediction Accuracy (93%), outperforming DQL, PPO, A2C, GNN-based variants, and knowledge sharing DDPG (KS-DDPG). Through minimised redundant signal changes and reduced vehicle delays, the method ushers in the next generation of smart transportation systems.
CONCLUSION: The proposed method facilitates decentralised yet coordinated control of traffic signals by utilising local observations and global context. The proposed method unites real-time traffic observations, e.g., traffic volume, vehicle speeds, weather, accident reports, and signal status, into a customised OpenAI Gym environment for training and evaluation.
Downloads
References
[1] Paul A, Haricharan J, Mitra S. An intelligent traffic signal management strategy to reduce vehicles CO2 emissions in fog oriented VANET. Wireless Personal Communications. 2022; 122(1):543-576.
[2] Qadri SSSM, Gökçe MA, Öner E. State-of-art review of traffic signal control methods: challenges and opportunities. European transport research review. 2020; 12:1-23.
[3] Wadud Z, MacKenzie D, Leiby P. Help or hindrance? The travel, energy and carbon impacts of highly automated vehicles. Transportation Research Part A: Policy and Practice. 2016; 86:1-18.
[4] Ahmed SK, Mohammed MG, Abdulqadir SO, El‐Kader RGA, El‐Shall NA, Chandran D, ... Dhama K. Road traffic accidental injuries and deaths: A neglected global health issue. Health science reports. 2023; 6(5):e1240.
[5] Wang Y, Yang X, Liang H, Liu Y. A review of the self‐adaptive traffic signal control system based on future traffic environment. Journal of Advanced Transportation. 2018; 2018(1):1096123.
[6] Eom M, Kim BI. The traffic signal control problem for intersections: a review. European transport research review. 2020; 12:1-20.
[7] Diakaki C, Papageorgiou M, Aboudolas K. A multivariable regulator approach to traffic-responsive network-wide signal control. Control Engineering Practice. 2002; 10(2):183-195.
[8] Zhang C, Chen F, Qin R, Li X, Wang H. Intelligent traffic management and control technology. In Intelligent Road Transport Systems: An Introduction to Key Technologies. 2022; 325-398. Singapore: Springer Nature Singapore.
[9] Jin J, Ma X, Kosonen I. An intelligent control system for traffic lights with simulation-based evaluation. Control Eng Pract. 2017; 58:24–33.
[10] Göttlich S, Potschka A, Ziegler U. Partial outer convexification for traffic light optimization in road networks. SIAM Journal on Scientific Computing. 2017; 39(1):B53-B75.
[11] Jamal AM, Al-Ahmadi H, Muhammad Butt F, Iqbal M, Almoshaogeh M, Ali S. Metaheuristics for Traffic Control and Optimization: Current Challenges and Prospects. IntechOpen. 2023. doi: 10.5772/intechopen.99395.
[12] Saleem M, Abbas S, Ghazal TM, Khan MA, Sahawneh N, Ahmad M. Smart cities: Fusion-based intelligent traffic congestion control system for vehicular networks using machine learning techniques. Egyptian Informatics Journal. 2022; 23(3):417-426.
[13] Zhu Y, Cai M, Schwarz CW, Li J, Xiao S. Intelligent traffic light via policy-based deep reinforcement learning. International Journal of Intelligent Transportation Systems Research. 2022; 20(3):734-744.
[14] Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, ... Hassabis D. Human-level control through deep reinforcement learning. Nature. 2015; 518(7540):529-533.
[15] Aslani M, Mesgari MS, Wiering M. Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events. Transportation Research Part C: Emerging Technologies. 2017; 85:732-752.
[16] Wei H, Zheng G, Gayah V, Li Z. Recent advances in reinforcement learning for traffic signal control: A survey of models and evaluation. ACM SIGKDD Explorations Newsletter. 2021; 22(2):12-18.
[17] Chu T, Wang J, Codecà L, Li Z. Multi-agent deep reinforcement learning for large-scale traffic signal control. IEEE transactions on intelligent transportation systems. 2019; 21(3):1086-1095.
[18] Eom M, Kim BI. The traffic signal control problem for intersections: a review. Eur. Transp. Res. Rev. 2020; 12:50. https://doi.org/10.1186/s12544-020-00440-8.
[19] Wu H, Yan S, Liu M. Recent advances in graph-based machine learning for applications in smart urban transportation systems. arXiv preprint arXiv:2306.01282. 2023.
[20] Kolat M, Kővári B, Bécsi T, Aradi S. Multi-agent reinforcement learning for traffic signal control: A cooperative approach. Sustainability. 2023; 15(4):3479.
[21] Swapno SMMR, Nobel SN, Meena P. et al. A reinforcement learning approach for reducing traffic congestion using deep Q learning. Sci Rep. 2024; 14:30452. https://doi.org/10.1038/s41598-024-75638-0.
[22] Tan J, Yuan Q, Guo W, Xie N, Liu F, Wei J, Zhang X. Deep reinforcement learning for traffic signal control model and adaptation study. Sensors. 2022; 22(22):8732.
[23] Jiang Q, Qin M, Shi S, Sun W, Zheng B. Multi-agent reinforcement learning for traffic signal control through universal communication method. arXiv preprint arXiv:2204.12190. 2022
[24] Goel H, Zhang Y, Damani M, Sartoretti G. Sociallight: Distributed cooperation learning towards network-wide traffic signal control. 2023. arXiv preprint arXiv:2305.16145.
[25] Kolat M, Kővári B, Bécsi T, Aradi S. Multi-agent reinforcement learning for traffic signal control: A cooperative approach. Sustainability. 2023; 15(4):3479.
[26] Li H, Zhao Y, Mao Z, Qin Y, Xiao Z, Feng J, ... Zhang M. A survey on graph neural networks in intelligent transportation systems. 2024. arXiv preprint arXiv:2401.00713.
[27] Yang G, Wen X, Chen F. Multi-Agent Deep Reinforcement Learning with Graph Attention Network for Traffic Signal Control in Multiple-Intersection Urban Areas. Transportation Research Record. 2025; 0(0). https://doi.org/10.1177/03611981241297979.
[28] Cai C, Wei M. Adaptive urban traffic signal control based on enhanced deep reinforcement learning. Sci Rep. 2024; 14:14116. https://doi.org/10.1038/s41598-024-64885-w
[29] Fu Y, Zhong L, Li Z, Di X. Federated Hierarchical Reinforcement Learning for Adaptive Traffic Signal Control. 2025. arXiv preprint arXiv:2504.05553.
[30] Azad‐Manjiri M, Afsharchi M, Abdoos M. DDPGAT: Integrating MADDPG and GAT for optimized urban traffic light control. IET Intelligent Transport Systems. 2025; 19(1):e70000.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 R. M. Bommi, E. Bhuvaneswari, M. Rohini, G. Uganya
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This is an open-access article distributed under the terms of the Creative Commons Attribution CC BY 4.0 license, which permits unlimited use, distribution, and reproduction in any medium so long as the original work is properly cited.
