research-article

Real-Time Cooperative Vehicle Coordination at Unsignalized Road Intersections

Authors:

Tingting Zhang,

Chunsheng Chen,

Qinyu ZhangAuthors Info & Claims

IEEE Transactions on Intelligent Transportation Systems, Volume 24, Issue 5

Pages 5390 - 5405

https://doi.org/10.1109/TITS.2023.3243940

Published: 01 May 2023 Publication History

Abstract

Cooperative coordination at unsignalized road intersections, which aims to improve the driving safety and traffic throughput for connected and automated vehicles (CAVs), has attracted increasing interests in recent years. However, most existing investigations either suffer from computational complexity or cannot harness the full potential of the road infrastructure. To this end, we first present a dedicated intersection coordination framework, where the involved vehicles hand over their control authorities and follow instructions from a centralized coordinator. Then a unified cooperative trajectory planning problem will be formulated to maximize the traffic throughput while ensuring driving safety. To address the key computational challenges in the real-world deployment, we reformulate this non-convex sequential decision-making problem into a model-free Markov Decision Process (MDP) and tackle it by devising a Twin Delayed Deep Deterministic Policy Gradient (TD3)-based strategy in the deep reinforcement learning (DRL) framework. Simulation and practical experiments show that the proposed strategy could achieve near-optimal performance in sub-static coordination scenarios and significantly improve the traffic throughput in the realistic continuous traffic flow. The most remarkable advantage is that our strategy could reduce the time complexity of computation to milliseconds, and is shown scalable when the road lanes increase.

References

[1]

J. Luoet al., “Cooperative trajectory planning at unsignalized intersections using deep reinforcement learning,” in Proc. IEEE/CIC Int. Conf. Commun. China (ICCC Workshops), Aug. 2022, pp. 227–232.

[2]

M. Alsabaan, W. Alasmary, A. Albasir, and K. Naik, “Vehicular networks for a greener environment: A survey,” IEEE Commun. Surveys Tuts., vol. 15, no. 3, pp. 1372–1388, 3rd Quart., 2013.

[3]

S. Dornbush and A. Joshi, “StreetSmart traffic: Discovering and disseminating automobile congestion using VANET’s,” in Proc. IEEE 65th Veh. Technol. Conf. (VTC-Spring), Apr. 2007, pp. 11–15.

[4]

G. Marfia and M. Roccetti, “Vehicular congestion detection and short-term forecasting: A new model with results,” IEEE Trans. Veh. Technol., vol. 60, no. 7, pp. 2936–2948, Sep. 2011.

[5]

K. Mandal, A. Sen, A. Chakraborty, S. Roy, S. Batabyal, and S. Bandyopadhyay, “Road traffic congestion monitoring and measurement using active RFID and GSM technology,” in Proc. 14th Int. IEEE Conf. Intell. Transp. Syst. (ITSC), Oct. 2011, pp. 1375–1379.

[6]

D. Zhao, Y. Dai, and Z. Zhang, “Computational intelligence in urban traffic signal control: A survey,” IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 42, no. 4, pp. 485–494, Jul. 2012.

Digital Library

[7]

D. Schrank, B. Eisele, T. Lomax, and J. Bak, “2015 urban mobility scorecard,” Texas A&M Transp. Inst., Bryan, TX, USA, Tech. Rep. 9, 2015, vol. 39, p. 5.

[8]

E. Bellis and J. Page, “National motor vehicle crash causation survey (NMVCCS) SAS analytical users manual,” Nat. Highway Traffic Saf. Admin., Washington, DC, USA, Tech. Rep. HS-811 053, 2008.

[9]

J. Gao, Y. Shen, J. Liu, M. Ito, and N. Shiratori, “Adaptive traffic signal control: Deep reinforcement learning algorithm with experience replay and target network,” 2017, arXiv:1705.02755.

[10]

B. Abdulhai and G. J. Karakoulas, “Reinforcement learning for true adaptive traffic signal control,” J. Transp. Eng., vol. 129, no. 3, pp. 278–285, 2003.

[11]

L. Li, Y. Lv, and F.-Y. Wang, “Traffic signal timing via deep reinforcement learning,” IEEE/CAA J. Autom. Sinica, vol. 3, no. 3, pp. 247–254, Jul. 2016.

[12]

H. Wei, G. Zheng, H. Yao, and Z. Li, “IntelliLight: A reinforcement learning approach for intelligent traffic light control,” in Proc. 24th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Jul. 2018, pp. 2496–2505.

[13]

X. Liang, X. Du, G. Wang, and Z. Han, “A deep reinforcement learning network for traffic light cycle control,” IEEE Trans. Veh. Technol., vol. 68, no. 2, pp. 1243–1253, Feb. 2019.

[14]

A. Haydari and Y. Yilmaz, “Deep reinforcement learning for intelligent transportation systems: A survey,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 1, pp. 11–32, Jan. 2020.

[15]

P. Dai, K. Liu, Q. Zhuge, E. H.-M. Sha, V. C. S. Lee, and S. H. Son, “Quality-of-experience-oriented autonomous intersection control in vehicular networks,” IEEE Trans. Intell. Transp. Syst., vol. 17, no. 7, pp. 1956–1967, Jul. 2016.

[16]

J. Rios-Torres and A. A. Malikopoulos, “A survey on the coordination of connected and automated vehicles at intersections and merging at highway on-ramps,” IEEE Trans. Intell. Transp. Syst., vol. 18, no. 5, pp. 1066–1077, May 2016.

Digital Library

[17]

L. Chen and C. Englund, “Cooperative intersection management: A survey,” IEEE Trans. Intell. Transp. Syst., vol. 17, no. 2, pp. 570–586, Feb. 2016.

Digital Library

[18]

S. Li, K. Shu, C. Chen, and D. Cao, “Planning and decision-making for connected autonomous vehicles at road intersections: A review,” Chin. J. Mech. Eng., vol. 34, no. 1, pp. 1–18, Dec. 2021.

[19]

Z. Zhang, R. Han, and J. Pan, “An efficient centralized planner for multiple automated guided vehicles at the crossroad of polynomial curves,” IEEE Robot. Autom. Lett., vol. 7, no. 1, pp. 398–405, Jan. 2021.

[20]

J. Lee and B. Park, “Development and evaluation of a cooperative vehicle intersection control algorithm under the connected vehicles environment,” IEEE Trans. Intell. Transp. Syst., vol. 13, no. 3, pp. 81–90, Mar. 2012.

Digital Library

[21]

A. Colombo and D. Del Vecchio, “Efficient algorithms for collision avoidance at intersections,” in Proc. 15th ACM Int. Conf. Hybrid Syst., Comput. Control, Apr. 2012, pp. 145–154.

[22]

R. Hult, G. R. Campos, P. Falcone, and H. Wymeersch, “An approximate solution to the optimal coordination problem for autonomous vehicles at intersections,” in Proc. Amer. Control Conf. (ACC), Jul. 2015, pp. 763–768.

[23]

R. Hult, G. R. Campos, E. Steinmetz, L. Hammarstrand, P. Falcone, and H. Wymeersch, “Coordination of cooperative autonomous vehicles: Toward safer and more efficient road transportation,” IEEE Signal Process. Mag., vol. 33, no. 6, pp. 74–84, Nov. 2016.

[24]

Y. Mo, M. Wang, T. Zhang, and Q. Zhang, “Autonomous cooperative vehicle coordination at road intersections,” J. Commun. Inf. Netw., vol. 4, no. 1, pp. 78–87, 2019.

[25]

M. Wang, T. Zhang, L. Gao, and Q. Zhang, “High throughput dynamic vehicle coordination for intersection ground traffic,” in Proc. IEEE 88th Veh. Technol. Conf. (VTC-Fall), Aug. 2018, pp. 1–6.

[26]

C. Liu, Y. Zhang, T. Zhang, X. Wu, L. Gao, and Q. Zhang, “High throughput vehicle coordination strategies at road intersections,” IEEE Trans. Veh. Technol., vol. 69, no. 12, pp. 14341–14354, Dec. 2020.

[27]

Y. Mo, M. Wang, T. Zhang, and Q. Zhang, “Intelligent offloading strategies for high throughput traffic intersection coordination,” in Proc. IEEE Wireless Commun. Netw. Conf. Workshop (WCNCW), Apr. 2019, pp. 1–6.

[28]

M. A. S. Kamal, J.-I. Imura, T. Hayakawa, A. Ohata, and K. Aihara, “A vehicle-intersection coordination scheme for smooth flows of traffic without using traffic lights,” IEEE Trans. Intell. Transp. Syst., vol. 16, no. 3, pp. 1136–1147, Jun. 2014.

Digital Library

[29]

H. Xu, Y. Zhang, L. Li, and W. Li, “Cooperative driving at unsignalized intersections using tree search,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 11, pp. 4563–4571, Nov. 2020.

[30]

H. Xu, C. G. Cassandras, L. Li, and Y. Zhang, “Comparison of cooperative driving strategies for CAVs at signal-free intersections,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 7, pp. 7614–7627, Jul. 2021.

[31]

C. Mavrogiannis, J. A. DeCastro, and S. S. Srinivasa, “Implicit multiagent coordination at unsignalized intersections via multimodal inference enabled by topological braids,” 2020, arXiv:2004.05205.

[32]

Y. Zhang, R. Hao, T. Zhang, X. Chang, Z. Xie, and Q. Zhang, “A trajectory optimization-based intersection coordination framework for cooperative autonomous vehicles,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 9, pp. 1–15, Dec. 2021.

[33]

P. Hang, C. Huang, Z. Hu, and C. Lv, “Driving conflict resolution of autonomous vehicles at unsignalized intersections: A differential game approach,” 2022, arXiv:2201.01424.

[34]

R. Firoozi, X. Zhang, and F. Borrelli, “Formation and reconfiguration of tight multi-lane platoons,” Control Eng. Pract., vol. 108, Mar. 2021, Art. no. 104714.

[35]

C. Ericson, Real-Time Collision Detection. Boca Raton, FL, USA: CRC Press, 2004.

[36]

H. Xu, W. Xiao, C. G. Cassandras, Y. Zhang, and L. Li, “A general framework for decentralized safe optimal control of connected and automated vehicles in multi-lane signal-free intersections,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 10, pp. 17382–17396, Oct. 2022.

[37]

K. Okamoto, M. Goldshtein, and P. Tsiotras, “Optimal covariance control for stochastic systems under chance constraints,” IEEE Control Syst. Lett., vol. 2, no. 2, pp. 266–271, Apr. 2018.

[38]

M. J. Neely, “Stochastic network optimization with application to communication and queueing systems,” Synth. Lectures Commun. Netw., vol. 3, no. 1 pp. 1–211, 2020.

[39]

D. Silver. (2015). Lectures on Reinforcement Learning. [Online]. Available: https://www.davidsilver.uk/teaching/

[40]

V. Mnihet al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, pp. 529–533, Feb. 2015.

[41]

Y. Song, M. Steinweg, E. Kaufmann, and D. Scaramuzza, “Autonomous drone racing with deep reinforcement learning,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), Sep. 2021, pp. 1205–1212.

[42]

J. Achiam. (2018). Spinning Up in Deep Reinforcement Learning. [Online]. Available: https://spinningup.openai.com/en/latest/

[43]

R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA, USA: MIT Press, 2018.

Digital Library

[44]

S. Fujimoto, H. Hoof, and D. Meger, “Addressing function approximation error in actor-critic methods,” in Proc. Int. Conf. Mach. Learn., 2018, pp. 1587–1596.

[45]

T. P. Lillicrapet al., “Continuous control with deep reinforcement learning,” 2015, arXiv:1509.02971.

[46]

D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller, “Deterministic policy gradient algorithms,” in Proc. Int. Conf. Mach. Learn., 2014, pp. 387–395.

[47]

M. Werling, J. Ziegler, S. Kammel, and S. Thrun, “Optimal trajectory generation for dynamic street scenarios in a Frenét frame,” in Proc. IEEE Int. Conf. Robot. Autom., May 2010, pp. 987–993.

[48]

M. Quigleyet al., “ROS: An open-source robot operating system,” in Proc. ICRA Workshop Open Source Softw., vol. 3, no. 3, Kobe, Japan, 2009, p. 5.

Cited By

Zhao RLi YWang KFan YGao FGao Z(2024)Centralized Cooperation for Connected Autonomous Vehicles at Intersections by Safe Deep Reinforcement LearningIEEE Transactions on Mobile Computing10.1109/TMC.2024.341744123:12(12830-12847)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TMC.2024.3417441
Li DZhang TLuo JLiang TCao BWu XZhang Q(2024)A Tightly Coupled Bi-Level Coordination Framework for CAVs at Road IntersectionsIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.337362725:7(7832-7847)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3373627

Recommendations

A Left Turn Assist Application using Wireless Vehicle-to-Vehicle Communications at Unsignalized Intersections
MoMM '13: Proceedings of International Conference on Advances in Mobile Computing & Multimedia

We present a wireless vehicle-to-vehicle communications-based safety application to assist drivers with safe left turns at unsignalized intersections. When a potentially dangerous situation such as an oncoming vehicle from the opposite lane is detected, ...
Location-based Cooperative Vehicle Collision Avoidance for Unsignalized Intersections: A Multi-sensor Integration Approach
ICCVE '12: Proceedings of the 2012 International Conference on Connected Vehicles and Expo

Safety assurance of vehicles is of great importance to improve the road transportation, especially at the unsignalized intersections. To eliminate safety threat that is mostly caused by vehicle-to-vehicle collisions, technologies and methodologies of ...
Cooperative Maneuver Planning for Mixed Traffic at Unsignalized Intersections Using Probabilistic Predictions
2022 IEEE Intelligent Vehicles Symposium (IV)
Intersections are among the scenarios that are most crucial for efficiency and traffic flow on roads. Several approaches to traffic control at intersections exist, each with its own advantages and drawbacks. These days, wireless connections between road ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Intelligent Transportation Systems

IEEE Transactions on Intelligent Transportation Systems Volume 24, Issue 5

May 2023

988 pages

ISSN:1524-9050

Issue’s Table of Contents

1558-0016 © 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Press

Publication History

Published: 01 May 2023

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhao RLi YWang KFan YGao FGao Z(2024)Centralized Cooperation for Connected Autonomous Vehicles at Intersections by Safe Deep Reinforcement LearningIEEE Transactions on Mobile Computing10.1109/TMC.2024.341744123:12(12830-12847)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TMC.2024.3417441
Li DZhang TLuo JLiang TCao BWu XZhang Q(2024)A Tightly Coupled Bi-Level Coordination Framework for CAVs at Road IntersectionsIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.337362725:7(7832-7847)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3373627

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents