research-article

The Green Choice: Learning and Influencing Human Decisions on Shared Roads

Authors:

Daniel A. Lazar,

Ramtin PedarsaniAuthors Info & Claims

2019 IEEE 58th Conference on Decision and Control (CDC)

Pages 347 - 354

https://doi.org/10.1109/CDC40024.2019.9030169

Published: 01 December 2019 Publication History

Abstract

Autonomous vehicles have the potential to increase the capacity of roads via platooning, even when human drivers and autonomous vehicles share roads. However, when users of a road network choose their routes selfishly, the resulting traffic configuration may be very inefficient. Because of this, we consider how to influence human decisions so as to decrease congestion on these roads. We consider a network of parallel roads with two modes of transportation: (i) human drivers who will choose the quickest route available to them, and (ii) ride hailing service which provides an array of autonomous vehicle ride options, each with different prices, to users. In this work, we seek to design these prices so that when autonomous service users choose from these options and human drivers selfishly choose their resulting routes, road usage is maximized and transit delay is minimized. To do so, we formalize a model of how autonomous service users make choices between routes with different price/delay values. Developing a preference-based algorithm to learn the preferences of the users, and using a vehicle flow model related to the Fundamental Diagram of Traffic, we formulate a planning optimization to maximize a social objective and demonstrate the benefit of the proposed routing and learning scheme.

References

[1]

D. Schrank, B. Eisele, T. Lomax, and J. Bak, “2015 urban mobility scorecard,” 2015.

[2]

E. Koutsoupias and C. Papadimitriou, “Worst-case equilibria,” in Annual Symposium on Theoretical Aspects of Computer Science. Springer, 1999, pp. 404–413.

Digital Library

[3]

N. Mehr and R. Horowitz, “Can the presence of autonomous vehicles worsen the equilibrium state of traffic networks?” in 2018 IEEE Conference on Decision and Control (CDC), 2018, pp. 1788–1793.

[4]

E. Bıyık, D. Lazar, R. Pedarsani, and D. Sadigh, “Altruistic autonomy: Beating congestion on shared roads,” in Workshop on Algorithmic Foundations of Robotics (WAFR), 2018.

[5]

C. Swamy, “The effectiveness of stackelberg strategies and tolls for network congestion games,” ACM Transactions on Algorithms (TALG), vol. 8, no. 4, p. 36, 2012.

Digital Library

[6]

W. Krichene, J. D. Reilly, S. Amin, and A. M. Bayen, “Stackelberg routing on parallel transportation networks,” Handbook of Dynamic Game Theory, pp. 1–35, 2017.

[7]

R. D. Luce, Individual choice behavior: A theoretical analysis. Courier Corporation, 2012.

[8]

M. E. Ben-Akiva, S. R. Lerman, and S. R. Lerman, Discrete choice analysis: theory and application to travel demand. MIT press, 1985, vol. 9.

[9]

J. R. Correa and N. E. Stier-Moses, “Wardrop equilibria,” Wiley encyclopedia of operations research and management science, 2011.

[10]

D. Sadigh, A. D. Dragan, S. Sastry, and S. A. Seshia, “Active preference-based learning of reward functions,” in Robotics: Science and Systems (RSS), 2017.

[11]

E. Biyik and D. Sadigh, “Batch active preference-based learning of reward functions,” in Conference on Robot Learning, 2018, pp. 519–528.

[12]

J. Lioris, R. Pedarsani, F. Y. Tascikaraoglu, and P. Varaiya, “Platoons of connected vehicles can double throughput in urban roads,” Transportation Research Part C: Emerging Technologies, vol. 77, pp. 292–305, 2017.

[13]

A. Askari, D. A. Farias, A. A. Kurzhanskiy, and P. Varaiya, “Effect of adaptive and cooperative adaptive cruise control on throughput of signalized arterials,” in Intelligent Vehicles Symposium (IV), 2017 IEEE, 2017, pp. 1287–1292.

[14]

R. E. Stern, S. Cui, M. L. Delle Monache, R. Bhadani, M. Bunting, M. Churchill, N. Hamilton, H. Pohlmann, F. Wu, B. Piccoli, et al., “Dissipation of stop-and-go waves via control of autonomous vehicles: Field experiments,” Transportation Research Part C: Emerging Technologies, vol. 89, pp. 205–221, 2018.

[15]

R. K. Bhadani, B. Piccoli, B. Seibold, J. Sprinkle, and D. Work, “Dissipation of emergent traffic waves in stop-and-go traffic using a supervisory controller,” in 2018 IEEE Conference on Decision and Control (CDC), 2018, pp. 3628–3633.

[16]

C. Wu, A. M. Bayen, and A. Mehta, “Stabilizing traffic with autonomous vehicles,” in To appear, International Conference on Robotics and Automation, 2018.

[17]

L. Jin, M. Čičič, S. Amin, and K. H. Johansson, “Modeling the impact of vehicle platooning on highway congestion: A fluid queuing approach,” in Proceedings of the 21st International Conference on Hybrid Systems: Computation and Control (part of CPS Week). ACM, 2018, pp. 237–246.

[18]

S. Sivaranjani, Y.-S. Wang, V. Gupta, and K. Savla, “Localization of disturbances in transportation systems,” in Decision and Control (CDC), 2015 IEEE 54th Annual Conference on, 2015, pp. 3439–3444.

[19]

T. Roughgarden and É. Tardos, “How bad is selfish routing?” Journal of the ACM (JACM), vol. 49, no. 2, pp. 236–259, 2002.

Digital Library

[20]

J. R. Correa, A. S. Schulz, and N. E. Stier-Moses, “A geometric approach to the price of anarchy in nonatomic congestion games,” Games Econ. Behavior, vol. 64, no. 2, pp. 457–469, 2008.

[21]

D. A. Lazar, S. Coogan, and R. Pedarsani, “Routing for traffic networks with mixed autonomy,” arXiv preprint arXiv:1809.01283, 2018.

[22]

M. Beckmann, C. B. McGuire, and C. B. Winsten, “Studies in the economics of transportation,” Tech. Rep., 1956.

[23]

L. Fleischer, K. Jain, and M. Mahdian, "Tolls for heterogeneous selfish users in multicommodity networks and generalized congestion games," in 45th Annual IEEE Symposium on Foundations of Computer Science. IEEE, 2004, pp. 277–285.

[24]

P. N. Brown and J. R. Marden, “The robustness of marginal-cost taxes in affine congestion games,” IEEE Transactions on Automatic Control, vol. 62, no. 8, pp. 3999–4004, 2017.

[25]

M. Salazar, M. Tsao, I. Aguiar, M. Schiffer, and M. Pavone, “A congestion-aware routing scheme for autonomous mobility-on-demand systems,” in European Control Conference (ECC), submitted, 2019.

[26]

N. D. Daw, J. P. O’doherty, P. Dayan, B. Seymour, and R. J. Dolan, “Cortical substrates for exploratory decisions in humans,” Nature, vol. 441, no. 7095, p. 876, 2006.

[27]

C. F. Daganzo, “The cell transmission model: A dynamic representation of highway traffic consistent with the hydrodynamic theory,” Transportation Research Part B: Methodological, vol. 28, no. 4, pp. 269–287, 1994.

[28]

P. Abbeel and A. Y. Ng, “Apprenticeship learning via inverse reinforcement learning,” in Proceedings of the twenty-first international conference on Machine learning. ACM, 2004, p. 1.

Digital Library

[29]

A. Y. Ng, S. J. Russell, et al., “Algorithms for inverse reinforcement learning.” in Icml, vol. 1, 2000, p. 2.

[30]

P. Abbeel and A. Y. Ng, “Exploration and apprenticeship learning in reinforcement learning,” in Proceedings of the 22nd international conference on Machine learning. ACM, 2005, pp. 1–8.

[31]

B. D. Ziebart, A. L. Maas, J. A. Bagnell, and A. K. Dey, “Maximum entropy inverse reinforcement learning.” in Aaai, vol. 8. Chicago, IL, USA, 2008, pp. 1433–1438.

[32]

R. Akrour, M. Schoenauer, and M. Sebag, “April: Active preference learning-based reinforcement learning,” in Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 2012, pp. 116–131.

[33]

P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” in Advances in Neural Information Processing Systems, 2017, pp. 4299–4307.

[34]

D. A. Lazar, S. Coogan, and R. Pedarsani, “The price of anarchy for transportation networks with mixed autonomy,” in 2018 Annual American Control Conference (ACC). IEEE, 2018, pp. 6359–6365.

[35]

D. A. Lazar, R. Pedarsani, K. Chandrasekher, and D. Sadigh, “Maximizing road capacity using cars that influence people,” in 2018 IEEE Conference on Decision and Control (CDC), 2018, pp. 1801–1808.

[36]

J. Wardrop, “Some theoretical aspects of road traffic research,” in Inst Civil Engineers Proc London/UK/, 1900.

[37]

A. De Palma and Y. Nesterov, “Optimization formulations and static equilibrium in congested transportation networks,” Tech. Rep., 1998.

[38]

G. Andrew and J. Gao, “Scalable training of l 1-regularized loglinear models,” in Proceedings of the 24th international conference on Machine learning. ACM, 2007, pp. 33–40.

[39]

R. H. Byrd, J. C. Gilbert, and J. Nocedal, “A trust region method based on interior point techniques for nonlinear programming,” Mathematical programming, vol. 89, no. 1, pp. 149–185, 2000.

[40]

R. A. Waltz, J. L. Morales, J. Nocedal, and D. Orban, “An interior algorithm for nonlinear optimization that combines line search and trust region steps,” Mathematical programming, vol. 107, no. 3, pp. 391–408, 2006.

[41]

E. Biyik, M. Palan, N. C. Landolfi, D. P. Losey, and D. Sadigh, “Asking easy questions: A user-friendly approach to active reward learning,” in Conference on Robot Learning (CoRL), October 2019.

[42]

P. N. Brown and J. R. Marden, “Fundamental limits of locallycomputed incentives in network routing,” in 2017 American Control Conference (ACC). IEEE, 2017, pp. 5263–5268.

Cited By

Yang SChen TZhou MSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)A dense reward view on aligning text-to-image diffusion with preferenceProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3694380(55998-56032)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3694380
Bıyık ETalati ASadigh DSakamoto DWeiss AHiatt LShiomi M(2022)APReLProceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3523760.3523841(613-617)Online publication date: 7-Mar-2022
https://dl.acm.org/doi/10.5555/3523760.3523841
Beliaev MBıyık ELazar DWang WSadigh DPedarsani RMaggio MWeimer JAl Faruque MOishi M(2021)Incentivizing routing choices for safe and efficient transportation in the face of the COVID-19 pandemicProceedings of the ACM/IEEE 12th International Conference on Cyber-Physical Systems10.1145/3450267.3450546(187-197)Online publication date: 19-May-2021
https://dl.acm.org/doi/10.1145/3450267.3450546
Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

2019 IEEE 58th Conference on Decision and Control (CDC)

7716 pages

Copyright © 2019.

Publisher

IEEE Press

Publication History

Published: 01 December 2019

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yang SChen TZhou MSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)A dense reward view on aligning text-to-image diffusion with preferenceProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3694380(55998-56032)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3694380
Bıyık ETalati ASadigh DSakamoto DWeiss AHiatt LShiomi M(2022)APReLProceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3523760.3523841(613-617)Online publication date: 7-Mar-2022
https://dl.acm.org/doi/10.5555/3523760.3523841
Beliaev MBıyık ELazar DWang WSadigh DPedarsani RMaggio MWeimer JAl Faruque MOishi M(2021)Incentivizing routing choices for safe and efficient transportation in the face of the COVID-19 pandemicProceedings of the ACM/IEEE 12th International Conference on Cyber-Physical Systems10.1145/3450267.3450546(187-197)Online publication date: 19-May-2021
https://dl.acm.org/doi/10.1145/3450267.3450546
Kwon MBiyik ETalati ABhasin KLosey DSadigh DBelpaeme TYoung JGunes HRiek L(2020)When Humans Aren't OptimalProceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3319502.3374832(43-52)Online publication date: 9-Mar-2020
https://dl.acm.org/doi/10.1145/3319502.3374832

View Options

View options

Figures

Tables

Media

View Table of Conten