Abstract
The goal of this work is to evaluate the task of autonomous driving in urban environment using Deep Q-Network Agents. For this purpose, several approaches based on DQN agents will be studied. The DQN agent learn a policy (set of actions) for lane following tasks using visual and driving features obtained from sensors onboard the vehicle and a model-based path planner. The policy objective is to drive as fast as possible following the center of the lane avoiding collisions and road departures. A dynamic urban simulation environment will be designed using CARLA simulator to validate our proposal. The results show that a DQN agent could be a promising technique for self-driving a vehicle in a urban environment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chan, C.: Advancements, prospects, and impacts of automated driving systems. Int. J. Transp. Sci. Technol. 6(3), 208–216 (2017)
Urmson, C., Anhalt, J., et al.: Autonomous driving in urban environments boss and the urban challenge. J. Field Robot. 25(8), 425–466 (2008)
Montemerlo, M., Becker, J., et al.: Junior: the stanford entry in the urban challenges. J. Field Robot. 25(9), 569–597 (2008)
Wang, F.: AI and intelligent vehicles future challenge (IVFC) in China: from cognitive intelligence to parallel intelligence. In: ITU Kaleidoscope: Challenges for a Data-Driven Society (ITU K), pp. 1–2 (2017)
Zhao, J. et al.: TiEV: the tongji intelligent electric vehicle. In: International Conference on Intelligent Transportation Systems(ITSC), pp. 1303–1309 (2018)
Liang, M., Yang, B. Wang, S. Urtasun, R.: Deep continuous fusion for multi-sensor 3D object detection. European Conference on Computer Vision (ECCV), pp. 641–656 (2018)
Barea, R., et al.: Vehicle detection and localization using 3D LIDAR point cloud and image semantic segmentation. In: 21st International Conference on Intelligent Transportation Systems (ITSC), pp. 3481–3486 (2018). https://doi.org/10.1109/ITSC.2018.8569962
Silver, D., et al.: Mastering chess and shogi by selfplay with a general reinforcement learning algorithm. CoRR, vol. abs/1712.01815 (2017)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529 (2015)
Kendall. A et al.: Learning to drive in a day. In: 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, pp. 8248–8254 (2019). https://doi.org/10.1109/ICRA.2019.8793742.
Lillicrap, T. et al.: Continuous control with deep reinforcement learning. In: International Conference on Learning Representations (ICLR) (2016)
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: CARLA: an open urban driving simulator. arXiv preprint arXiv:1711.03938 (2017)
Quigley, M., et al.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software (2009). https://www.ros.org/
Levinson, J., et al.: Towards fully autonomous driving: systems and algorithms. In: 2011 IEEE Intelligent Vehicles Symposium (IV), pp. 163–168. IEEE (2011)
Chen, J., Yuan, B., Tomizuka, M.: Deep imitation learning for autonomous driving in generic urban scenarios with enhanced safety. arXiv preprint arXiv:1903.00640 (2019)
Codevilla, F., Miiller, M., Lopez, A., Koltun, V., Dosovitskiy, A.: End-to-end driving via conditional imitation learning. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 1–9 (2018)
Yurtsever, E., Capito, L., Redmill, K., Ozguner, U.: Integrating deep reinforcement learning with model-based path planners for automated driving. ArXiv:2002.00434 (2020)
Vitelli, M. Nayebi, A.: CARMA: a deep reinforcement learning approach to autonomous driving. Technical report, Stanford University (2016)
Ganesh, A., Charalel, J., Sarma, M.D., Xu, N.: Deep reinforcement learning for simulated autonomous driving. In: Stanford University (2016)
Jaritz, M, Charette, R., Toromanoff, M., Perot, E., Nashashibi, F.: End-to-end race driving with deep reinforcement learning. In: ICRA 2018, pp. 2070–2075 (2018). https://doi.org/10.1109/ICRA.2018.8460934
Chen, J., Shengbo, E., Tomizuka, M.: Interpretable end-to-end urban autonomous driving with latent deep reinforcement learning. arXiv preprint arXiv:2001.08726 (2019)
Fan, J., Wang, Z., Xie, Y., Yang Z.: A theoretical analysis of deep Q-learning. ArXiv:1901.00137 (2020)
CARLA: Open-source simulator for autonomous driving research.https://carla.org
Sentdex: Carla-RL (2020).https://github.com/Sentdex/Carla-RL
Keras. https://keras.io/
Tensorflow. https://www.tensorflow.org
Bojarski, M., et al.: End to end learning for self-driving cars (2016). arXiv:1604.07316
Gutiérrez, R., López-Guillén, E., Bergasa, L.M., Barea, R., Pérez, Ó., Gómez-Huélamo, C., Arango, F., del Egido, J., López-Fernández, J.: A waypoint tracking controller for autonomous road vehicles using ROS framework. Sensors 20, 4062 (2020)
Smart Elderly Car Project. https://www.robesafe.uah.es/index.php/en/smartelderlycar
Acknowledgment
This work has been funded in part from the Spanish MICINN/FEDER through the Techs4AgeCar project (RTI2018-099263-B-C21) and from the RoboCity2030-DIH-CM project (P2018/NMT- 4331), funded by Programas de actividades I+D (CAM) and cofunded by EU Structural Funds.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pérez-Gil, Ó. et al. (2021). DQN-Based Deep Reinforcement Learning for Autonomous Driving. In: Bergasa, L.M., Ocaña, M., Barea, R., López-Guillén, E., Revenga, P. (eds) Advances in Physical Agents II. WAF 2020. Advances in Intelligent Systems and Computing, vol 1285. Springer, Cham. https://doi.org/10.1007/978-3-030-62579-5_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-62579-5_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-62578-8
Online ISBN: 978-3-030-62579-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)