Abstract
Deep reinforcement learning (DRL) has been successfully adopted in many tasks, such as autonomous driving and gaming, to achieve or surpass human-level performance. This paper proposes a DRL-based trajectory planner for automated parking systems (APS). A thorough review of literature in this field is presented. A simulation study is conducted to investigate the trajectory planning performance of the parking agent for: (i) different neural-network architectures; (ii) different training set-ups; (iii) efficacy of human-demonstration. Real-time capability of the proposed planner on various embedded hardware platforms is also discussed by the paper, showing promising performance. Insights of the use of DRL for APS are concluded at the end of the paper.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Abbreviations
- a :
-
action
- l :
-
length
- L(s, a) :
-
loss function
- Q(s, a, θ) :
-
action-value function
- r :
-
reward
- s :
-
state
- θ :
-
optimal parameter
- y :
-
discount factor
- β :
-
slip angle
- π :
-
policy
- ψ :
-
yaw angle
- S :
-
steering angle of front axle
- i:
-
index
- N:
-
total training number
- R:
-
reserved
- t:
-
time step
References
Badrinarayanan, V., Kendall, A. and Cipolla, R. (2015). Segnet: A deep convolutional encoder-decoder architecture for robust semantic pixel-wise labeling. arXiv: 1511.00561.
Beniz (2017). Deep Learning API and Server in C++11 with Python Bindings and Support for Caffe, Tensorflow, XGBoost and TSNE. https://github.com/beniz/deepdetect
Chae, H., Kang, C., Kim, B., Kim, J., Chung, C. and Choi, J. (2017). Autonomous braking system via deep reinforcement learning. arXiv: 1702.02302.
Gardener and Benoitsteiner (2017). An Open-source Software Library for Machine Intelligence. https://www.tensorflow.org
Hausknecht, M. and Stone, P. (2015). Deep recurrent Q-learning for partially observable MDPs. arXiv: 1507.06527.
Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural Computation9, 8, 1735–1780.
Huval, B., Wang, T., Tandon, S., Kiske, J., Song, W., Pazhayampallil, J. et al. (2015). An empirical evaluation of deep learning on highway driving. arXiv: 1504.01716.
Jaritz, M., Charette, R., Toromanoff, M., Perot, E. and Nashashibi, F. (2018). End-to-end race driving with deep reinforcement learning. arXiv: 1807.02371.
Kardell, S. and Kuosku, M. (2017). Autonomous Vehicle Control via Deep Reinforcement Learning. http://publications.lib.chalmers.se/records/fulltext/252902/252902.pdf
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D. and Kavukcuoglu, K. (2016). Asynchronous methods for deep reinforcement learning. Int. Conf. Machine Learning, 1928–1937.
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D. and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv: 1312.5602.
OpenCV (2017). https://opencv.org
Sallab, A. E., Abdou, M., Perot, E. and Yogamani, S. (2016). End-to-end deep reinforcement learning for lane keeping assist. arXiv: 1612.04340.
Sallab, A. E., Abdou, M., Perot, E. and Yogamani, S. (2017). Deep reinforcement learning framework for autonomous driving. Electronic Imaging2017, 19, 70–76.
Shiba, S. (2017). Deep Parking: An Implementation of Automatic Parking with Deep Reinforcementation Learning. http://www.slideshare.net/shintaroshiba5
Silver, D., Huang, A., Maddison, C., Guez, A., Sifre, L., Driessche, G, Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T. and Hassabis, D. (2016). Mastering the game of go with deep neural networks and tree search. Nature, 529, 484–489.
Watkins, C. J. and Dayan, P. (1992). Q-learning. Machine Learning8, 3-4, 279–292.
Acknowledgement
This project is funded by Leapmotor Technology and National Key R&D Program of China (2018YFB0105204).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Du, Z., Miao, Q. & Zong, C. Trajectory Planning for Automated Parking Systems Using Deep Reinforcement Learning. Int.J Automot. Technol. 21, 881–887 (2020). https://doi.org/10.1007/s12239-020-0085-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12239-020-0085-9