Abstract
The navigation of robots in dynamic urban environments, requires elaborated anticipative strategies for the robot to avoid collisions with dynamic objects, like bicycles or pedestrians, and to be human aware. We have developed and analyzed three anticipative strategies in motion planning taking into account the future motion of the mobile objects that can move up to 18 km/h. First, we have used our hybrid policy resulting from a Deep Deterministic Policy Gradient (DDPG) training and the Social Force Model (SFM), and we have tested it in simulation in four complex map scenarios with many pedestrians. Second, we have used these anticipative strategies in real-life experiments using the hybrid motion planning method and the ROS Navigation Stack with Dynamic Windows Approach (NS-DWA). The results in simulations and real-life experiments show very good results in open environments and also in mixed scenarios with narrow spaces.
O. Gil—Work supported under the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016-0656) and ROCOTRANSP project (PID2019-106702RB-C21/AEI/10.13039/501100011033). Óscar Gil is also supported by the Spanish Ministry of Science and Innovation under an FPI-grant, BES-2017-082126.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ajoudani, A., Zanchettin, A.M., Ivaldi, S., Albu-Schäffer, A., Kosuge, K., Khatib, O.: Progress and prospects of the human-robot collaboration. Auton. Robot. 42(5), 957–975 (2018)
Charalampous, K., Kostavelis, I., Gasteratos, A.: Recent trends in social aware robot navigation: A survey. Robot. Auton. Syst. 93, 85–104 (2017)
Chiang, H.T., Malone, N., Lesser, K., Oishi, M., Tapia, L.: Path-guided artificial potential fields with stochastic reachable sets for motion planning in highly dynamic environments. In: 2015 IEEE international conference on robotics and automation (ICRA), pp. 2347–2354. IEEE (2015)
Chiang, H.T.L., Faust, A., Fiser, M., Francis, A.: Learning navigation behaviors end-to-end with autorl. IEEE Robot. Autom. Lett. 4(2), 2007–2014 (2019)
Cosgun, A., Sisbot, E.A., Christensen, H.I.: Anticipatory robot path planning in human environments. In: 2016 25th IEEE international symposium on robot and human interactive communication (RO-MAN), pp. 562–569. IEEE (2016)
Dalmasso, M., Garrell, A., Domínguez, J.E., Jiménez, P., Sanfeliu, A.: Human-robot collaborative multi-agent path planning using monte carlo tree search and social reward sources. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 10133–10138 (2021)
Faust, A., Francis, A., Mehta, D.: Evolving rewards to automate reinforcement learning. arXiv preprint arXiv:1905.07628 (2019)
Ferrer, G., Sanfeliu, A.: Anticipative kinodynamic planning: multi-objective robot navigation in urban and dynamic environments. Auton. Robot. 43(6), 1473–1488 (2019)
Fox, D., Burgard, W., Thrun, S.: The dynamic window approach to collision avoidance. IEEE Robot. Autom. Mag. 4(1), 23–33 (1997)
Francis, A., et al.: Long-range indoor navigation with prm-rl. IEEE Trans. Robot. (2020)
Gil, O., Garrell, A., Sanfeliu, A.: Social robot navigation tasks: Combining machine learning techniques and social force model. Sensors 21(21), 7087 (2021)
Haarnoja, T., Zhou, A., Abbeel, P., Levine, S.: Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: International Conference on Machine Learning, pp. 1861–1870 (2018)
Han, Y., Zhan, I.H., Zhao, W., Pan, J., Zhang, Z., Wang, Y., Liu, Y.J.: Deep reinforcement learning for robot collision avoidance with self-state-attention and sensor fusion. IEEE Robot. Autom. Lett. 7(3), 6886–6893 (2022)
Helbing, D., Molnar, P.: Social force model for pedestrian dynamics. Phys. Rev. E 51(5), 4282 (1995)
Lillicrap, T.P., et al.: Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015)
Linder, T., Breuers, S., Leibe, B., Arras, K.O.: On multi-modal people tracking from mobile platforms in very crowded and dynamic environments. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 5512–5519 (2016)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Repiso, E., Garrell, A., Sanfeliu, A.: People’s adaptive side-by-side model evolved to accompany groups of people by social robots. IEEE Robot. Automat. Lett. 5(2), 2387–2394 (2020)
Rudenko, A., Palmieri, L., Herman, M., Kitani, K.M., Gavrila, D.M., Arras, K.O.: Human motion trajectory prediction: A survey. Int. J. Robot. Res. 39(8), 895–935 (2020)
Schöller, C., Aravantinos, V., Lay, F., Knoll, A.: What the constant velocity model can teach us about pedestrian motion prediction. IEEE Robot. Autom. Lett. 5(2), 1696–1703 (2020)
Urdiales, C., et al.: A new multi-criteria optimization strategy for shared control in wheelchair assisted navigation. Auton. Robot. 30(2), 179–197 (2011)
Vaquero, V., Repiso, E., Sanfeliu, A.: Robust and real-time detection and tracking of moving objects with minimum 2d lidar information to advance autonomous cargo handling in ports. Sensors 19(1), 107 (2019)
Zanlungo, F., Ikeda, T., Kanda, T.: Social force model with explicit collision prediction. EPL (Europhysics Lett.) 93(6), 68005 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gil, Ó., Sanfeliu, A. (2023). Robot Navigation Anticipative Strategies in Deep Reinforcement Motion Planning. In: Tardioli, D., Matellán, V., Heredia, G., Silva, M.F., Marques, L. (eds) ROBOT2022: Fifth Iberian Robotics Conference. ROBOT 2022. Lecture Notes in Networks and Systems, vol 590. Springer, Cham. https://doi.org/10.1007/978-3-031-21062-4_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-21062-4_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-21061-7
Online ISBN: 978-3-031-21062-4
eBook Packages: EngineeringEngineering (R0)