Modeling and Simulation of UAV Autonomous Obstacle Avoidance Based on DQN
Article No.: 74, Pages 1 - 6
Abstract
During the obstacle avoidance process of UAV, the route planning and obstacle avoidance decision-making is dynamic and sequential due to the change of obstacles in real time, so it is difficult to build a dynamic and accurate route planning model. This paper breaks through the traditional research thinking of route modeling and optimization solution for UAV obstacle avoidance, and applies deep reinforcement learning to UAV autonomous obstacle avoidance decision-making; through designing the environment model, autonomous decision-making model and DQN algorithm model for UAV obstacle avoidance, four simulation experimental environments for UAV obstacle avoidance are constructed to verify the superiority and effectiveness of deep reinforcement learning in solving the decision-making problems related to dynamic model and timing sequence, and provide a new solution for future UAV route planning.
References
[1]
Ling,ZHANG Zhu-feng,WU Wei.Coordinated test allocation in multi-UCAV based on distributed constrained optimization[J].Journal of Naval University of Engineering,2018,12(6) :64-68.
[2]
Hu Teng, LIU Zhangjun, LIU Yang,et al.3D surveillance path planning for multi-UAVs[J].Systems Engineering and Electronics[J].2019,7)7(:1551-1559.
[3]
Lucian Bu, Robert Babu, Bart De Schutter, A comprehensive survey of multiagent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 2008, 38(2):156–172,.
[4]
Matignon, Laurent Jeanpierre, and Abdel-Illah Mouaddib. Coordinated multi-robot exploration under communication constraints using decentralized markov decision processes. In Twenty-sixth AAAI conference on artificial intelligence, 2012.
[5]
Peng, Quan Yuan, Ying Wen, Yaodong Yang, Zhenkun Tang, Haitao Long, and Jun Wang. Multiagent bidirectionally-coordinated nets for learning to play starcraft combat games. arXiv preprint arXiv:1703.10069, 2, 2017.
[6]
M L. Markov games a framework for multi-agent reinforcement learning[M].New Brunswick: Machine Learning Proceedings,1994:157-163.
[7]
Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershel-vam, Marc Lanctot, Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484, 2016.
[8]
Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Grae-pel, Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint arXiv:1712.01815, 2017.
[9]
S, HOLLY E,LILLICRAP T,et a1. Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates[C]//IEEE International Conference on Robotics and Automation. Singapore:IEEE Press, 2017:3389-3396.
[10]
J, ASSAEL I, DE FREITAS N, et a1. Learning to communicate with deep multi-agent reinforcement learning[C]//Advances in Neural Information Processing Systems.Spain: NIPS Press,2016:2137-2145.
[11]
WU Y, et a1. Mu1ti-agent actor-critic for mixed cooperative-competitive Environments[C]//advances in Neural Information Processing Systems. Los Angeles :NIPS Press, 2017:6379-6390.
[12]
M, ZAMBALDI V,GRUSLYS A, et a1.A unified game-theoretic approach to multi-agent reinforcement learning[C]//Advances in Neural Information Processing Systems. Los Angles : NIPS Press,2017:4190-4203.
[13]
J, ZAMBALDI V, LANCTOT M, Multi-agent reinforcement learning in sequential social dilemmas[C]//Proceedings of the 16th Conference on Autonomous Agents and Multi-agent Systems, Singapore: AAMAS Press, 2017:464-473.
[14]
SHALEV-SHWARTZ S, SHAMMMAH S, SHASHUA A. Safe, multi-agent reinforcement learning for autonomous driving [J].https://arxiv.org/abs/1610.03295.
[15]
JIN J, SONG C, LI H, et a1. Rea1-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising[J/OL].https//arxiv.org/abs/1802. 09756.
[16]
L, CHEV J, HUANG Y, et a1. Smart generation control based on multi -agent reinforcement learning with the idea of the time tunnel[J].Energy, 2018, 153:977-987.
[17]
J,LEIBO JZ,ZAMBALDI V, et a1.A multi-agent reinforcement learning model of common-pool resource appropriation[C] Advances in pleural Information Processing Systems. Los Angeles: NIPS Press, 2017:3643-3652.
Recommendations
Real-time obstacle avoidance with deep reinforcement learning Three-Dimensional Autonomous Obstacle Avoidance for UAV
RICAI '19: Proceedings of the 2019 International Conference on Robotics, Intelligent Control and Artificial IntelligenceAt present, drones are rapidly developing in the aviation industry and are applied to all aspects of life. However, letting drones autonomously avoid obstacles is still the focus of research by aviation scholars at this stage. However, the current ...
Autonomous UAV Trail Navigation with Obstacle Avoidance Using Deep Neural Networks
AbstractThis paper proposes a vision-based bike trail following approach with obstacle avoidance using CNN (Convolutional Neural Network) for the UAV (Unmanned Aerial Vehicle). The UAV is controlled to follow a given trail while keeping its position near ...
Autonomous Vision-Based Helicopter Flights Through Obstacle Gates
The challenge for unmanned aerial vehicles to sense and avoid obstacles becomes even harder if narrow passages have to be crossed. An approach to solve a mission scenario that tackles the problem of such narrow passages is presented here. The task is to ...
Comments
Information & Contributors
Information
Published In
December 2021
699 pages
ISBN:9781450385053
DOI:10.1145/3508546
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 25 February 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ACAI'21
ACAI'21: 2021 4th International Conference on Algorithms, Computing and Artificial Intelligence
December 22 - 24, 2021
Sanya, China
Acceptance Rates
Overall Acceptance Rate 173 of 395 submissions, 44%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 76Total Downloads
- Downloads (Last 12 months)19
- Downloads (Last 6 weeks)0
Reflects downloads up to 26 Jul 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format