research-article

Decision Modeling and Simulation of Fighter Air-to-ground Combat Based on Reinforcement Learning

Authors:

Yan WangAuthors Info & Claims

IPMV '22: Proceedings of the 4th International Conference on Image Processing and Machine Vision

Pages 102 - 109

https://doi.org/10.1145/3529446.3529463

Published: 15 July 2022 Publication History

Abstract

With the Artificial Intelligence (AI) widely used in air combat simulation system, the decision-making system of fighter has reached a high level of complexity. Traditionally, the pure theoretical analysis and the rule-based system are not enough to represent the cognitive behavior of pilots. In order to properly specify the autonomous decision-making of fighter, hence, we proposed a unified framework which combines the combat simulation and machine learning in this paper. This framework adopts deep reinforcement learning modelling by using the supervised learning and the Deep Q-Network (DQN) methods. As a proof of concept, we built an autonomous decision-making training scenario based on the Weapon Effectiveness Simulation System (WESS). The simulation results show that the intelligent decision-making model based on the proposed framework has better combat effects than the traditional decision-making model based on knowledge engineering.

References

[1]

GAO Zhen-yang, QIN Bin. Progress in Deep Reinforcement Learning research[J]. Computer knowledge and technology, 2019,15(04): 163-165+179.

[2]

B Xue, Y He, F Jing, Robot Target Recognition using Deep Federated Learning [J]. International Journal of Intelligent Systems, 2021, 36(12): 7754-7769.

[3]

Lu Y, Xu X, Zhang X, Hierarchical Reinforcement Learning for Autonomous Decision Making and Motion Planning of Intelligent Vehicles[J]. IEEE Access, 2020, PP(99):1-1.

[4]

Peng X, Chen R, Zhang J, Enhanced Autonomous Navigation of Robots by Deep Reinforcement Learning Algorithm with Multistep Method[J]. Sensors and Materials, 2021, 33(2):825.

[5]

ZUO Jialiang, YANG Rennong, ZHANG Ying, LI Zhonglin, WU Meng. Intelligent decision-making in air combat maneuvering based on heuristic reinforcement learning[J]. Acta Aeronautica et Astronautica Sinica, 2017(10):217-230.

[6]

LI ANG ZHANG, JIA XU, DARA GOLD, JEFF HAGEN, AJAY K. KOCHHAR, ANDREW J. LOHN, OSONDE A. OSOBA. Air Dominance Through Machine Learning.

[7]

Li K, Zhang K, Zhang Z, A UAV Maneuver Decision-Making Algorithm for Autonomous Airdrop Based on Deep Reinforcement Learning[J]. Sensors, 2021, 21(6): 2233.

[8]

Yang Qiming, Zhang Jiandong, Shi Guoqing, Hu Jinwen, Wu Yong. Maneuver Decision of UAV in Short-Range Air Combat Based on Deep Reinforcement Learning[J].IEEE ACCESS, 2020: 363-378.

[9]

Song Xiagan, Jiang Ju. Research on intelligent air combat decision under uncertain environment[D]. Nanjing: Nanjing University of Aeronautics and Astronautics, 2017.03.

[10]

LIU Chao, LI Qingwei. Design of Simulation Software System for Intelligent Decision – Making in Tactical – Level Air Combat[A]. 2019 China System Simulation and Virtual Reality Technology High-level Forum[C],2019.

[11]

Lei Yonglin, Yao Jian, Zhu Ning, Zhu Yifan, Wang Weiping. Weapon Effectiveness Simulation System (WESS)[J]. Journal of System Simulation. 2017,29(6): 1244-1252.

[12]

AUSTIN F, CARBONE G, MICHAEL F, el a1. Game Theory for Automated Maneuvering during Air-to—Air Combat[J].Journal of Guidance 1990,13(6):1143—1147.

[13]

VIRLANEN K, RAIVIO T, HAMALAINEN R P. Decision Theoretical Approach to Pilot Simulation[J].Journal of Aircraft, 1999,27(4):632—641.

[14]

HORIE K, CONWAY B. Optimal Fighter Pursuit-Evasion Maneuvers Found via Two-Sided Optimization[J]. Journal of Guidance, Control and Dynamics, 2006, 29(1): 105—112.

[15]

ZHU Shihu, DONG Chaoyang, ZHANG Jinpeng, CHEN Zongji. An intelligent decision-making system based on neural networks and expert system [J].Electro-optical and control, 2006,13(1): 8-11.

[16]

SMITH R E, DIKE B A, MEHRA R K, Classifier systems in combat: two-sided learning of maneuvers for advanced fighter aircraft[J].Computer Methods in Applied Mechanics and Engineering, 2000,186(2): 421. 00395−3.

[17]

NICHOLAS E, COHEN K, SCHUMACHER D. Collaborative tasking of UAVs using a genetic fuzzy approach[C]. /51st AIAA Aerospace Sciences Meeting including the New Horizons Forum and Aerospace Exposition. Grapevine: AIAA, 2013: 1032. /6. 2013-1032.

Cited By

Wang DZhang JYang QLiu JShi GZhang Y(2024)An Autonomous Attack Decision-Making Method Based on Hierarchical Virtual Bayesian Reinforcement LearningIEEE Transactions on Aerospace and Electronic Systems10.1109/TAES.2024.341024960:5(7075-7088)Online publication date: Oct-2024
https://doi.org/10.1109/TAES.2024.3410249
ZHANG JWANG DYANG QSHI ZJI LSHI GWU Y(2024)Loyal wingman task execution for future aerial combat: A hierarchical prior-based reinforcement learning approachChinese Journal of Aeronautics10.1016/j.cja.2024.03.00937:5(462-481)Online publication date: May-2024
https://doi.org/10.1016/j.cja.2024.03.009
Wang XWang YSu XWang LLu CPeng HLiu J(2023)Deep reinforcement learning-based air combat maneuver decision-making: literature review, implementation tutorial and future directionArtificial Intelligence Review10.1007/s10462-023-10620-257:1Online publication date: 28-Dec-2023
https://dl.acm.org/doi/10.1007/s10462-023-10620-2

Recommendations

Infrared Air Combat Simulation Model for Deep Reinforcement Learning
CSAE '21: Proceedings of the 5th International Conference on Computer Science and Application Engineering

Aiming at the problem of lacking credible and realistic infrared air combat simulation platform for applying the deep reinforcement learning method, this paper explored the design requirements for the construction of simulation system, built the overall ...
Air combat maneuver decision based on deep reinforcement learning with auxiliary reward
Abstract
For air combat maneuvering decision, the sparse reward during the application of deep reinforcement learning limits the exploration efficiency of the agents. To address this challenge, we propose an auxiliary reward function considering the impact ...
Autonomous air combat decision‐making of UAV based on parallel self‐play reinforcement learning
Abstract
Aiming at addressing the problem of manoeuvring decision‐making in UAV air combat, this study establishes a one‐to‐one air combat model, defines missile attack areas, and uses the non‐deterministic policy Soft‐Actor‐Critic (SAC) algorithm in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

IPMV '22: Proceedings of the 4th International Conference on Image Processing and Machine Vision

March 2022

121 pages

ISBN:9781450395823

DOI:10.1145/3529446

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

IPMV 2022

IPMV 2022: 2022 4th International Conference on Image Processing and Machine Vision

March 25 - 27, 2022

Hong Kong, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
67
Total Downloads

Downloads (Last 12 months)26
Downloads (Last 6 weeks)2

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang DZhang JYang QLiu JShi GZhang Y(2024)An Autonomous Attack Decision-Making Method Based on Hierarchical Virtual Bayesian Reinforcement LearningIEEE Transactions on Aerospace and Electronic Systems10.1109/TAES.2024.341024960:5(7075-7088)Online publication date: Oct-2024
https://doi.org/10.1109/TAES.2024.3410249
ZHANG JWANG DYANG QSHI ZJI LSHI GWU Y(2024)Loyal wingman task execution for future aerial combat: A hierarchical prior-based reinforcement learning approachChinese Journal of Aeronautics10.1016/j.cja.2024.03.00937:5(462-481)Online publication date: May-2024
https://doi.org/10.1016/j.cja.2024.03.009
Wang XWang YSu XWang LLu CPeng HLiu J(2023)Deep reinforcement learning-based air combat maneuver decision-making: literature review, implementation tutorial and future directionArtificial Intelligence Review10.1007/s10462-023-10620-257:1Online publication date: 28-Dec-2023
https://dl.acm.org/doi/10.1007/s10462-023-10620-2

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents