Abstract
It is challenging for a security analyst to detect or defend against cyber-attacks. Moreover, traditional defense deployment methods require the security analyst to manually enforce the defenses in the presence of uncertainties about the defense to deploy. As a result, it is essential to develop an automated and resilient defense deployment mechanism to thwart the new generation of attacks. In this paper, we propose a framework based on Markov Decision Process (MDP) and Q-learning to automatically generate optimal defense solutions for networked system states. The framework consists of four phases namely; the model initialization phase, model generation phase, Q-learning phase, and the conclusion phase. The proposed model collects real network information as inputs and then builds them into structural data. We implement a Q-learning process in the model to learn the quality of a defense action in a particular state. To investigate the feasibility of the proposed model, we perform simulation experiments and the result reveals that the model can reduce the risk of network systems from cyber attacks. Furthermore, the experiment shows that the model has shown a certain level of flexibility when different parameters are used for Q-learning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alavizadeh, H., et al.: A survey on cyber situation awareness systems: framework, techniques, and insights. ACM Comput. Surv. (CSUR) 55(5), 1–37 (2022)
Applebaum, A., Miller, D., Strom, B., Korban, C., Wolf, R.: Intelligent, Automated Red Team Emulation. In: Proceedings of the 32nd Annual Conference on Computer Security Applications, pp. 363–373 (2016)
Booker, L.B., Musman, S.A.: A model-based, decision-theoretic perspective on automated cyber response. arXiv preprint. arXiv:2002.08957 (2020)
Enoch, S.Y., Mendonça, J., Hong, J.B., Ge, M., Kim, D.S.: An integrated security hardening optimization for dynamic networks using security and availability modeling with multi-objective algorithm. Comput. Netw. 208, 108864 (2022)
Enoch, S.Y., Moon, C.Y., Lee, D., Ahn, M.K., Kim, D.S.: A practical framework for cyber defense generation, enforcement and evaluation. Comput. Netw. 208, 108878 (2022)
FIRST: CVSS v3.1: Specification Document. Forum of Incident Response and Security Teams (2019). https://www.first.org/cvss/v3.1/specification-document
Iqbal, Z., Anwar, Z.: SCERM-a novel framework for automated management of cyber threat response activities. Future Gener. Comput. Syst. 108, 687–708 (2020)
Kaloudi, N., Li, J.: The AI-based cyber threat landscape: a survey. ACM Comput. Surv. (CSUR) 53(1), 1–34 (2020)
McAfee: Mcafee labs 2020 threats predictions report (2019). https://www.mcafee.com/blogs/other-blogs/mcafee-labs/mcafee-labs-2020-threats-predictions-report/
Noor, U., Anwar, Z., Malik, A.W., Khan, S., Saleem, S.: A machine learning framework for investigating data breaches based on semantic analysis of adversary’s attack patterns in threat intelligence repositories. Futur. Gener. Comput. Syst. 95, 467–487 (2019)
Park, M., Seo, J., Han, J., Oh, H., Lee, K.: Situational awareness framework for threat intelligence measurement of android malware. JoWUA 9(3), 25–38 (2018)
Ray, H.T., Vemuri, R., Kantubhukta, H.R.: Toward an automated attack model for red teams. IEEE Secur. Priv. 3(4), 18–25 (2005)
Stoecklin, M.P.: Deeplocker: how AI can power a stealthy new breed of malware. Security Intell. (2018)
Zheng, J., Namin, A.S.: Defending sdn-based iot networks against ddos attacks using markov decision process. In: 2018 IEEE International Conference on Big Data (Big Data). IEEE (2018)
Zheng, J., Namin, A.S.: Markov decision process to enforce moving target defence policies. arXiv preprint. arXiv:1905.09222 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhou, X., Enoch, S.Y., Kim, D.S. (2023). Markov Decision Process for Automatic Cyber Defense. In: You, I., Youn, TY. (eds) Information Security Applications. WISA 2022. Lecture Notes in Computer Science, vol 13720. Springer, Cham. https://doi.org/10.1007/978-3-031-25659-2_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-25659-2_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25658-5
Online ISBN: 978-3-031-25659-2
eBook Packages: Computer ScienceComputer Science (R0)