research-article

Public Access

Online Algorithms for Adaptive Cyber Defense on Bayesian Attack Graphs

Authors:

Peng LiuAuthors Info & Claims

MTD '17: Proceedings of the 2017 Workshop on Moving Target Defense

Pages 99 - 109

https://doi.org/10.1145/3140549.3140556

Published: 30 October 2017 Publication History

Abstract

Emerging zero-day vulnerabilities in information and communications technology systems make cyber defenses very challenging. In particular, the defender faces uncertainties of; e.g., system states and the locations and the impacts of vulnerabilities. In this paper, we study the defense problem on a computer network that is modeled as a partially observable Markov decision process on a Bayesian attack graph. We propose online algorithms which allow the defender to identify effective defense policies when utility functions are unknown a priori. The algorithm performance is verified via numerical simulations based on real-world attacks.

References

[1]

Karl J. Âström. 1965. Optimal control of Markov processes with incomplete state information. J. Math. Anal. Appl. 10, 1 (1965), 174 -- 205. https://doi.org/10.1016/0022-247X(65)90154-X

[2]

Richard E. Bellman. 2003. Dynamic Programming. Dover Publications.

[3]

Richard E. Bellman and Stuart E. Dreyfus. 1962. Applied dynamic programming. Princeton University Press.

[4]

Dimitri P. Bertsekas and John N. Tsitsiklis. 1996. Neuro-Dynamic Programming (1st ed.). Athena Scientific.

[5]

David Bigelow, Thomas Hobson, Robert Rudd, William Streilein, and Hamed Okhravi. 2015. Timely Rerandomization for Mitigating Memory Disclosures. In ACM SIGSAC Conference on Computer and Communications Security (CCS '15). Denver, Colorado, USA, 268--279.

Digital Library

[6]

Anthony R. Cassandra, Leslie Pack Kaelbling, and Michael L. Littman. 1994. Acting Optimally in Partially Observable Stochastic Domains. In Proceedings of the Twelfth National Conference on Artificial Intelligence (Vol. 2) (AAAI'94). American Association for Artificial Intelligence, Menlo Park, CA, USA, 1023--1028.

[7]

Ping Chen, Jun Xu, Zhiqiang Lin, Dongyan Xu, Bing Mao, and Peng Liu. 2015. A Practical Approach for Adaptive Data Structure Layout Randomization. In Proceedings of the 20th European Symposium on Research in Computer Security (ESORICS'15). Vienna, Austria, 69--89.

Digital Library

[8]

CVE-2014-0160. 2014. Heartbleed Bug. https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-0160. (2014).

[9]

George Cybenko, Sushil Jajodia, Michael P. Wellman, and Peng Liu. 2014. Adversarial and uncertain reasoning for adaptive cyber defense: Building the scientific foundation. In International Conference on Information Systems Security (ICISS 2014). Hyderabad, India, 1--8.

[10]

Zakir Durumeric, James Kasten, David Adrian, Alex Halderman, Michael Bailey, Frank Li, Nicolas Weaver, Johanna Amann, Jethro Beekman, Mathias Payer, and Vern Paxson. 2014. The Matter of Heartbleed. In Proceedings of the 2014 Conference on Internet Measurement Conference (IMC '14). Vancouver, BC, Canada, 475--488.

Digital Library

[11]

Zhisheng Hu, Ping Chen, Yang Lu, Minghui Zhu, and Peng Liu. 2016. Towards a science for adaptive defense: Revisit server protection. In IEEE International Conference on Collaboration and Internet Computing. Pittsburgh, 112--121.

[12]

Zhisheng Hu, Minghui Zhu, Ping Chen, and Peng Liu. 2016. On convergence rates of robust adaptive game theoretic learning algorithms. ArXiv e-prints (Dec. 2016). arXiv:math.OC/1612.04724 https://arxiv.org/abs/1612.04724.

[13]

Jeff Hughes, Lawrence Carin, and George Cybenko. 2008. Cybersecurity Strategies: The QuERIES Methodology. Computer 41 (2008), 20--26. https://doi.org/doi.ieeecomputersociety.org/10.1109/MC.2008.295

Digital Library

[14]

Marcus Hutter and Jan Poland. 2005. Adaptive Online Prediction by Following the Perturbed Leader. Journal of Machine Learning Research 6 (2005), 639--660.

Digital Library

[15]

Adam Kalai and Santosh Vempala. 2005. Efficient Algorithms for Online Decision Problems. J. Comput. System Sci. 71, 3 (Oct. 2005), 291--307. https://doi.org/10.1016/j.jcss.2004.10.016

Digital Library

[16]

Per Larsen, Andrei Homescu, Stefan Brunthaler, and Michael Franz. 2014. SoK: Automated Software Diversity. In Proceedings of the 2014 IEEE Symposium on Security and Privacy (SP '14). San Jose, CA, USA.

Digital Library

[17]

Yu Liu and Hong Man. 2005. Network Vulnerability Assessment Using Bayesian Networks. In Data Mining, Intrusion Detection, Information Assurance, and Data Networks Security 2005. 61--71.

[18]

Erik Miehling, Mohammad Rasouli, and Demosthenis Teneketzis. 2015. Optimal Defense Policies for Partially Observable Spreading Processes on Bayesian Attack Graphs. In Proceedings of the Second ACM Workshop on Moving Target Defense (MTD '15). ACM, New York, NY, USA, 67--76. https://doi.org/10.1145/2808475.2808482

Digital Library

[19]

George E. Monahan. 1982. A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms. Management Science 28, 1 (1982), 1--16. http://www.jstor.org/stable/2631070

Digital Library

[20]

Hamed Okhravi, James Riordan, and Kevin Carter. 2014. Quantitative evaluation of dynamic platform techniques as a defensive mechanism. In Research in Attacks, Intrusions and Defenses: 17th International Symposium, RAID 2014. Gothenburg, Sweden, 405--425.

[21]

Nayot Poolsappasit, Rinku Dewri, and Indrajit Ray. 2012. Dynamic Security Risk Management Using Bayesian Attack Graphs. IEEE Transactions on Dependable and Secure Computing 9, 1 (Jan 2012), 61--74. https://doi.org/10.1109/TDSC.2011.34

Digital Library

[22]

Martin L. Puterman. 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming (1st ed.). John Wiley & Sons, Inc., New York, NY, USA.

[23]

Carlos Sarraute, Olivier Buffet, and Jörg Hoffmann. 2012. POMDPs Make Better Hackers: Accounting for Uncertainty in Penetration Testing. In Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence (AAAI'12). AAAI Press, 1816--1824. http://dl.acm.org/citation.cfm?id=2900929.2900985

[24]

Mike Schiffman. 2017. Common Vulnerability Scoring System (CVSS). (2017). http://www.first.org/cvss.

[25]

Mike Schiffman. 2017. Common Vulnerability Scoring System v3.0: Specification Document. (2017). https://www.first.org/cvss/cvss-v30-specification-v1.7.pdf.

[26]

Guy Shani, Joelle Pineau, and Robert Kaplow. 2013. A survey of point-based POMDP solvers. Autonomous Agents and Multi-Agent Systems 27, 1 (01 Jul 2013), 1--51. https://doi.org/10.1007/s10458-012-9200-2

Digital Library

[27]

Edward J. Sondik. 1978. The Optimal Control of Partially Observable Markov Processes Over the Infinite Horizon: Discounted Costs. Operations Research 26, 2 (1978), 282--304.

Digital Library

[28]

Symantec. 2015. Internet Security Threat Report. (2015). https://know.elq.symantec.com/LP=1542.

[29]

Christopher J. C. H. Watkins and Peter Dayan. 1992. Q-learning. In Machine Learning. 279--292.

[30]

Herbert Weisberg. 1992. Central tendency and variability. Number 83. Sage University Paper Series on Quantitative Applications in the Social Sciences.

[31]

Jun Xu, Pinyao Guo, Mingyi Zhao, Robert F. Erbacher, Minghui Zhu, and Peng Liu. 2014. Comparing different moving target defense techniques. In First ACM Workshop on Moving Target Defense, in Association with 2014 ACM Conference on Computer and Communications Security. Scottsdale, Arizona, 97--107.

Digital Library

[32]

Lu Yu and Richard R. Brooks. 2013. Applying POMDP to Moving Target Optimization. In Proceedings of the Eighth Annual Cyber Security and Information Intelligence Research Workshop (CSIIRW '13). ACM, New York, NY, USA, Article 49, 4 pages. https://doi.org/10.1145/2459976.2460032

[33]

Emmanuele Zambon and Damiano Bolzoni. 2006. Network Intrusion Detection Systems: False Positive reduction through Anomaly Detection. (2006). http://www.blackhat.com/presentations/bh-usa-06/BH-US-06-Zambon.pdf.

[34]

Minghui Zhu, Zhisheng Hu, and Peng Liu. 2014. Reinforcement learning algorithms for adaptive cyber defense against Heartbleed. In First ACM Workshop on Moving Target Defense (MTD '14). Scottsdale, Arizona, USA, 51--58.

Digital Library

[35]

Minghui Zhu and Sonia Martínez. 2014. On attack-resilient distributed formation control in operator-vehicle networks. SIAM Journal on Control and Optimization 52, 5 (2014), 3176--3202.

[36]

Quanyan Zhu and Tamer Başar. 2009. Dynamic policy-based IDS configuration. In Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference. 8600--8605. https://doi.org/10.1109/CDC.2009.5399894

[37]

Quanyan Zhu, Hamidou Tembine, and Tamer Başar. 2013. Hybrid learning in stochastic games and its applications in network security. Reinforcement Learning and Approximate Dynamic Programming for Feedback Control (2013), 305--329.

Cited By

Kazeminajafabadi AImani M(2024)Optimal Joint Defense and Monitoring for Networks Security under Uncertainty: A POMDP-Based ApproachIET Information Security10.1049/2024/79667132024(1-20)Online publication date: 27-May-2024
https://doi.org/10.1049/2024/7966713
Kazeminajafabadi AImani M(2023)Optimal monitoring and attack detection of networks modeled by Bayesian attack graphsCybersecurity10.1186/s42400-023-00155-y6:1Online publication date: 1-Sep-2023
https://doi.org/10.1186/s42400-023-00155-y
Outkin ASchulz PSchulz TTarman TPinar A(2023)Defender Policy Evaluation and Resource Allocation With MITRE ATT&CK Evaluations DataIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2022.316562420:3(1909-1926)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1109/TDSC.2022.3165624
Show More Cited By

Recommendations

Adaptive Cyber Defense Against Multi-Stage Attacks Using Learning-Based POMDP

Growing multi-stage attacks in computer networks impose significant security risks and necessitate the development of effective defense schemes that are able to autonomously respond to intrusions during vulnerability windows. However, the defender faces ...
Optimal Defense Policies for Partially Observable Spreading Processes on Bayesian Attack Graphs
MTD '15: Proceedings of the Second ACM Workshop on Moving Target Defense

The defense of computer networks from intruders is becoming a problem of great importance as networks and devices become increasingly connected. We develop an automated approach to defending a network against continuous attacks from intruders, using the ...
Moving Target Defense Against Injection Attacks
Algorithms and Architectures for Parallel Processing
Abstract
With the development of network technology, web services become more convenient and popular. However, web services are also facing serious security threats, especially SQL injection attack(SQLIA). Due to the diversity of attack techniques and the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MTD '17: Proceedings of the 2017 Workshop on Moving Target Defense

October 2017

126 pages

ISBN:9781450351768

DOI:10.1145/3140549

Program Chairs:
Hamed Okhravi
MIT Lincoln Laboratory, USA
,
Xinming Ou
University of South Florida, USA

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 October 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

CCS '17

Sponsor:

SIGSAC

CCS '17: 2017 ACM SIGSAC Conference on Computer and Communications Security

October 30, 2017

Texas, Dallas, USA

Acceptance Rates

MTD '17 Paper Acceptance Rate 9 of 26 submissions, 35%;

Overall Acceptance Rate 40 of 92 submissions, 43%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
592
Total Downloads

Downloads (Last 12 months)113
Downloads (Last 6 weeks)13

Reflects downloads up to 09 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kazeminajafabadi AImani M(2024)Optimal Joint Defense and Monitoring for Networks Security under Uncertainty: A POMDP-Based ApproachIET Information Security10.1049/2024/79667132024(1-20)Online publication date: 27-May-2024
https://doi.org/10.1049/2024/7966713
Kazeminajafabadi AImani M(2023)Optimal monitoring and attack detection of networks modeled by Bayesian attack graphsCybersecurity10.1186/s42400-023-00155-y6:1Online publication date: 1-Sep-2023
https://doi.org/10.1186/s42400-023-00155-y
Outkin ASchulz PSchulz TTarman TPinar A(2023)Defender Policy Evaluation and Resource Allocation With MITRE ATT&CK Evaluations DataIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2022.316562420:3(1909-1926)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1109/TDSC.2022.3165624
Kim HHwang EKim DCho JMoore TNelson FLim H(2023)Time-Based Moving Target Defense Using Bayesian Attack Graph AnalysisIEEE Access10.1109/ACCESS.2023.326901811(40511-40524)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3269018
Zenitani K(2023)Attack graph analysisComputers and Security10.1016/j.cose.2022.103081126:COnline publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1016/j.cose.2022.103081
Wang WSun DJiang FChen XZhu C(2022)Research and Challenges of Reinforcement Learning in Cyber Defense Decision-Making for Intranet SecurityAlgorithms10.3390/a1504013415:4(134)Online publication date: 18-Apr-2022
https://doi.org/10.3390/a15040134
Niknami NWu J(2021)A constraint partially observable semi-Markov decision process for the attack–defence relationships in various critical infrastructuresCyber-Physical Systems10.1080/23335777.2021.18799358:2(85-110)Online publication date: 8-Feb-2021
https://doi.org/10.1080/23335777.2021.1879935
Zhang YChang XMišić JMišić VCai Y(2021)Cost-effective migration-based dynamic platform defense technique: a CTMDP approachPeer-to-Peer Networking and Applications10.1007/s12083-021-01084-8Online publication date: 30-Jan-2021
https://doi.org/10.1007/s12083-021-01084-8
Hu ZZhu MLiu P(2020)Adaptive Cyber Defense Against Multi-Stage Attacks Using Learning-Based POMDPACM Transactions on Privacy and Security10.1145/341889724:1(1-25)Online publication date: 8-Nov-2020
https://dl.acm.org/doi/10.1145/3418897
Xie YJi LLi LGuo ZBaker T(2020)An adaptive defense mechanism to prevent advanced persistent threatsConnection Science10.1080/09540091.2020.1832960(1-21)Online publication date: 16-Oct-2020
https://doi.org/10.1080/09540091.2020.1832960
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents