Reinforcement Learning for Autonomous Defence in Software-Defined Networking
Authors: 
Yi Han, Benjamin I. P. Rubinstein, Tamas Abraham, Tansu Alpcan, Olivier De Vel, Sarah Erfani, David Hubczenko, Christopher Leckie, Paul MontagueAuthors Info & Claims
Yi Han, Benjamin I. P. Rubinstein, Tamas Abraham, Tansu Alpcan, Olivier De Vel, Sarah Erfani, David Hubczenko, Christopher Leckie, Paul MontagueAuthors Info & ClaimsPages 145 - 165
Abstract
Despite the successful application of machine learning (ML) in a wide range of domains, adaptability—the very property that makes machine learning desirable—can be exploited by adversaries to contaminate training and evade classification. In this paper, we investigate the feasibility of applying a specific class of machine learning algorithms, namely, reinforcement learning (RL) algorithms, for autonomous cyber defence in software-defined networking (SDN). In particular, we focus on how an RL agent reacts towards different forms of causative attacks that poison its training process, including indiscriminate and targeted, white-box and black-box attacks. In addition, we also study the impact of the attack timing, and explore potential countermeasures such as adversarial training.
References
[1]
Amazon EC2 Instance Types – Amazon Web Services (AWS). https://aws.amazon.com/ec2/instance-types/
[2]
SDN architecture. Technical report, June 2014. https://www.opennetworking.org/wp-content/uploads/2013/02/TR_SDN_ARCH_1.0_06062014.pdf
[3]
Mininet: An Instant Virtual Network on your Laptop (2017). http://mininet.org/
[4]
OpenDaylight (2017). https://www.opendaylight.org/
[5]
Athalye, A., Carlini, N., Wagner, D.: Obfuscated gradients give a false sense of security: circumventing defenses to adversarial examples. arXiv:1802.00420 [cs], February 2018
[6]
Barreno M, Nelson B, Joseph AD, and Tygar JD The security of machine learning Mach. Learn. 2010 81 2 121-148
[7]
Beaudoin, L.: Autonomic computer network defence using risk states and reinforcement learning. Ph.D. thesis, University of Ottawa (Canada) (2009)
[8]
Behzadan Vahid and Munir Arslan Vulnerability of Deep Reinforcement Learning to Policy Induction Attacks Machine Learning and Data Mining in Pattern Recognition 2017 Cham Springer International Publishing 262-275
[9]
Bhagoji, A.N., Cullina, D., Mittal, P.: Dimensionality reduction as a defense against evasion attacks on machine learning classifiers. arXiv:1704.02654 (2017)
[10]
Biggio B et al. Ma Y, Guo G, et al. Security evaluation of support vector machines in adversarial environments Support Vector Machines Applications 2014 Cham Springer 105-153
[11]
Biggio, B., Nelson, B., Laskov, P.: Poisoning attacks against support vector machines. In: Proceedings of the 29th International Conference on International Conference on Machine Learning, pp. 1467–1474. Omnipress, Edinburgh (2012)
[12]
Burkard, C., Lagesse, B.: Analysis of causative attacks against SVMs learning from data streams. In: Proceedings of the 3rd ACM on International Workshop on Security And Privacy Analytics, pp. 31–36. ACM, New York (2017)
[13]
Carlini, N., Wagner, D.: Defensive distillation is not robust to adversarial examples. arXiv:1607.04311 (2016)
[14]
Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. eprint arXiv:1608.04644 (2016)
[15]
Carlini, N., Wagner, D.: Adversarial examples are not easily detected: bypassing ten detection methods. eprint arXiv:1705.07263 (2017)
[16]
Chung SP and Mok AK Kruegel C, Lippmann R, and Clark A Advanced allergy attacks: does a corpus really help? Recent Advances in Intrusion Detection 2007 Heidelberg Springer 236-255
[17]
Das, N., et al.: Keeping the bad guys out: protecting and vaccinating deep learning with JPEG compression. eprint arXiv:1705.02900, May 2017
[18]
Diakonikolas, I., Kamath, G., Kane, D.M., Li, J., Moitra, A., Stewart, A.: Robust estimators in high dimensions without the computational intractability. In: Proceedings of the 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS), pp. 655–664, October 2016
[19]
Everitt, T., Krakovna, V., Orseau, L., Hutter, M., Legg, S.: Reinforcement learning with a corrupted reward channel. eprint arXiv:1705.08417 (2017)
[20]
Feinman, R., Curtin, R.R., Shintre, S., Gardner, A.B.: Detecting adversarial samples from artifacts. eprint arXiv:1703.00410 (2017)
[21]
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. eprint arXiv:1412.6572 (2014)
[22]
Han, Y., Rubinstein, B.I.P.: Adequacy of the gradient-descent method for classifier evasion attacks. arXiv:1704.01704, April 2017
[23]
Hasselt, H.V.: Double Q-learning. In: Lafferty, J.D., Williams, C.K.I., Shawe-Taylor, J., Zemel, R.S., Culotta, A. (eds.) Advances in Neural Information Processing Systems 23, pp. 2613–2621. Curran Associates, Inc. (2010)
[24]
Hasselt, H.V., Guez, A., Silver, D.: Deep reinforcement learning with double Q-learning. eprint arXiv:1509.06461, September 2015
[25]
He, W., Wei, J., Chen, X., Carlini, N., Song, D.: Adversarial example defenses: ensembles of weak defenses are not strong. eprint arXiv:1706.04701 (2017)
[26]
Hosseini, H., Chen, Y., Kannan, S., Zhang, B., Poovendran, R.: Blocking transferability of adversarial examples in black-box learning systems. eprint arXiv:1703.04318 (2017)
[27]
Huang, L., Joseph, A.D., Nelson, B., Rubinstein, B.I., Tygar, J.: Adversarial machine learning. In: Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence, pp. 43–58. ACM (2011)
[28]
Huang, S., Papernot, N., Goodfellow, I., Duan, Y., Abbeel, P.: Adversarial attacks on neural network policies. eprint arXiv:1702.02284 (2017)
[29]
Koh, P.W., Liang, P.: understanding black-box predictions via influence functions. arXiv:1703.04730 [cs, stat], March 2017
[30]
Laishram, R., Phoha, V.V.: Curie: a method for protecting SVM Classifier from poisoning attack. arXiv:1606.01584 [cs], June 2016
[31]
Li, B., Vorobeychik, Y.: Feature cross-substitution in adversarial classification. In: Proceedings of the 2014 NIPS, NIPS 2014, pp. 2087–2095, MIT Press, Cambridge (2014)
[32]
Li, B., Wang, Y., Singh, A., Vorobeychik, Y.: Data poisoning attacks on factorization-based collaborative filtering. eprint arXiv:1608.08182 (2016)
[33]
Li, X., Li, F.: Adversarial examples detection in deep networks with convolutional filter statistics. arXiv:1612.07767 [cs], December 2016
[34]
Lin, Y.C., Hong, Z.W., Liao, Y.H., Shih, M.L., Liu, M.Y., Sun, M.: Tactics of adversarial attack on deep reinforcement learning agents. eprint arXiv:1703.06748, March 2017
[35]
Medved, J., Varga, R., Tkacik, A., Gray, K.: OpenDaylight: towards a model-driven SDN controller architecture. In: Proceedings of IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks, pp. 1–6 (2014)
[36]
Mei, S., Zhu, X.: Using machine teaching to identify optimal training-set attacks on machine learners. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, pp. 2871–2877. AAAI Press, Austin (2015)
[37]
Metzen, J.H., Genewein, T., Fischer, V., Bischoff, B.: On detecting adversarial perturbations. eprint arXiv:1702.04267 (2017)
[38]
Mnih, V., et al.: Asynchronous methods for deep reinforcement learning. In: Proceedings of the 33rd International Conference on International Conference on Machine Learning, ICML 2016, vol. 48, pp. 1928–1937. JMLR.org, New York (2016)
[39]
Mnih, V., et al.: Playing Atari with Deep Reinforcement Learning. CoRR abs/1312.5602 (2013)
[40]
Moore D, Shannon C, Brown DJ, Voelker GM, and Savage S Inferring internet denial-of-service activity ACM Trans. Comput. Syst. 2006 24 2 115-139
[41]
Moosavi-Dezfooli, S.M., Fawzi, A., Fawzi, O., Frossard, P.: Universal adversarial perturbations. eprint arXiv:1610.08401 (2016)
[42]
Moosavi-Dezfooli, S.M., Fawzi, A., Frossard, P.: DeepFool: a simple and accurate method to fool deep neural networks. In: CVPR, pp. 2574–2582 (2016)
[43]
Nelson, B., et al.: Exploiting machine learning to subvert your spam filter. In: Proceedings of the First USENIX Workshop on Large-scale Exploits and Emergent Threats (LEET 2008) (2008)
[44]
Nelson B et al. Query strategies for evading convex-inducing classifiers J. Mach. Learn. Res. 2012 13 May 1293-1332
[45]
Newsome J, Karp B, and Song D Zamboni D and Kruegel C Paragraph: thwarting signature learning by training maliciously Recent Advances in Intrusion Detection 2006 Heidelberg Springer 81-105
[46]
Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. In: CVPR, pp. 427–436 (2015)
[47]
Papernot, N., McDaniel, P., Goodfellow, I.: Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. eprint arXiv:1605.07277 (2016)
[48]
Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z.B., Swami, A.: Practical black-box attacks against deep learning systems using adversarial examples. eprint arXiv:1602.02697 (2016)
[49]
Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z.B., Swami, A.: The limitations of deep learning in adversarial settings. In: Proceedings of the European Symposium on Security & Privacy, pp. 372–387 (2016)
[50]
Papernot, N., McDaniel, P., Wu, X., Jha, S., Swami, A.: Distillation as a defense to adversarial perturbations against deep neural networks. eprint arXiv:1511.04508 (2015)
[51]
Pinto, L., Davidson, J., Sukthankar, R., Gupta, A.: Robust adversarial reinforcement learning. eprint arXiv:1703.02702 (2017)
[52]
Rubinstein, B.I., et al.: ANTIDOTE: understanding and defending against poisoning of anomaly detectors. In: Proceedings of the 9th ACM SIGCOMM Conference on Internet Measurement, pp. 1–14. ACM (2009)
[53]
Schaul, T., Quan, J., Antonoglou, I., Silver, D.: Prioritized Experience Replay. CoRR abs/1511.05952 (2015)
[54]
Sengupta, S., Chakraborti, T., Kambhampati, S.: Securing deep neural nets against adversarial attacks with moving target defense. eprint arXiv:1705.07213, May 2017
[55]
Steinhardt, J., Koh, P.W., Liang, P.: Certified defenses for data poisoning attacks. eprint arXiv:1706.03691, June 2017
[56]
Sutton RS and Barto AG Introduction to Reinforcement Learning 1998 1 Cambridge MIT Press
[57]
Szegedy, C., et al.: Intriguing properties of neural networks. eprint arXiv:1312.6199 (2013)
[58]
Tramèr, F., Kurakin, A., Papernot, N., Boneh, D., McDaniel, P.: Ensemble adversarial training: attacks and defenses. eprint arXiv:1705.07204, May 2017
[59]
Wang, B., Gao, J., Qi, Y.: A theoretical framework for robustness of (deep) classifiers against adversarial examples. eprint arXiv:1612.00334 (2016)
[60]
Xiao, H., Xiao, H., Eckert, C.: Adversarial label flips attack on support vector machines. In: Proceedings of the 20th European Conference on Artificial Intelligence. ECAI 2012, pp. 870–875, IOS Press, Amsterdam (2012)
[61]
Zhang F, Chan PPK, Biggio B, Yeung DS, and Roli F Adversarial feature selection against evasion attacks IEEE Trans. Cybern. 2016 46 3 766-777
[62]
Zheng, S., Song, Y., Leung, T., Goodfellow, I.: Improving the robustness of deep neural networks via stability training. eprint arXiv:1604.04326 (2016)
Recommendations
POSTER: Toward Intelligent Cyber Attacks for Moving Target Defense Techniques in Software-Defined Networking
ASIA CCS '23: Proceedings of the 2023 ACM Asia Conference on Computer and Communications SecurityMoving Target Defenses (MTD) are proactive security countermeasures that change the attack surface in a system in ways that make it harder for attackers to succeed. These techniques have been shown to be effective, and their application in software-...
Software-defined Networking-based DDoS Defense Mechanisms
Distributed Denial of Service attack (DDoS) is recognized to be one of the most catastrophic attacks against various digital communication entities. Software-defined networking (SDN) is an emerging technology for computer networks that uses open ...
Instance-based defense against adversarial attacks in Deep Reinforcement Learning
AbstractDeep Reinforcement Learning systems are now a hot topic in Machine Learning for their effectiveness in many complex tasks, but their application in safety-critical domains (e.g., robot control or self-autonomous driving) remains ...
Comments
Information & Contributors
Information
Published In
Oct 2018
651 pages
ISBN:978-3-030-01553-4
DOI:10.1007/978-3-030-01554-1
- Editors:
- Linda Bushnell,
- Radha Poovendran,
- Tamer Başar
© Springer Nature Switzerland AG 2018.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 29 October 2018
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 04 Oct 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in