research-article

Open access

Your Attack Is Too DUMB: Formalizing Attacker Scenarios for Adversarial Transferability

Authors:

Francesco Marchiori,

Luca Martinelli,

Luca PajolaAuthors Info & Claims

RAID '23: Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses

Pages 315 - 329

https://doi.org/10.1145/3607199.3607227

Published: 16 October 2023 Publication History

All formats PDF

Abstract

Evasion attacks are a threat to machine learning models, where adversaries attempt to affect classifiers by injecting malicious samples. An alarming side-effect of evasion attacks is their ability to transfer among different models: this property is called transferability. Therefore, an attacker can produce adversarial samples on a custom model (surrogate) to conduct the attack on a victim’s organization later. Although literature widely discusses how adversaries can transfer their attacks, their experimental settings are limited and far from reality. For instance, many experiments consider both attacker and defender sharing the same dataset, balance level (i.e., how the ground truth is distributed), and model architecture.

In this work, we propose the DUMB attacker model. This framework allows analyzing if evasion attacks fail to transfer when the training conditions of surrogate and victim models differ. DUMB considers the following conditions: Dataset soUrces, Model architecture, and the Balance of the ground truth. We then propose a novel testbed to evaluate many state-of-the-art evasion attacks with DUMB; the testbed consists of three computer vision tasks with two distinct datasets each, four types of balance levels, and three model architectures. Our analysis, which generated 13K tests over 14 distinct attacks, led to numerous novel findings in the scope of transferable attacks with surrogate models. In particular, mismatches between attackers and victims in terms of dataset source, balance levels, and model architecture lead to non-negligible loss of attack performance.

References

[1]

Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. 2020. Square attack: a query-efficient black-box adversarial attack via random search. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIII. Springer, 484–501.

[2]

Giovanni Apruzzese, Hyrum S Anderson, Savino Dambra, David Freeman, Fabio Pierazzi, and Kevin A Roundy. 2022. " Real Attackers Don’t Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practice. arXiv preprint arXiv:2212.14315 (2022).

[3]

Giovanni Apruzzese and Michele Colajanni. 2018. Evading botnet detectors based on flows and random forest with adversarial samples. In 2018 IEEE 17th International Symposium on Network Computing and Applications (NCA). IEEE, 1–8.

[4]

Marco Barreno, Blaine Nelson, Russell Sears, Anthony D. Joseph, and J. D. Tygar. 2006. Can Machine Learning Be Secure?. In Proceedings of the 2006 ACM Symposium on Information, computer and communications security (Taipei, Taiwan) (ASIACCS ’06). Association for Computing Machinery, New York, NY, USA, 16–25.

Digital Library

[5]

Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. 2002. SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research 16 (2002), 321–357.

[6]

Mauro Conti, Luca Pajola, and Pier Paolo Tricomi. 2022. Captcha Attack: Turning Captchas Against Humanity. arXiv preprint arXiv:2201.04014 (2022).

[7]

Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of the international AAAI conference on web and social media, Vol. 11. 512–515.

[8]

Ambra Demontis, Marco Melis, Maura Pintor, Jagielski Matthew, Battista Biggio, Oprea Alina, Nita-Rotaru Cristina, Fabio Roli, 2019. Why do adversarial attacks transfer? explaining transferability of evasion and poisoning attacks. In 28th USENIX security symposium. USENIX Association, 321–338.

[9]

Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. 2019. Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks. 4307–4316. https://doi.org/10.1109/CVPR.2019.00444

[10]

Maayan Frid-Adar, Eyal Klang, Michal Amitai, Jacob Goldberger, and Hayit Greenspan. 2018. Synthetic data augmentation using GAN for improved liver lesion classification. In 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018). IEEE, 289–293.

[11]

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014).

[12]

Tommi Gröndahl, Luca Pajola, Mika Juuti, Mauro Conti, and N Asokan. 2018. All you need is" love" evading hate speech detection. In Proceedings of the 11th ACM workshop on artificial intelligence and security. 2–12.

Digital Library

[13]

Kathrin Grosse, Lukas Bieringer, Tarek Richard Besold, Battista Biggio, and Katharina Krombholz. 2022. " Why do so?"–A Practical Perspective on Machine Learning Security. arXiv preprint arXiv:2207.05164 (2022).

[14]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.

[15]

Hoki Kim. 2020. Torchattacks: A pytorch repository for adversarial attacks. arXiv preprint arXiv:2010.01950 (2020).

[16]

Alex Krizhevsky. 2014. One weird trick for parallelizing convolutional neural networks. arXiv preprint arXiv:1404.5997 (2014).

[17]

Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. 2018. Adversarial examples in the physical world. In Artificial intelligence safety and security. Chapman and Hall/CRC, 99–112.

[18]

Alyssa Lees, Vinh Q Tran, Yi Tay, Jeffrey Sorensen, Jai Gupta, Donald Metzler, and Lucy Vasserman. 2022. A new generation of perspective api: Efficient multilingual character-level transformers. arXiv preprint arXiv:2202.11176 (2022).

[19]

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017).

[20]

Yuhao Mao, Chong Fu, Saizhuo Wang, Shouling Ji, Xuhong Zhang, Zhenguang Liu, Jun Zhou, Alex X Liu, Raheem Beyah, and Ting Wang. 2022. Transfer Attacks Revisited: A Large-Scale Empirical Study in Real Computer Vision Settings. In 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 1423–1439.

[21]

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. 2016. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2574–2582.

[22]

Muhammad Muzammal Naseer, Salman H Khan, Muhammad Haris Khan, Fahad Shahbaz Khan, and Fatih Porikli. 2019. Cross-domain transferability of adversarial perturbations. Advances in Neural Information Processing Systems 32 (2019).

[23]

Luca Pajola and Mauro Conti. 2021. Fall of Giants: How popular text-based MLaaS fall against a simple evasion attack. In 2021 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, 198–211.

[24]

Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. 2016. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277 (2016).

[25]

Feargus Pendlebury, Fabio Pierazzi, Roberto Jordaney, Johannes Kinder, Lorenzo Cavallaro, 2019. TESSERACT: Eliminating experimental bias in malware classification across space and time. In Proceedings of the 28th USENIX Security Symposium. USENIX Association, 729–746.

[26]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

[27]

Octavian Suciu, Radu Marginean, Yigitcan Kaya, Hal Daume III, and Tudor Dumitras. 2018. When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attacks. In 27th USENIX Security Symposium (USENIX Security 18). USENIX Association, 1299–1316.

[28]

Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. 2017. Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv:1705.07204 (2017).

[29]

Xueping Wang, Shasha Li, Min Liu, Yaonan Wang, and Amit K Roy-Chowdhury. 2021. Multi-expert adversarial attack detection in person re-identification using context inconsistency. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 15097–15107.

[30]

Ruoyu Wu, Taegyu Kim, Dave Jing Tian, Antonio Bianchi, and Dongyan Xu. 2022. { DnD} : A { Cross-Architecture} Deep Neural Network Decompiler. In 31st USENIX Security Symposium (USENIX Security 22). 2135–2152.

[31]

Qixue Xiao, Yufei Chen, Chao Shen, Yu Chen, and Kang Li. 2019. Seeing is Not Believing: Camouflage Attacks on Image Scaling Algorithms. In USENIX Security Symposium. 443–460.

[32]

Qixue Xiao, Kang Li, Deyue Zhang, and Weilin Xu. 2018. Security risks in deep learning implementations. In 2018 IEEE Security and privacy workshops (SPW). IEEE, 123–128.

[33]

Kan Yuan, Di Tang, Xiaojing Liao, XiaoFeng Wang, Xuan Feng, Yi Chen, Menghan Sun, Haoran Lu, and Kehuan Zhang. 2019. Stealthy porn: Understanding real-world adversarial images for illicit online promotion. In 2019 IEEE Symposium on Security and Privacy (SP). IEEE, 952–966.

[34]

Jie Zhang, Bo Li, Jianghe Xu, Shuang Wu, Shouhong Ding, Lei Zhang, and Chao Wu. 2022. Towards efficient data free black-box adversarial attack. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 15115–15125.

Cited By

Zhu HZhao YZhang SChen K(2024)NeuralSanitizer: Detecting Backdoors in Neural NetworksIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.339059919(4970-4985)Online publication date: 2024
https://doi.org/10.1109/TIFS.2024.3390599
Marchiori FBrighente AConti M(2024)Work-in-Progress: Crash Course: Can (Under Attack) Autonomous Driving Beat Human Drivers?2024 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW)10.1109/EuroSPW61312.2024.00047(367-372)Online publication date: 8-Jul-2024
https://doi.org/10.1109/EuroSPW61312.2024.00047
Patel UBhilare SHati A(2024)Enhancing cross-domain transferability of black-box adversarial attacks on speaker recognition systems using linearized backpropagationPattern Analysis and Applications10.1007/s10044-024-01269-w27:2Online publication date: 13-May-2024
https://doi.org/10.1007/s10044-024-01269-w

Index Terms

Your Attack Is Too DUMB: Formalizing Attacker Scenarios for Adversarial Transferability
1. Computing methodologies
  1. Machine learning
2. Security and privacy
  1. Systems security

Recommendations

Improving the Transferability of Adversarial Attacks Through Both Front and Rear Vector Method
Digital Forensics and Watermarking
Abstract
Deep Neural Networks (DNNs) are vulnerable to adversarial attacks, which makes adversarial attacks serve as a method to evaluate the robustness of DNNs. However, adversarial attacks have the disadvantage of high white-box attack success rates but ...
Adversarial Machine Learning Attacks and Defense Methods in the Cyber Security Domain

In recent years, machine learning algorithms, and more specifically deep learning algorithms, have been widely used in many fields, including cyber security. However, machine learning systems are vulnerable to adversarial attacks, and this limits the ...
Machine Learning under Attack: Vulnerability Exploitation and Security Measures
IH&MMSec '16: Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security

Learning to discriminate between secure and hostile patterns is a crucial problem for species to survive in nature. Mimetism and camouflage are well-known examples of evolving weapons and defenses in the arms race between predators and preys. It is thus ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

RAID '23: Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses

October 2023

769 pages

ISBN:9798400707650

DOI:10.1145/3607199

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 October 2023

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

RAID 2023

RAID 2023: The 26th International Symposium on Research in Attacks, Intrusions and Defenses

October 16 - 18, 2023

Hong Kong, China

Acceptance Rates

Overall Acceptance Rate 43 of 173 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
474
Total Downloads

Downloads (Last 12 months)474
Downloads (Last 6 weeks)56

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhu HZhao YZhang SChen K(2024)NeuralSanitizer: Detecting Backdoors in Neural NetworksIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.339059919(4970-4985)Online publication date: 2024
https://doi.org/10.1109/TIFS.2024.3390599
Marchiori FBrighente AConti M(2024)Work-in-Progress: Crash Course: Can (Under Attack) Autonomous Driving Beat Human Drivers?2024 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW)10.1109/EuroSPW61312.2024.00047(367-372)Online publication date: 8-Jul-2024
https://doi.org/10.1109/EuroSPW61312.2024.00047
Patel UBhilare SHati A(2024)Enhancing cross-domain transferability of black-box adversarial attacks on speaker recognition systems using linearized backpropagationPattern Analysis and Applications10.1007/s10044-024-01269-w27:2Online publication date: 13-May-2024
https://doi.org/10.1007/s10044-024-01269-w
Efatinasab EMarchiori FBrighente ARampazzo MConti M(2024)FaultGuard: A Generative Approach to Resilient Fault Prediction in Smart Electrical GridsDetection of Intrusions and Malware, and Vulnerability Assessment10.1007/978-3-031-64171-8_26(503-524)Online publication date: 9-Jul-2024
https://doi.org/10.1007/978-3-031-64171-8_26

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents