research-article

Open access

GenAttack: practical black-box attacks with gradient-free optimization

Authors:

Moustafa Alzantot,

Supriyo Chakraborty,

Mani B. SrivastavaAuthors Info & Claims

GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference

Pages 1111 - 1119

https://doi.org/10.1145/3321707.3321749

Published: 13 July 2019 Publication History

Abstract

Deep neural networks are vulnerable to adversarial examples, even in the black-box setting, where the attacker is restricted solely to query access. Existing black-box approaches to generating adversarial examples typically require a significant number of queries, either for training a substitute network or performing gradient estimation. We introduce GenAttack, a gradient-free optimization technique that uses genetic algorithms for synthesizing adversarial examples in the black-box setting. Our experiments on different datasets (MNIST, CIFAR-10, and ImageNet) show that GenAttack can successfully generate visually imperceptible adversarial examples against state-of-the-art image recognition models with orders of magnitude fewer queries than previous approaches. Against MNIST and CIFAR-10 models, GenAttack required roughly 2,126 and 2,568 times fewer queries respectively, than ZOO, the prior state-of-the-art black-box attack. In order to scale up the attack to large-scale high-dimensional ImageNet models, we perform a series of optimizations that further improve the query efficiency of our attack leading to 237 times fewer queries against the Inception-v3 model than ZOO. Furthermore, we show that GenAttack can successfully attack some state-of-the-art ImageNet defenses, including ensemble adversarial training and non-differentiable or randomized input transformations. Our results suggest that evolutionary algorithms open up a promising area of research into effective black-box attacks.

References

[1]

Moustafa Alzantot, Bharathan Balaji, and Mani Srivastava. 2017. Did you hear that? adversarial examples against automatic speech recognition. Machine Deception Workshop, Neural Information Processing Systems (NIPS) 2017 (2017).

[2]

Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. 2018. Generating Natural Language Adversarial Examples. EMNLP: Conference on Empirical Methods in Natural Language Processing (2018).

[3]

A. Athalye, N. Carlini, and D. Wagner. 2018. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. arXiv preprint arXiv:1802.00420 (2018).

[4]

A. Athalye, L. Engstrom, A. Ilyas, and K. Kwok. 2017. Synthesizing robust adversarial examples. arXiv preprint arXiv:1707.07397 (2017).

[5]

Dinabandhu Bhandari, CA Murthy, and Sankar K Pal. 1996. Genetic algorithm with elitist model and its convergence. International journal of pattern recognition and artificial intelligence 10, 06 (1996), 731--747.

[6]

Wieland Brendel, Jonas Rauber, and Matthias Bethge. 2018. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. International Conference on Learning Representations (ICLR) (2018).

[7]

N. Carlini and D. Wagner. 2017. Towards evaluating the robustness of neural networks. arXiv preprint arXiv:1608.04644 (2017).

[8]

P. Y. Chen, Y. Sharma, H. Zhang, J. Yi, and C. Hsieh. 2017. EAD: Elasticnet attacks to deep neural networks via adversarial examples. arXiv preprint arXiv:1709.0414 (2017).

[9]

P. Y. Chen, H. Zhang, Y. Sharma, J. Yi, and C. Hsieh. 2017. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security. ACM, 15--26.

Digital Library

[10]

J. Deng, W. Dong, R. Socher, J. Li, K. Li, and L. Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 248--255.

[11]

I. Goodfellow, J. Shlens, and C. Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014).

[12]

S. Gu and L. Rigazio. 2014. Towards deep neural network architectures robust to adversarial examples. arXiv preprint arXiv:1412.5068 (2014).

[13]

C. Guo, M. Rana, and L. van der Maaten. 2017. Countering adversarial images using input transformations. arXiv preprint arXiv:1711.00117 (2017).

[14]

A. Ilyas, L. Engstrom, A. Athalye, and J. Lin. 2018. Black-box Adversarial Attacks with Limited Queries and Information. arXiv preprint arXiv:1804.08598 (2018).

[15]

D. Kingma and J. Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[16]

A. Kurakin, I. Goodfellow, and S. Bengio. 2016. Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533 (2016).

[17]

A. Kurakin, I. Goodfellow, S. Bengio, Y. Dong, F. Liao, M. Liang, T. Pang, J. Zhu, X. Hu, C. Xie, J. Wang, Z. Zhang, Z. Ren, A. Yuille, S. Huang, Y. Zhao, Y. Zhao, Z. Han, J Long, Y. Berdibekov, T. Akiba, S. Tokui, and M. Abe. 2018. Adversarial attacks and defences competition. arXiv preprint arXiv:1804.00097 (2018).

[18]

Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. 2018. Delving into transferable adversarial examples and black-box attacks. International Conference on Learning Representations (ICLR) (2018).

[19]

P. H. Lu, P. Y. Chen, K. C. Chen, and C. M. Yu. 2018. On the limitation of MagNet defense against L1-based adversarial examples. arXiv preprint arXiv:1805.00310 (2018).

[20]

A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. 2017. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017).

[21]

S. Moosavi-Dezfooli, A. Fawzi, and P. Frossard. 2016. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]

N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, and A. Swami. 2017. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security. ACM, 506--519.

Digital Library

[23]

Lukas Schott, Jonas Rauber, Matthias Bethge, and Wieland Brendel. 2019. Towards the first adversarially robust neural network model on MNIST. International Conference on Learning Representations (ICLR) (2019).

[24]

Y. Sharma and P. Y. Chen. 2017. Attacking the Madry defense model with L1-based adversarial examples. arXiv preprint arXiv:1710.10733 (2017).

[25]

Y. Sharma and P. Y. Chen. 2018. Bypassing feature squeezing by increasing adversary strength. arXiv preprint arXiv:1803.09868 (2018).

[26]

Y. Sharma, T. Le, and M. Alzantot. 2018. CAAD 2018: Generating Transferable Adversarial Examples. arXiv preprint arXiv:1810.01268 (2018).

[27]

Dong Su, Huan Zhang, Hongge Chen, Jinfeng Yi, Pin-Yu Chen, and Yupeng Gao. 2018. Is Robustness the Cost of Accuracy?-A Comprehensive Study on the Robustness of 18 Deep Image Classification Models. In Proceedings of the European Conference on Computer Vision (ECCV). 631--648.

[28]

C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. 2015. Going deeper with convolutions. CVPR.

[29]

C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, and I. Goodfellow. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013).

[30]

F. Tramér, A. Kurakin, N. Papernot, D. Boneh, and P. McDaniel. 2017. Ensemble adversarial training: Attacks and Defenses. arXiv preprint arXiv:1705.07204 (2017).

[31]

Chun-Chen Tu, Paishun Ting, Pin-Yu Chen, Sijia Liu, Huan Zhang, Jinfeng Yi, Cho-Jui Hsieh, and Shin-Ming Cheng. 2018. AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networks. arXiv preprint arXiv:1805.11770 (2018).

[32]

Huan Zhang, Hongge Chen, Zhao Song, Duane Boning, Inderjit S Dhillon, and Cho-Jui Hsieh. 2019. The Limitations of Adversarial Training and the Blind-Spot Attack. International Conference on Learning Representations (ICLR) (2019).

Cited By

Li CJiang TWang HYao WWang D(2025)Optimizing Latent Variables in Integrating Transfer and Query Based Attack FrameworkIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.346168647:1(161-171)Online publication date: Jan-2025
https://doi.org/10.1109/TPAMI.2024.3461686
Zhou CWang YWang ZKang X(2025) -norm distortion-efficient adversarial attack Signal Processing: Image Communication10.1016/j.image.2024.117241131(117241)Online publication date: Mar-2025
https://doi.org/10.1016/j.image.2024.117241
Etehadi-Abari MNaghsh-Nilchi AHoseinnezhad R(2025)An Optimized Non-deep Learning Defense Against Adversarial Attacks for Pedestrian DetectionJournal of Signal Processing Systems10.1007/s11265-024-01941-8Online publication date: 16-Jan-2025
https://doi.org/10.1007/s11265-024-01941-8
Show More Cited By

Index Terms

GenAttack: practical black-box attacks with gradient-free optimization
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Theory of computation
  1. Design and analysis of algorithms
    1. Mathematical optimization
      1. Discrete optimization
        Optimization with randomized search heuristics
        Evolutionary algorithms

Recommendations

Resisting Adversarial Examples via Wavelet Extension and Denoising
Smart Computing and Communication
Abstract
It is well known that Deep Neural Networks are vulnerable to adversarial examples. An adversary can inject carefully-crafted perturbations on clean input to manipulate the model output. In this paper, we propose a novel method, WED (Wavelet ...
Generate Usable Adversarial Examples via Simulating Additional Light Sources
Abstract
Deep neural networks have been shown to be critically vulnerable under adversarial attacks. This has led to the proliferation of methods to generate different adversarial examples from different perspectives. The adversarial examples generated by ...
Attack as defense: characterizing adversarial examples using robustness
ISSTA 2021: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis

As a new programming paradigm, deep learning has expanded its application to many real-world problems. At the same time, deep learning based software are found to be vulnerable to adversarial attacks. Though various defense mechanisms have been proposed ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference

July 2019

1545 pages

ISBN:9781450361118

DOI:10.1145/3321707

Editor:
Manuel López-Ibáñez
University of Manchester, UK
,
General Chairs:
Anne Auger
Inria and Ecole Polytechnique, France
,
Thomas Stützle
IRIDIA, Université libre de Bruxelles Belgium

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

GECCO '19

Sponsor:

SIGEVO

GECCO '19: Genetic and Evolutionary Computation Conference

July 13 - 17, 2019

Prague, Czech Republic

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

150
Total Citations
View Citations
2,117
Total Downloads

Downloads (Last 12 months)402
Downloads (Last 6 weeks)64

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li CJiang TWang HYao WWang D(2025)Optimizing Latent Variables in Integrating Transfer and Query Based Attack FrameworkIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.346168647:1(161-171)Online publication date: Jan-2025
https://doi.org/10.1109/TPAMI.2024.3461686
Zhou CWang YWang ZKang X(2025) -norm distortion-efficient adversarial attack Signal Processing: Image Communication10.1016/j.image.2024.117241131(117241)Online publication date: Mar-2025
https://doi.org/10.1016/j.image.2024.117241
Etehadi-Abari MNaghsh-Nilchi AHoseinnezhad R(2025)An Optimized Non-deep Learning Defense Against Adversarial Attacks for Pedestrian DetectionJournal of Signal Processing Systems10.1007/s11265-024-01941-8Online publication date: 16-Jan-2025
https://doi.org/10.1007/s11265-024-01941-8
Lee HChoi KKwon DPark SJaiswal MPark NChoi JLee JSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)DataFreeShieldProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693127(26515-26545)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693127
Cheng SMiao YDong YYang XGao XZhu JSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Efficient black-box adversarial attacks via Bayesian optimization guided by a function priorProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692392(8163-8183)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3692392
Cheng ZLiu ZGuo TFeng SLiu DTang MZhang XSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)BadPartProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692389(8104-8122)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3692389
Zhang FLi KRen Z(2024)Improving Adversarial Robustness of Ensemble Classifiers by Diversified Feature Selection and Stochastic AggregationMathematics10.3390/math1206083412:6(834)Online publication date: 12-Mar-2024
https://doi.org/10.3390/math12060834
Mathkoor MJassim RAl-Sabur R(2024)Application of Pattern Search and Genetic Algorithms to Optimize HDPE Pipe Joint Profiles and Strength in the Butt Fusion Welding ProcessJournal of Manufacturing and Materials Processing10.3390/jmmp80501878:5(187)Online publication date: 25-Aug-2024
https://doi.org/10.3390/jmmp8050187
Gao JZheng KWang XWu CWu B(2024)EFSAttack: Edge Noise-Constrained Black-Box Attack Using Artificial Fish Swarm AlgorithmElectronics10.3390/electronics1313244613:13(2446)Online publication date: 21-Jun-2024
https://doi.org/10.3390/electronics13132446
Shan JMa HLi J(2024)Research on Network Attack Sample Generation and Defence Techniques Based on Generative Adversarial NetworksApplied Mathematics and Nonlinear Sciences10.2478/amns-2024-35509:1Online publication date: 27-Nov-2024
https://doi.org/10.2478/amns-2024-3550
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten