Maximizing Feature Distribution Variance for Robust Neural Networks
Authors: 
Hao Yang, Min Wang, Zhengfei Yu, Zhi Zeng, Mingrui Lao, Yun ZhouAuthors Info & Claims
Hao Yang, Min Wang, Zhengfei Yu, Zhi Zeng, Mingrui Lao, Yun ZhouAuthors Info & ClaimsPages 9943 - 9951
Abstract
The security of Deep Neural Networks (DNNs) has proven to be critical for their applicabilities in real-world scenarios. However, DNNs are well-known to be vulnerable against adversarial attacks, such as adding artificially designed imperceptible magnitude perturbation to the benign input. Therefore, adversarial robustness is essential for DNNs to defend against malicious attacks. Stochastic Neural Networks (SNNs) have recently shown effective performance on enhancing adversarial robustness by injecting uncertainty into models. Nevertheless, existing SNNs are still limited for adversarial defense, as their insufficient representation capability from the fixed uncertainty. In this paper, to elevate feature representation capability of SNNs, we propose a novel yet practical stochastic neural network that maximizes feature distribution variance (MFDV-SNN). In addition, we provide theoretical insights to support the adversarial resistance of MFDV, which primarily derived from the stochastic noise we injected into DNNs. Our research demonstrates that by gradually increasing the level of stochastic noise in a DNN, the model naturally becomes more resistant to input perturbations. Since adversarial training is not required, MFDV-SNN does not compromise clean data accuracy and saves up to 7.5 times computation time. Extensive experiments on various attacks demonstrate that MFDV-SNN improves adversarial robustness significantly compared to other methods.
References
[1]
Anish Athalye, Nicholas Carlini, and David Wagner. 2018. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning. PMLR, 274--283.
[2]
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. 2018. Synthesizing robust adversarial examples. In International conference on machine learning. PMLR, 284--293.
[3]
Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp). Ieee, 39--57.
[4]
Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. 2017. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM workshop on artificial intelligence and security. 15--26.
[5]
Francesco Croce and Matthias Hein. 2020. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning. PMLR, 2206--2216.
[6]
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. In International Conference on Learning Representations.
[7]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.
[8]
Zhezhi He, Adnan Siraj Rakin, and Deliang Fan. 2019. Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness Against Adversarial Attack. In Proceedings of the IEEE conference on computer vision and pattern recognition. 588--597.
[9]
José Miguel Hernández-Lobato and Ryan Adams. 2015. Probabilistic backpropagation for scalable learning of bayesian neural networks. In International conference on machine learning. PMLR, 1861--1869.
[10]
Ahmadreza Jeddi, Mohammad Javad Shafiee, Michelle Karg, Christian Scharfenberger, and Alexander Wong. 2020. Learn2perturb: an end-to-end feature perturbation learning to improve adversarial robustness. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1241--1250.
[11]
Diederik P Kingma and Max Welling. 2014. Auto-encoding variational bayes. In International Conference on Learning Representations.
[12]
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2017. Adversarial machine learning at scale. In International Conference on Learning Representations.
[13]
Xuanqing Liu, Minhao Cheng, Huan Zhang, and Cho-Jui Hsieh. 2018. Towards robust neural networks via random self-ensemble. In Proceedings of the European Conference on Computer Vision. 369--385.
[14]
Xuanqing Liu, Yao Li, Chongruo Wu, and Cho-Jui Hsieh. 2019. Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network. In International Conference on Learning Representations.
[15]
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In International Conference on Learning Representations.
[16]
Rafael Pinot, Laurent Meunier, Alexandre Araujo, Hisashi Kashima, Florian Yger, Cédric Gouy-Pailler, and Jamal Atif. 2019. Theoretical evidence for adversarial robustness through randomization. In Advances in Neural Information Processing Systems.
[17]
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. 2019. One Pixel Attack for Fooling Deep Neural Networks. IEEE Transactions on Evolutionary Computation, Vol. 23, 5 (2019), 828--841.
[18]
Mingmin Wu, Yuxue Hu, Yongcheng Zhang, Zeng Zhi, Guixin Su, and Ying Sha. 2024. Mitigating Idiom Inconsistency: A Multi-Semantic Contrastive Learning Method for Chinese Idiom Reading Comprehension. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 19243--19251.
[19]
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan L. Yuille, and Kaiming He. 2019. Feature Denoising for Improving Adversarial Robustness. In Proceedings of the IEEE conference on computer vision and pattern recognition.
[20]
Hao Yang, Min Wang, Qi Wang, Zhengfei Yu, Guangyin Jin, Chunlai Zhou, and Yun Zhou. 2024. Non-informative noise-enhanced stochastic neural networks for improving adversarial robustness. Information Fusion, Vol. 108 (2024), 102397.
[21]
Hao Yang, Min Wang, Zhengfei Yu, and Yun Zhou. 2023. A simple stochastic neural network for improving adversarial robustness. In 2023 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2297--2302.
[22]
Hao Yang, Min Wang, Zhengfei Yu, and Yun Zhou. 2023. Weight-based regularization for improving robustness in image classification. In 2023 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1775--1780.
[23]
Hao Yang, Min Wang, Yun Zhou, and Yongxin Yang. 2021. Towards Stochastic Neural Network via Feature Distribution Calibration. In 2021 IEEE International Conference on Data Mining (ICDM). IEEE, 1445--1450.
[24]
Hao Yang and Yun Zhou. 2021. Ida-gan: A novel imbalanced data augmentation gan. In 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 8299--8305.
[25]
Tianyuan Yu, Yongxin Yang, Da Li, Timothy Hospedales, and Tao Xiang. 2021. Simple and effective stochastic neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 3252--3260.
[26]
Zhi Zeng, Mingmin Wu, Guodong Li, Xiang Li, Zhongqiang Huang, and Ying Sha. 2023. Correcting the Bias: Mitigating Multimodal Inconsistency Contrastive Learning for Multimodal Fake News Detection. In 2023 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2861--2866.
Index Terms
- Maximizing Feature Distribution Variance for Robust Neural Networks
Recommendations
A robust defense for spiking neural networks against adversarial examples via input filtering
AbstractSpiking Neural Networks (SNNs) are increasingly deployed in applications on resource constraint embedding systems due to their low power. Unfortunately, SNNs are vulnerable to adversarial examples which threaten the application security. Existing ...
Towards Defending Multiple -Norm Bounded Adversarial Perturbations via Gated Batch Normalization
AbstractThere has been extensive evidence demonstrating that deep neural networks are vulnerable to adversarial examples, which motivates the development of defenses against adversarial attacks. Existing adversarial defenses typically improve model ...
Non-informative noise-enhanced stochastic neural networks for improving adversarial robustness
AbstractStochastic Neural Networks (SNNs) have emerged as a promising tool for improving model adversarial robustness by injecting uncertainty into model activations or weights. However, the current implementations are dominated by injecting fixed ...
Graphical abstractDisplay Omitted
Highlights- Introduce a novel stochastic neural network.
- Extend beyond Gaussian by placing an arbitrary distribution over a non-informative prior.
- Augment model uncertainty through network training.
- The proposed method as an implicit ...
Comments
Information & Contributors
Information
Published In
October 2024
11719 pages
ISBN:9798400706868
DOI:10.1145/3664647
- General Chairs:
- Jianfei Cai,
- Mohan Kankanhalli,
- Balakrishnan Prabhakaran,
- Susanne Boll,
- Program Chairs:
- Ramanathan Subramanian,
- Liang Zheng,
- Vivek K. Singh,
- Pablo Cesar,
- Lexing Xie,
- Dong Xu
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Training Program for Excellent Young Innovators of Changsha
- Science and Technology Innovation Program of Hunan Province
- National Natural Science Foundation of China
Conference
MM '24
Sponsor:
MM '24: The 32nd ACM International Conference on Multimedia
October 28 - November 1, 2024
Melbourne VIC, Australia
Acceptance Rates
MM '24 Paper Acceptance Rate 1,150 of 4,385 submissions, 26%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 35Total Downloads
- Downloads (Last 12 months)35
- Downloads (Last 6 weeks)25
Reflects downloads up to 25 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in