Have You Poisoned My Data? Defending Neural Networks Against Data Poisoning
Pages 85 - 104
Abstract
The unprecedented availability of training data fueled the rapid development of powerful neural networks in recent years. However, the need for such large amounts of data leads to potential threats such as poisoning attacks: adversarial manipulations of the training data aimed at compromising the learned model to achieve a given adversarial goal.
This paper investigates defenses against clean-label poisoning attacks and proposes a novel approach to detect and filter poisoned datapoints in the transfer learning setting. We define a new characteristic vector representation of datapoints and show that it effectively captures the intrinsic properties of the data distribution. Through experimental analysis, we demonstrate that effective poison datapoints can be successfully differentiated from clean datapoints in the characteristic vector space. We thoroughly evaluate our proposed approach and compare it to existing state-of-the-art defenses using multiple architectures, datasets, and poison budgets. Our evaluation shows that our proposal outperforms existing approaches in defense rate and final trained model performance across all experimental settings.
References
[1]
Aghakhani, H., Meng, D., Wang, Y.X., Kruegel, C., Vigna, G.: Bullseye polytope: a scalable clean-label poisoning attack with improved transferability. In: IEEE European Symposium on Security and Privacy, EuroS &P, pp. 159–178 (2021)
[2]
Borgnia, E., et al.: Strong data augmentation sanitizes poisoning and backdoor attacks without an accuracy tradeoff. In: IEEE International Conference on Acoustics, Speech and Signal Processing. ICASSP, pp. 3855–3859 (2021)
[3]
Chen, B., et al.: Detecting backdoor attacks on deep neural networks by activation clustering. In: AAAI’s Workshop on Artificial Intelligence Safety. SafeAI (2018)
[4]
Cinà AE et al. Wild patterns reloaded: a survey of machine learning security against training data poisoning ACM Comput. Surv. 2023 55 13s 1-39
[5]
Darlow, L.N., Crowley, E.J., Antoniou, A., Storkey, A.J.: CINIC-10 is not imagenet or CIFAR-10. arXiv preprint arXiv:1810.03505 (2018)
[6]
De Gaspari F, Hitaj D, Pagnotta G, De Carli L, and Mancini LV Evading behavioral classifiers: a comprehensive analysis on evading ransomware detection techniques Neural Comput. Appl. 2022 34 14 12077-12096
[7]
De Gaspari F, Hitaj D, Pagnotta G, De Carli L, and Mancini LV Reliable detection of compressed and encrypted data Neural Comput. Appl. 2022 34 22 20379-20393
[8]
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 248–255 (2009)
[9]
Fowl, L., Geiping, J., Somepalli, G., Goldstein, T., Taylor, G.: Industrial scale data poisoning (2023). https://github.com/JonasGeiping/data-poisoning
[10]
Geiping, J., Fowl, L., Somepalli, G., Goldblum, M., Moeller, M., Goldstein, T.: What doesn’t kill you makes you robust (ER): how to adversarially train against data poisoning. In: ICLR Workshop on Security and Safety in Machine Learning Systems (2021)
[11]
Geiping, J., et al.: Witches’ brew: industrial scale data poisoning via gradient matching. In: International Conference on Learning Representations. ICLR (2020)
[12]
Hitaj, D., et al.: Do you trust your model? Emerging malware threats in the deep learning ecosystem. arXiv preprint arXiv:2403.03593 (2024)
[13]
Hitaj, D., Pagnotta, G., Hitaj, B., Mancini, L.V., Perez-Cruz, F.: MaleficNet: hiding malware into deep neural networks using spread-spectrum channel coding. In: European Symposium on Research in Computer Security, ESORIC, pp. 425–444S (2022)
[14]
Hitaj D, Pagnotta G, Hitaj B, Perez-Cruz F, and Mancini LV FedComm: federated learning as a medium for covert communication IEEE Trans. Depend. Secure Comput. 2023 21 1695-1707
[15]
Hong, S., Chandrasekaran, V., Kaya, Y., Dumitraş, T., Papernot, N.: On the effectiveness of mitigating data poisoning attacks with gradient shaping. arXiv preprint arXiv:2002.11497 (2020)
[16]
Koh, P.W., Steinhardt, J., Liang, P.: Stronger data poisoning attacks break data sanitization defenses. Mach. Learning, 1–47 (2022)
[17]
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
[18]
Levine, A., Feizi, S.: Deep partition aggregation: provable defenses against general poisoning attacks. In: International Conference on Learning Representations. ICLR (2020)
[19]
Li Y, Lyu X, Koren N, Lyu L, Li B, and Ma X Anti-backdoor learning: training clean models on poisoned data Adv. Neural. Inf. Process. Syst. 2021 34 14900-14912
[20]
Liu, Y., et al.: Trojaning attack on neural networks. In: 25th Annual Network And Distributed System Security Symposium, NDSS (2018)
[21]
Ma, Y., Zhu, X., Hsu, J.: Data poisoning against differentially-private learners: attacks and defenses. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 4732–4738. AAAI (2019)
[22]
Meta: Code llama (2023). https://github.com/facebookresearch/llama
[23]
Meta: Llama 2 (2023). https://github.com/facebookresearch/llama
[24]
Miller DJ, Xiang Z, and Kesidis G Adversarial learning targeting deep neural network classification: a comprehensive review of defenses against attacks Proc. IEEE 2020 108 3 402-433
[25]
Nguyen, T.A., Tran, A.: Input-aware dynamic backdoor attack. Adv. Neural Inf. Process. Syst., 3454–3464 (2020)
[26]
Pagnotta G, De Gaspari F, Hitaj D, Andreolini M, Colajanni M, and Mancini LV DOLOS: a novel architecture for moving target defense IEEE Trans. Inf. Forensics Secur. 2023 18 5890-5905
[27]
Pagnotta, G., Hitaj, D., De Gaspari, F., Mancini, L.V.: PassFlow: guessing passwords with generative flows. In: 2022 52nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 251–262. IEEE (2022)
[28]
Paudice, A., Muñoz-González, L., Lupu, E.C.: Label sanitization against label flipping poisoning attacks. In: ECML PKDD 2018 Workshops, pp. 5–15. ECML PKDD (2019)
[29]
Peri N et al. Bartoli A, Fusiello A, et al. Deep k-NN defense against clean-label data poisoning attacks Computer Vision – ECCV 2020 Workshops 2020 Cham Springer 55-70
[30]
Piskozub, M., De Gaspari, F., Barr-Smith, F., Mancini, L., Martinovic, I.: MalPhase: fine-grained malware detection using network flow data. In: ACM Asia Conference on Computer and Communications Security, ASIACCS, pp. 774–786 (2021)
[31]
Schwarzschild, A., Goldblum, M., Gupta, A., Dickerson, J.P., Goldstein, T.: Just how toxic is data poisoning? A unified benchmark for backdoor and data poisoning attacks. In: International Conference on Machine Learning. ICML (2021)
[32]
Shafahi, A., et al.: Poison frogs! Targeted clean-label poisoning attacks on neural networks. In: Advances in Neural Information Processing Systems. NIPS (2018)
[33]
Shejwalkar, V., Houmansadr, A., Kairouz, P., Ramage, D.: Back to the drawing board: a critical evaluation of poisoning attacks on production federated learning. In: IEEE Symposium on Security and Privacy, pp. 1354–1371 (2022)
[34]
Shokri, R., et al.: Bypassing backdoor detection algorithms in deep learning. In: IEEE European Symposium on Security and Privacy, EuroS &P, pp. 175–183 (2020)
[35]
Steinhardt, J., Koh, P.W.W., Liang, P.S.: Certified defenses for data poisoning attacks. Adv. Neural Inf. Process. Syst. 30 (2017)
[36]
Tian Z, Cui L, Liang J, and Yu S A comprehensive survey on poisoning attacks and countermeasures in machine learning ACM Comput. Surv. 2022 55 8 1-35
[37]
Touvron, H., et al.: LLaMA: open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)
[38]
Tran, B., Li, J., Madry, A.: Spectral signatures in backdoor attacks. In: Advances in Neural Information Processing Systems. NIPS (2018)
[39]
Weber, M., Xu, X., Karlaš, B., Zhang, C., Li, B.: RAB: provable robustness against backdoor attacks. In: IEEE Symposium on Security and Privacy, pp. 1311–1328. S &P (2023)
[40]
Yang, Y., Liu, T.Y., Mirzasoleiman, B.: Not all poisons are created equal: robust training against data poisoning. In: International Conference on Machine Learning. ICML (2022)
[41]
Yin, H., et al.: Dreaming to distill: data-free knowledge transfer via deepinversion. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8715–8724. CVPR (2020)
[42]
Zhu, C., Huang, W.R., Li, H., Taylor, G., Studer, C., Goldstein, T.: Transferable clean-label poisoning attacks on deep neural nets. In: International Conference on Machine Learning, pp. 7614–7623. ICML (2019)
Index Terms
- Have You Poisoned My Data? Defending Neural Networks Against Data Poisoning
Index terms have been assigned to the content through auto-classification.
Recommendations
Defending Against Adversarial Denial-of-Service Data Poisoning Attacks
DYNAMICS '20: Proceedings of the 2020 Workshop on DYnamic and Novel Advances in Machine Learning and Intelligent Cyber SecurityData poisoning is one of the most relevant security threats against machine learning and data-driven technologies. Since many applications rely on untrusted training data, an attacker can easily craft malicious samples and inject them into the training ...
Stronger data poisoning attacks break data sanitization defenses
AbstractMachine learning models trained on data from the outside world can be corrupted by data poisoning attacks that inject malicious points into the models’ training sets. A common defense against these attacks is data sanitization: first filter out ...
Data poisoning attacks against machine learning algorithms
AbstractFor the past decade, machine learning technology has increasingly become popular and it has been contributing to many areas that have the potential to influence the society considerably. Generally, machine learning is used by various ...
Highlights- A new approach to analyze robustness of machine learning.
- Machine learning ...
Comments
Information & Contributors
Information
Published In
Sep 2024
410 pages
ISBN:978-3-031-70878-7
DOI:10.1007/978-3-031-70879-4
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 16 September 2024
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 19 Feb 2025