Article

Using machine teaching to identify optimal training-set attacks on machine learners

Authors:

Xiaojin ZhuAuthors Info & Claims

AAAI'15: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence

Pages 2871 - 2877

Published: 25 January 2015 Publication History

Abstract

We investigate a problem at the intersection of machine learning and security: training-set attacks on machine learners. In such attacks an attacker contaminates the training data so that a specific learning algorithm would produce a model profitable to the attacker. Understanding training-set attacks is important as more intelligent agents (e.g. spam filters and robots) are equipped with learning capability and can potentially be hacked via data they receive from the environment. This paper identifies the optimal training-set attack on a broad family of machine learners. First we show that optimal training-set attack can be formulated as a bilevel optimization problem. Then we show that for machine learners with certain Karush-Kuhn-Tucker conditions we can solve the bilevel problem efficiently using gradient methods on an implicit function. As examples, we demonstrate optimal training-set attacks on Support Vector Machines, logistic regression, and linear regression with extensive experiments. Finally, we discuss potential defenses against such attacks.

References

[1]

Bache, K., and Lichman, M. 2013. UCI machine learning repository.

[2]

Bard, J. F. 1998. Practical Bilevel Optimization: Algorithms And Applications. Kluwer Academic Publishers.

[3]

Barreno, M.; Nelson, B.; Sears, R.; Joseph, A.; and Tygar, J. 2006. Can machine learning be secure? In CCS.

[4]

Barreno, M.; Nelson, B.; Joseph, A. D.; and Tygar, J. 2010. The security of machine learning. Machine Learning Journal 81(2):121-148.

[5]

Biggio, B.; Fumera, G.; and Roli, F. 2013. Security evaluation of pattern classifiers under attack. IEEE TKDE.

[6]

Biggio, B.; Nelson, B.; and Laskov, P. 2012. Poisoning attacks against support vector machines. In ICML.

[7]

Burges, C. 1998. A tutorial on support vector machines for pattern recognition. Knowledge Discovery and Data Mining 2(2).

[8]

Chung, S., and Mok, A. 2007. Advanced allergy attacks: Does a corpus really help. In RAID.

[9]

Colson, B.; Marcotte, P.; and Savard, G. 2007. An overview of bilevel optimization. Annals of operations research 153(1):235-256.

[10]

Cortez, P.; Cerdeira, A.; Almeida, F.; Matos, T.; and Reis, J. 2009. Modeling wine preferences by data mining from physicochemical properties. Decision Support Systems 47(4):547-553.

[11]

Dalvi, N.; Domingos, P.; Mausam; Sanghai, S.; and Verma, D. 2004. Adversarial classification. In SIGKDD.

[12]

Dekel, O.; Shamir, O.; and Xiao, L. 2010. Learning to classify with missing and corrupted features. Machine learning 81(2):149-178.

[13]

El Ghaoui, L.; Lanckriet, G. R. G.; Natsoulis, G.; et al. 2003. Robust classification with interval data. Computer Science Division, University of California.

[14]

Fan, R.-E.; Chang, K.-W.; Hsieh, C.-J.; Wang, X.-R.; and Lin, C.-J. 2008. Liblinear: A library for large linear classification. The Journal of Machine Learning Research 9:1871-1874.

[15]

Globerson, A., and Roweis, S. T. 2006. Nightmare at test time: robust learning by feature deletion. In ICML.

[16]

Kim, S.-J.; Magnani, A.; and Boyd, S. 2005. Robust Fisher discriminant analysis. In Advances in Neural Information Processing Systems, 659-666.

[17]

Laskov, P., and Kloft, M. 2009. A framework for quantitative security analysis of machine learning. In The 2nd ACM Workshop on AISec.

[18]

Laskov, P., and Lippmann, R. 2010. Machine learning in adversarial environments. Machine Learning 81(2):115-119.

[19]

Liu, W., and Chawla, S. 2009. A game theoretical model for adversarial learning. In ICDM Workshops.

[20]

Lowd, D., and Meek, C. 2005. Good word attacks on statistical spam filters. In CEAS.

[21]

Mei, S., and Zhu, X. 2014. Using machine teaching to identify optimal training-set attacks on machine learners. Technical Report Computer Science TR1813, University of Wisconsin-Madison.

[22]

Nelson, B.; Barreno, M.; Chi, F.; Joseph, A.; Rubinstein, B.; Saini, U.; Sutton, C.; Tygar, J.; and Xia, K. 2009. Misleading learners: Co-opting your spam filter. In Machine Learning in Cyber Trust: Security, Privacy, Reliability. Springer.

[23]

Patil, K.; Zhu, X.; Kopec, L.; and Love, B. 2014. Optimal teaching for limited-capacity human learners. In Advances in Neural Information Processing Systems (NIPS).

[24]

Rubinstein, B.; Nelson, B.; Huang, L.; Joseph, A.; Lau, S.; Taft, N.; and Tygar, D. 2008. Compromising PCA-based anomaly detectors for network-wide traffic. Technical Report UCB/EECS-2008-73, EECS Department, University of California, Berkeley.

[25]

Tan, K.; Killourhy, K.; and Maxion, R. 2002. Undermining an anomaly-based intrusion detection system using common exploits. In RAID.

[26]

Torkamani, M., and Lowd, D. 2013. Convex adversarial collective classification. In Proceedings of The 30th International Conference on Machine Learning, 642-650.

[27]

Wittel, G., and Wu, S. 2004. On Attacking Statistical Spam Filters. In Proc. of the Conference on Email and Anti-Spam (CEAS).

[28]

Xiao, H.; Xiao, H.; and Eckert, C. 2012. Adversarial label flips attack on support vector machines. In ECAI.

[29]

Xu, H.; Caramanis, C.; and Mannor, S. 2009. Robustness and regularization of support vector machines. The Journal of Machine Learning Research 10:1485-1510.

[30]

Zhu, X. 2013. Machine teaching for Bayesian learners in the exponential family. In NIPS.

Cited By

Zhang JZhang CLi GChai C(2024)PACE: Poisoning Attacks on Learned Cardinality EstimationProceedings of the ACM on Management of Data10.1145/36392922:1(1-27)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3639292
Chen PYang YYang DSun HChen ZLin PElkind E(2023)Black-box data poisoning attacks on crowdsourcingProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/332(2975-2983)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/332
Çelikok MMurena PKaski SWilliams BChen YNeville J(2023)Teaching to learnProceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v37i5.25735(5939-5947)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1609/aaai.v37i5.25735
Show More Cited By

Index Terms

Using machine teaching to identify optimal training-set attacks on machine learners
1. Computing methodologies
  1. Machine learning

Index terms have been assigned to the content through auto-classification.

Recommendations

Blind attacks on machine learners
NIPS'16: Proceedings of the 30th International Conference on Neural Information Processing Systems

The importance of studying the robustness of learners to malicious data is well established. While much work has been done establishing both robust estimators and effective data injection attacks when the attacker is omniscient, the ability of an ...
Detecting Blind Cross-Site Scripting Attacks Using Machine Learning
SPML '18: Proceedings of the 2018 International Conference on Signal Processing and Machine Learning

Cross-site scripting (XSS) is a scripting attack targeting web applications by injecting malicious scripts into web pages. Blind XSS is a subset of stored XSS, where an attacker blindly deploys malicious payloads in web pages that are stored in a ...
Machine learning combating DOS and DDOS attacks

In recent years, technology is booming at a breakneck speed as so the need of security. Vulnerabilities in the layers of the OSI model and the networks are paving new ways for intruders and hackers to steal the confidential information. Security attacks ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

AAAI'15: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence

January 2015

4331 pages

ISBN:0262511290

Sponsors

Association for the Advancement of Artificial Intelligence

Publisher

AAAI Press

Publication History

Published: 25 January 2015

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

41
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang JZhang CLi GChai C(2024)PACE: Poisoning Attacks on Learned Cardinality EstimationProceedings of the ACM on Management of Data10.1145/36392922:1(1-27)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3639292
Chen PYang YYang DSun HChen ZLin PElkind E(2023)Black-box data poisoning attacks on crowdsourcingProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/332(2975-2983)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/332
Çelikok MMurena PKaski SWilliams BChen YNeville J(2023)Teaching to learnProceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v37i5.25735(5939-5947)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1609/aaai.v37i5.25735
Ilbert RHoang TZhang ZPalpanas TBlanc GTakahashi TZhang Z(2023)Breaking Boundaries: Balancing Performance and Robustness in Deep Wireless Traffic ForecastingProceedings of the 2023 Workshop on Recent Advances in Resilient and Trustworthy ML Systems in Autonomous Networks10.1145/3605772.3624002(17-28)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3605772.3624002
Kerkouche RÁcs GFritz MKnijnenburg BPapadimitratos P(2023)Client-specific Property Inference against Secure Aggregation in Federated LearningProceedings of the 22nd Workshop on Privacy in the Electronic Society10.1145/3603216.3624964(45-60)Online publication date: 26-Nov-2023
https://dl.acm.org/doi/10.1145/3603216.3624964
Cinà AGrosse KDemontis AVascon SZellinger WMoser BOprea ABiggio BPelillo MRoli F(2023)Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data PoisoningACM Computing Surveys10.1145/358538555:13s(1-39)Online publication date: 13-Jul-2023
https://dl.acm.org/doi/10.1145/3585385
Tian ZCui LLiang JYu S(2022)A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine LearningACM Computing Surveys10.1145/355163655:8(1-35)Online publication date: 23-Dec-2022
https://dl.acm.org/doi/10.1145/3551636
Franci ACordy MGubri MPapadakis MTraon YCrnkovic I(2022)Influence-driven data poisoning in graph-based semi-supervised classifiersProceedings of the 1st International Conference on AI Engineering: Software Engineering for AI10.1145/3522664.3528606(77-87)Online publication date: 16-May-2022
https://dl.acm.org/doi/10.1145/3522664.3528606
Liu WLiu ZWang HPaull LSchölkopf BWeller ARanzato MBeygelzimer ADauphin YLiang PVaughan J(2021)Iterative teaching by label synthesisProceedings of the 35th International Conference on Neural Information Processing Systems10.5555/3540261.3541920(21681-21695)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.5555/3540261.3541920
Xu ZChen BLi CLiu WSong LLin YShrivastava ARanzato MBeygelzimer ADauphin YLiang PVaughan J(2021)Locality sensitive teachingProceedings of the 35th International Conference on Neural Information Processing Systems10.5555/3540261.3541642(18049-18062)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.5555/3540261.3541642
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents