short-paper

Adversarial Training of Gradient-Boosted Decision Trees

Authors:

Stefano Calzavara,

Claudio Lucchese,

Gabriele TolomeiAuthors Info & Claims

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Pages 2429 - 2432

https://doi.org/10.1145/3357384.3358149

Published: 03 November 2019 Publication History

Get Access

Abstract

Adversarial training is a prominent approach to make machine learning (ML) models resilient to adversarial examples. Unfortunately, such approach assumes the use of differentiable learning models, hence it cannot be applied to relevant ML techniques, such as ensembles of decision trees. In this paper, we generalize adversarial training to gradient-boosted decision trees (GBDTs). Our experiments show that the performance of classifiers based on existing learning techniques either sharply decreases upon attack or is unsatisfactory in absence of attacks, while adversarial training provides a very good trade-off between resiliency to attacks and accuracy in the unattacked setting.

References

[1]

Biggio, B., and Roli, F. Wild patterns: Ten years after the rise of adversarial machine learning. CoRR abs/1712.03141 (2017).

Google Scholar

[2]

Breiman, L., Friedman, J. H., Olshen, R. A., and Stone, C. J. Classification and Regression Trees. Wadsworth, 1984.

Google Scholar

[3]

Chollet, F. Deep Learning with Python, 1st ed. Manning Publications Co., Greenwich, CT, USA, 2017.

Digital Library

Google Scholar

[4]

Friedman, J. H. Greedy function approximation: a gradient boosting machine. Annals of statistics (2001), 1189--1232.

Google Scholar

[5]

Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. CoRR abs/1412.6572 (2014).

Google Scholar

[6]

Huang, L., Joseph, A. D., Nelson, B., Rubinstein, B. I. P., and Tygar, J. D. Adversarial machine learning. In AISec (2011), pp. 43--58.

Digital Library

Google Scholar

[7]

Kantchelian, A., Tygar, J. D., and Joseph, A. D. Evasion and hardening of tree ensemble classifiers. In ICML (2016), pp. 2387--2396.

Google Scholar

[8]

Kurakin, A., Goodfellow, I. J., and Bengio, S. Adversarial machine learning at scale. CoRR abs/1611.01236 (2016).

Google Scholar

[9]

Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. Towards deep learning models resistant to adversarial attacks. CoRR abs/1706.06083 (2017).

Google Scholar

[10]

Moosavi-Dezfooli, S., Fawzi, A., and Frossard, P. Deepfool: A simple and accurate method to fool deep neural networks. In CVPR (2016), pp. 2574--2582.

Crossref

Google Scholar

[11]

Ng, A. Y. Feature selection, l1 vs. l2 regularization, and rotational invariance. In ICML '04 (2004), ACM, pp. 78--.

Digital Library

Google Scholar

[12]

Rumelhart, D. E., Hinton, G. E., and Williams, R. J. Parallel distributed processing: Explorations in the microstructure of cognition, vol. 1. Cambridge, MA, USA, 1986, ch. Learning Internal Representations by Error Propagation, pp. 318--362.

Google Scholar

[13]

Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I. J., and Fergus, R. Intriguing properties of neural networks. CoRR abs/1312.6199 (2013).

Google Scholar

Cited By

View all

Yang YLin CLi QZhao ZFan HZhou DWang NLiu TShen C(2024)Quantization Aware Attack: Enhancing Transferable Adversarial Attacks by Model QuantizationIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.336089119(3265-3278)Online publication date: 2024
https://doi.org/10.1109/TIFS.2024.3360891
Calzavara SFerrara PLucchese C(2023)Certifying machine learning models against evasion attacks by program analysisJournal of Computer Security10.3233/JCS-21013331:1(57-84)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.3233/JCS-210133
Khan SMynuddin MAdom IMahmoud M(2023)Mitigating Targeted Universal Adversarial Attacks on Time Series Power Quality Disturbances Models2023 5th IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA)10.1109/TPS-ISA58951.2023.00021(91-100)Online publication date: 1-Nov-2023
https://doi.org/10.1109/TPS-ISA58951.2023.00021
Show More Cited By

Index Terms

Adversarial Training of Gradient-Boosted Decision Trees
1. Information systems
  1. Information systems applications
    1. Data mining
2. Security and privacy
  1. Formal methods and theory of security

Recommendations

Genetic adversarial training of decision trees
GECCO '21: Proceedings of the Genetic and Evolutionary Computation Conference

We put forward a novel learning methodology for ensembles of decision trees based on a genetic algorithm that is able to train a decision tree for maximizing both its accuracy and its robustness to adversarial perturbations. This learning algorithm ...
Stochastic gradient boosted distributed decision trees
CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management

Stochastic Gradient Boosted Decision Trees (GBDT) is one of the most widely used learning algorithms in machine learning today. It is adaptable, easy to interpret, and produces highly accurate models. However, most implementations today are ...
Learning multiple tasks with boosted decision trees
ECMLPKDD'12: Proceedings of the 2012th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I

We address the problem of multi-task learning with no label correspondence among tasks. Learning multiple related tasks simultaneously, by exploiting their shared knowledge can improve the predictive performance on every task. We develop the multi-task ...

Comments

Information & Contributors

Information

Published In

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

November 2019

3373 pages

ISBN:9781450369763

DOI:10.1145/3357384

General Chairs:
Wenwu Zhu
Tsinghua University, China
,
Dacheng Tao
University of Massachusetts, USA
,
Xueqi Cheng
Institute of Computing Technology, CAS, China
,
Program Chairs:
Peng Cui
Tsinghua University, China
,
Elke Rundensteiner
Worcester Polytechnic Institute, USA
,
David Carmel
Amazon Research, USA
,
Qi He
LinkedIn, USA
,
Jeffrey Xu Yu
Chinese University of Hong Kong, China

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

CIKM '19

Sponsor:

CIKM '19: The 28th ACM International Conference on Information and Knowledge Management

November 3 - 7, 2019

Beijing, China

Acceptance Rates

CIKM '19 Paper Acceptance Rate 202 of 1,031 submissions, 20%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
342
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Yang YLin CLi QZhao ZFan HZhou DWang NLiu TShen C(2024)Quantization Aware Attack: Enhancing Transferable Adversarial Attacks by Model QuantizationIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.336089119(3265-3278)Online publication date: 2024
https://doi.org/10.1109/TIFS.2024.3360891
Calzavara SFerrara PLucchese C(2023)Certifying machine learning models against evasion attacks by program analysisJournal of Computer Security10.3233/JCS-21013331:1(57-84)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.3233/JCS-210133
Khan SMynuddin MAdom IMahmoud M(2023)Mitigating Targeted Universal Adversarial Attacks on Time Series Power Quality Disturbances Models2023 5th IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA)10.1109/TPS-ISA58951.2023.00021(91-100)Online publication date: 1-Nov-2023
https://doi.org/10.1109/TPS-ISA58951.2023.00021
Owezarski P(2023)Investigating adversarial attacks against Random Forest-based network attack detection systemsNOMS 2023-2023 IEEE/IFIP Network Operations and Management Symposium10.1109/NOMS56928.2023.10154328(1-6)Online publication date: 8-May-2023
https://doi.org/10.1109/NOMS56928.2023.10154328
Shah AMuhammad WLee K(2022)Examining the Determinants of Patient Perception of Physician Review Helpfulness across Different Disease SeveritiesComputational Intelligence and Neuroscience10.1155/2022/86235862022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/8623586
Nanfack GTemple PFrénay B(2022)Constraint Enforcement on Decision Trees: A SurveyACM Computing Surveys10.1145/350673454:10s(1-36)Online publication date: 13-Sep-2022
https://dl.acm.org/doi/10.1145/3506734
Krichen MMihoub AAlzahrani MAdoni WNahhal T(2022)Are Formal Methods Applicable To Machine Learning And Artificial Intelligence?2022 2nd International Conference of Smart Systems and Emerging Technologies (SMARTTECH)10.1109/SMARTTECH54121.2022.00025(48-53)Online publication date: May-2022
https://doi.org/10.1109/SMARTTECH54121.2022.00025
Calzavara SCazzaro LLucchese CMarcuzzi FOrlando S(2022)Beyond robustnessComputers and Security10.1016/j.cose.2022.102843121:COnline publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1016/j.cose.2022.102843
Han YRoundy KTamersoy A(2021)Towards Stalkerware Detection with Precise WarningsProceedings of the 37th Annual Computer Security Applications Conference10.1145/3485832.3485901(957-969)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.1145/3485832.3485901
Ranzato FUrban CZanella MDemartini GZuccon GCulpepper JHuang ZTong H(2021)Fairness-Aware Training of Decision Trees by Abstract InterpretationProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482342(1508-1517)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482342
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Genetic adversarial training of decision trees

Stochastic gradient boosted distributed decision trees

Learning multiple tasks with boosted decision trees