abstract

Brief Announcement: Byzantine-Tolerant Machine Learning

Authors: Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, Julien StainerAuthors Info & Claims

PODC '17: Proceedings of the ACM Symposium on Principles of Distributed Computing

Pages 455 - 457

https://doi.org/10.1145/3087801.3087861

Published: 25 July 2017 Publication History

Get Access

Abstract

We report on Krum, the first provably Byzantine-tolerant aggregation rule for distributed Stochastic Gradient Descent (SGD). Krum guarantees the convergence of SGD even in a distributed setting where (asymptotically) up to half of the workers can be malicious adversaries trying to attack the learning system.

References

[1]

M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, et al. Tensorflow: A system for large-scale machine learning. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI). Savannah, Georgia, USA, 2016.

Digital Library

Google Scholar

[2]

P. Blanchard, E.M. El Mhamdi, R. Guerraoui, J. Stainer. Byzantine-TolerantMachine Learning. In arXiv preprint arXiv:1703.02757, 2017.

Google Scholar

[3]

L. Bottou. Online learning and stochastic approximations. Online learning in neural networks, 17(9):142, 1998.

Google Scholar

[4]

L. Bottou. Large-scale machine learning with stochastic gradient descent. In Proceedings of COMPSTAT'2010, pages 177--186. Springer, 2010.

Crossref

Google Scholar

[5]

L. Lamport, R. Shostak, and M. Pease. The byzantine generals problem. ACM Transactions on Programming Languages and Systems (TOPLAS), 4(3):382--401, 1982.

Digital Library

Google Scholar

[6]

X. Lian, Y. Huang, Y. Li, and J. Liu. Asynchronous parallel stochastic gradient for nonconvex optimization. In Advances in Neural Information Processing Systems, pages 2737--2745, 2015.

Digital Library

Google Scholar

[7]

N. A. Lynch. Distributed algorithms. Morgan Kaufmann, 1996.

Google Scholar

[8]

J. Markoff. How many computers to identify a cat? 16,000. New York Times, pages 06--25, 2012.

Google Scholar

[9]

H. Mendes and M. Herlihy. Multidimensional approximate agreement in byzantine asynchronous systems. In Proceedings of the forty-fifth annual ACM symposium on Theory of computing, pages 391--400. ACM, 2013.

Digital Library

Google Scholar

[10]

B. T. Polyak and A. B. Juditsky. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization, 30(4):838--855, 1992.

Digital Library

Google Scholar

[11]

F. B. Schneider. Implementing fault-tolerant services using the state machine approach: A tutorial. ACM Computing Surveys (CSUR), 22(4):299--319, 1990.

Digital Library

Google Scholar

[12]

S. Zhang, A. E. Choromanska, and Y. LeCun. Deep learning with elastic averaging sgd. In Advances in Neural Information Processing Systems, pages 685--693, 2015.

Digital Library

Google Scholar

Cited By

View all

Teo ZZhang XYang YJin LZhang CJieh Poh SYu WChen YJonas JWang YWu WLai CLiu YMong Goh RWei Ting D(2024)Privacy Preserving Technology using Federated Learning and Blockchain in protecting against Adversarial Attacks for Retinal ImagingOphthalmology10.1016/j.ophtha.2024.10.017Online publication date: Oct-2024
https://doi.org/10.1016/j.ophtha.2024.10.017
Kucharavy AMonti MGuerraoui RDolamic LSilva SPaquete L(2023)Byzantine-Resilient Learning Beyond Gradients: Distributing Evolutionary SearchProceedings of the Companion Conference on Genetic and Evolutionary Computation10.1145/3583133.3590719(295-298)Online publication date: 15-Jul-2023
https://dl.acm.org/doi/10.1145/3583133.3590719
Blanchard PEl Mhamdi EGuerraoui RStainer J(2017)Machine learning with adversariesProceedings of the 31st International Conference on Neural Information Processing Systems10.5555/3294771.3294783(118-128)Online publication date: 4-Dec-2017
https://dl.acm.org/doi/10.5555/3294771.3294783

Index Terms

Brief Announcement: Byzantine-Tolerant Machine Learning

Recommendations

Adversarial machine learning in IoT from an insider point of view
Abstract
With the rapid progress and significant successes in various applications, machine learning has been considered a crucial component in the Internet of Things ecosystem. However, machine learning models have recently been vulnerable to ...
Adversarial Machine Learning Attacks and Defense Methods in the Cyber Security Domain

In recent years, machine learning algorithms, and more specifically deep learning algorithms, have been widely used in many fields, including cyber security. However, machine learning systems are vulnerable to adversarial attacks, and this limits the ...
Adversarial Machine Learning Beyond the Image Domain
DAC '19: Proceedings of the 56th Annual Design Automation Conference 2019

Machine learning systems have had enormous success in a wide range of fields from computer vision, natural language processing, and anomaly detection. However, such systems are vulnerable to attackers who can cause deliberate misclassification by ...

Comments

Information & Contributors

Information

Published In

PODC '17: Proceedings of the ACM Symposium on Principles of Distributed Computing

July 2017

480 pages

ISBN:9781450349925

DOI:10.1145/3087801

General Chair:
Elad Michael Schiller
Chalmers University of Technology, Sweden
,
Program Chair:
Alexander A. Schwarzmann
University of Connecticut, USA

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 July 2017

Check for updates

Author Tags

Qualifiers

Abstract

Funding Sources

The Swiss National Science Foundation
European ERC

Conference

PODC '17

Sponsor:

PODC '17: ACM Symposium on Principles of Distributed Computing

July 25 - 27, 2017

DC, Washington, USA

Acceptance Rates

PODC '17 Paper Acceptance Rate 38 of 154 submissions, 25%;

Overall Acceptance Rate 740 of 2,477 submissions, 30%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
228
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)1

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Teo ZZhang XYang YJin LZhang CJieh Poh SYu WChen YJonas JWang YWu WLai CLiu YMong Goh RWei Ting D(2024)Privacy Preserving Technology using Federated Learning and Blockchain in protecting against Adversarial Attacks for Retinal ImagingOphthalmology10.1016/j.ophtha.2024.10.017Online publication date: Oct-2024
https://doi.org/10.1016/j.ophtha.2024.10.017
Kucharavy AMonti MGuerraoui RDolamic LSilva SPaquete L(2023)Byzantine-Resilient Learning Beyond Gradients: Distributing Evolutionary SearchProceedings of the Companion Conference on Genetic and Evolutionary Computation10.1145/3583133.3590719(295-298)Online publication date: 15-Jul-2023
https://dl.acm.org/doi/10.1145/3583133.3590719
Blanchard PEl Mhamdi EGuerraoui RStainer J(2017)Machine learning with adversariesProceedings of the 31st International Conference on Neural Information Processing Systems10.5555/3294771.3294783(118-128)Online publication date: 4-Dec-2017
https://dl.acm.org/doi/10.5555/3294771.3294783

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Adversarial machine learning in IoT from an insider point of view

Adversarial Machine Learning Attacks and Defense Methods in the Cyber Security Domain

Adversarial Machine Learning Beyond the Image Domain