Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3534678.3542631acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
abstract

A Practical Introduction to Federated Learning

Published: 14 August 2022 Publication History

Abstract

As Internet users attach importance to their own privacy, and a number of laws and regulations go into effect in most countries, Internet products need to provide users with privacy protection. As one of the feasible solutions to provide such privacy protection, federated learning has rapidly gained popularity in both academia and industry in recent years. In this tutorial, we will start off with some real-world tasks to illustrate the topic of federated learning, and cover some basic concepts and important scenarios including cross-device and cross-silo settings. Along with it, we will give several demonstrations with popular federated learning frameworks. We will also show how to do the automatic hyperparameter tuning with federated learning to significantly save their efforts in practice. Then we dive into three parallel hot topics, Personalized Federated Learning, Federated Graph Learning, and Attack in Federated Learning. For each of them, we will motivate it with real-world applications, illustrate the state-of-the-art methods, and discuss their pros and cons using concrete examples. As the last part, we will point out some future research directions.

References

[1]
Vincent D Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre. 2008. Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment (2008), P10008.
[2]
Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub KonečnỴ, Stefano Mazzocchi, Brendan McMahan, et al. 2019. Towards federated learning at scale: System design. Proceedings of Machine Learning and Systems, Vol. 1 (2019), 374--388.
[3]
Daoyuan Chen, Dawei Gao, Weirui Kuang, Yaliang Li, and Bolin Ding. 2022. pFL-Bench: A Comprehensive Benchmark for Personalized Federated Learning. arXiv preprint arxiv.2206.03655 (2022).
[4]
Cynthia Dwork, Aaron Roth, et al. 2014. The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci., Vol. 9, 3--4 (2014), 211--407.
[5]
Matthias Fey and Jan E. Lenssen. 2019. Fast Graph Representation Learning with PyTorch Geometric. In ICLR Workshop.
[6]
Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller. 2020. Inverting gradients-how easy is it to break privacy in federated learning? NeurIPS, Vol. 33 (2020), 16937--16947.
[7]
Stephen Hardy, Wilko Henecka, Hamish Ivey-Law, Richard Nock, Giorgio Patrini, Guillaume Smith, and Brian Thorne. 2017. Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption. arXiv preprint arXiv:1711.10677 (2017).
[8]
Briland Hitaj, Giuseppe Ateniese, and Fernando Perez-Cruz. 2017. Deep models under the GAN: information leakage from collaborative deep learning. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security. 603--618.
[9]
Mikhail Khodak, Renbo Tu, Tian Li, Liam Li, Nina Balcan, Virginia Smith, and Ameet Talwalkar. 2021. Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing. NeurIPS, Vol. 34 (2021).
[10]
Antti Koskela and Antti Honkela. 2020. Learning Rate Adaptation for Differentially Private Learning. In AISTATS. 2465--2475.
[11]
Liam Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. 2018. Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization. Journal of Machine Learning Research, Vol. 18--185 (2018), 1--52.
[12]
Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. 2021 a. Ditto: Fair and robust federated learning through personalization. In ICML. 6357--6368.
[13]
Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, and Qi Dou. 2021 b. FedBN: Federated learning on non-iid features via local batch normalization. arXiv preprint arXiv:2102.07623 (2021).
[14]
Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2019. DARTS: Differentiable Architecture Search. In ICLR.
[15]
Othmane Marfoq, Giovanni Neglia, Aurélien Bellet, Laetitia Kameni, and Richard Vidal. 2021. Federated multi-task learning under a mixture of distributions. NeurIPS, Vol. 34 (2021).
[16]
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics. 1273--1282.
[17]
Pascal Paillier. 1999. Public-key cryptosystems based on composite degree residuosity classes. In International conference on the theory and applications of cryptographic techniques. 223--238.
[18]
Toyotaro Suzumura, Yi Zhou, Natahalie Baracaldo, Guangnan Ye, Keith Houck, Ryo Kawahara, Ali Anwar, Lucia Larise Stavarache, Yuji Watanabe, Pablo Loyola, et almbox. 2019. Towards federated graph learning for collaborative financial crimes detection. arXiv preprint arXiv:1909.12946 (2019).
[19]
Canh T Dinh, Nguyen Tran, and Josh Nguyen. 2020. Personalized federated learning with moreau envelopes. NeurIPS, Vol. 33 (2020), 21394--21405.
[20]
Zhen Wang, Weirui Kuang, Yuexiang Xie, Liuyi Yao, Yaliang Li, Bolin Ding, and Jingren Zhou. 2022 a. FederatedScope-GNN: Towards a Unified, Comprehensive and Efficient Package for Federated Graph Learning. In Proc. of the SIGKDD International Conference on Knowledge Discovery and Data Mining .
[21]
Zhen Wang, Weirui Kuang, Ce Zhang, Bolin Ding, and Yaliang Li. 2022 b. FedHPO-B: A Benchmark Suite for Federated Hyperparameter Optimization. arXiv preprint arxiv.2206.03966 (2022).
[22]
Chuhan Wu, Fangzhao Wu, Yang Cao, Yongfeng Huang, and Xing Xie. 2021. Fedgnn: Federated graph neural network for privacy-preserving recommendation. arXiv preprint arXiv:2102.04925 (2021).
[23]
Han Xie, Jing Ma, Li Xiong, and Carl Yang. 2021. Federated graph classification over non-iid graphs. NeurIPS, Vol. 34 (2021).
[24]
Yuexiang Xie, Zhen Wang, Daoyuan Chen, Dawei Gao, Liuyi Yao, Weirui Kuang, Yaliang Li, Bolin Ding, and Jingren Zhou. 2022. FederatedScope: A Flexible Federated Learning Platform for Heterogeneity. arXiv preprint arxiv.2204.05011 (2022).
[25]
Qiang Yang, Yang Liu, Yong Cheng, Yan Kang, Tianjian Chen, and Han Yu. 2019. Federated learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, Vol. 13, 3 (2019), 1--207.
[26]
Andrew C Yao. 1982. Protocols for secure computations. In annual symposium on foundations of computer science. 160--164.
[27]
Liuyi Yao, Dawei Gao, Zhen Wang, Yuexiang Xie, Weirui Kuang, Daoyuan Chen, Haohui Wang, Chenhe Dong, Bolin Ding, and Yaliang Li. 2022. A Benchmark for Federated Hetero-Task Learning. arXiv preprint arxiv.2206.03436 (2022).
[28]
Huanle Zhang, Mi Zhang, Xin Liu, Prasant Mohapatra, and Michael DeLucia. 2021 b. Automatic Tuning of Federated Learning Hyper-Parameters from System Perspective. arXiv preprint arXiv:2110.03061 (2021).
[29]
Ke Zhang, Carl Yang, Xiaoxiao Li, Lichao Sun, and Siu Ming Yiu. 2021 a. Subgraph federated learning with missing neighbor generation. NeurIPS, Vol. 34 (2021).
[30]
Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. 2020. iDLG: Improved deep leakage from gradients. arXiv preprint arXiv:2001.02610 (2020).
[31]
Yi Zhou, Parikshit Ram, Theodoros Salonidis, Nathalie Baracaldo, Horst Samulowitz, and Heiko Ludwig. 2021. FLoRA: Single-shot Hyper-parameter Optimization for Federated Learning. arXiv preprint arXiv:2112.08524 (2021).
[32]
Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep leakage from gradients. NeurIPS, Vol. 32 (2019).

Cited By

View all
  • (2024)Discrete Federated Multi-behavior Recommendation for Privacy-Preserving Heterogeneous One-Class Collaborative FilteringACM Transactions on Information Systems10.1145/365285342:5(1-50)Online publication date: 29-Apr-2024
  • (2024)Secure and Privacy-Preserving Decentralized Federated Learning for Personalized Recommendations in Consumer Electronics Using Blockchain and Homomorphic EncryptionIEEE Transactions on Consumer Electronics10.1109/TCE.2023.332948070:1(2546-2556)Online publication date: Feb-2024
  • (2022)Applications and Challenges of Federated Learning Paradigm in the Big Data Era with Special Emphasis on COVID-19Big Data and Cognitive Computing10.3390/bdcc60401276:4(127)Online publication date: 26-Oct-2022

Index Terms

  1. A Practical Introduction to Federated Learning

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
    August 2022
    5033 pages
    ISBN:9781450393850
    DOI:10.1145/3534678
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 14 August 2022

    Check for updates

    Qualifiers

    • Abstract

    Conference

    KDD '22
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

    Upcoming Conference

    KDD '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)84
    • Downloads (Last 6 weeks)7
    Reflects downloads up to 13 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Discrete Federated Multi-behavior Recommendation for Privacy-Preserving Heterogeneous One-Class Collaborative FilteringACM Transactions on Information Systems10.1145/365285342:5(1-50)Online publication date: 29-Apr-2024
    • (2024)Secure and Privacy-Preserving Decentralized Federated Learning for Personalized Recommendations in Consumer Electronics Using Blockchain and Homomorphic EncryptionIEEE Transactions on Consumer Electronics10.1109/TCE.2023.332948070:1(2546-2556)Online publication date: Feb-2024
    • (2022)Applications and Challenges of Federated Learning Paradigm in the Big Data Era with Special Emphasis on COVID-19Big Data and Cognitive Computing10.3390/bdcc60401276:4(127)Online publication date: 26-Oct-2022

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media