FedSLS: Exploring Federated Aggregation in Saliency Latent Space
Pages 7182 - 7190
Abstract
Federated Learning (FL) is an emerging direction in distributed machine learning that enables jointly training a global model without sharing data with server. However, data heterogeneity biases the parameter aggregation at the server, leading to slower convergence and poorer accuracy of the global model. To cope with this, most of the existing works involve enforcing regularization in local optimization or improving the model aggregation scheme at the server. Though effective, they lack a deep understanding of cross-client features. In this paper, we propose a saliency latent space feature aggregation method (FedSLS) across federated clients. By Guided BackPropagation (GBP), we transform deep models into powerful and flexible visual fidelity encoders, applicable to general state inputs across different image domains, and achieve powerful aggregation in the form of saliency latent features. Notably, since GBP is label-insensitive, it is sufficient to capture saliency features only once on each client. Experimental results demonstrate that FedSLS leads to significant improvements over the state-of-the-arts in terms of accuracies, especially in highly heterogeneous settings. For example, on CIFAR-10 dataset, FedSLS achieves 63.43% accuracy within the strongly heterogeneous environment α=0.05, which is 6% to 23% higher than other baselines.
References
[1]
Durmus Alp Emre Acar, Yue Zhao, Ramon Matas, Matthew Mattina, Paul Whatmough, and Venkatesh Saligrama. 2021. Federated Learning Based on Dynamic Regularization. In International Conference on Learning Representations.
[2]
Durmus Alp Emre Acar, Yue Zhao, Ramon Matas Navarro, Matthew Mattina, Paul N. Whatmough, and Venkatesh Saligrama. 2021. Federated Learning Based on Dynamic Regularization. arXiv preprint arXiv:2111.04263 (2021).
[3]
Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. 2018. Sanity Checks for Saliency Maps. Neural Information Processing Systems 31 (2018).
[4]
Dengsheng Chen, Jie Hu, Vince Junkai Tan, Xiaoming Wei, and Enhua Wu. 2023. Elastic Aggregation for Federated Optimization. In Computer Vision and Pattern Recognition. 12187--12197.
[5]
Luke N Darlow, Elliot J Crowley, Antreas Antoniou, and Amos J Storkey. 2018. CINIC-10 is Not ImageNet Or CIFAR-10. arXiv preprint arXiv:1810.03505 (2018).
[6]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A Large-Scale Hierarchical Image Database. In Computer Vision and Pattern Recognition. 248--255.
[7]
David Froelicher, Juan Ramón Troncoso-Pastoriza, Jean Louis Raisaro, Michel Alain Cuendet, João Sá Sousa, Hyunghoon Cho, Bonnie Berger, Jacques Fellay, and Jean-Pierre Hubaux. [n. d.]. Truly Privacy-Preserving Federated Analytics for Precision Medicine with Multiparty Homomorphic Encryption. Nature Communications 12, 1 ([n. d.]), 5910.
[8]
Jack Goetz and Ambuj Tewari. 2020. Federated Learning via Synthetic Data. arXiv preprint arXiv:2008.04489 (2020).
[9]
Weituo Hao, Mostafa El-Khamy, Jungwon Lee, Jianyi Zhang, Kevin J Liang, Changyou Chen, and Lawrence Carin. 2021. Towards Fair Federated Learning with Zero-Shot Data Augmentation. Computer Vision and Pattern Recognition Workshops (2021), 3310--3319.
[10]
Chaoyang He, Alay Dilipbhai Shah, Zhenheng Tang, Di Fan1Adarshan Naiynar Sivashunmugam, Keerti Bhogaraju, Mita Shimpi, Li Shen, Xiaowen Chu, Mahdi Soltanolkotabi, and Salman Avestimehr. 2021. FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks. arXiv preprint arXiv:2111.11066 (2021).
[11]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In Computer Vision and Pattern Recognition. 770--778.
[12]
Kevin Hsieh, Amar Phanishayee, Onur Mutlu, and Phillip Gibbons. 2020. The Non-IID Data Quagmire of Decentralized Machine Learning. In International Conference on Machine Learning. PMLR, 4387--4398.
[13]
Tzu-Ming Harry Hsu, Qi, and Matthew Brown. 2019. Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification. arXiv preprint arXiv:1909.06335 (2019).
[14]
Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. 2020. Federated Visual Classification with Real-World Data Distribution. In European Conference on Computer Vision. Springer, 76--92.
[15]
Eunjeong Jeong, Seungeun Oh, Hyesung Kim, Jihong Park, Mehdi Bennis, and Seong-Lyun Kim. 2018. Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data. arXiv preprint arXiv:1811.11479 (2018).
[16]
Cheng Jin, Xuandong Chen, Yi Gu, and Qun Li. 2023. FedDyn: A Dynamic and Efficient Federated Distillation Approach on Recommender System. In IEEE International Conference on Parallel and Distributed Systems. IEEE, 786--793.
[17]
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. 2019. Advances and Open Problems in Federated Learning. arXiv preprint arXiv:1912.04977 (2019).
[18]
Georgios Kaissis, Alexander Ziller, Jonathan Passerat-Palmbach, Théo Ryffel, Dmitrii Usynin, Andrew Trask, Ionésio Lima, Jason V. Mancuso, Friederike Jungmann, Marc-Matthias Steinborn, Andreas Saleh, Marcus R. Makowski, Daniel Rueckert, and Rickmer F. Braren. 2021. End-to-End Privacy Preserving Deep Learning on Multi-institutional Medical Imaging. Nature Machine Intelligence 3, 6 (2021), 473--484.
[19]
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, and Ananda Theertha Suresh. 2019. SCAFFOLD: Stochastic Controlled Averaging for Federated Learning. In International Conference on Machine Learning.
[20]
Diederik P Kingma and Max Welling. 2013. Auto-Encoding Variational Bayes. arXiv preprint arXiv:1312.6114 (2013).
[21]
Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. (2009).
[22]
Daixun Li, Weiying Xie, Jiaqing Zhang, and Yunsong Li. 2024. MDFL: Multi- Domain Diffusion-Driven Feature Learning. In AAAI Conference on Artificial Intelligence.
[23]
Qinbin Li, Bingsheng He, and Dawn Song. 2021. Model-Contrastive Federated Learning. In Computer Vision and Pattern Recognition. 10713--10722.
[24]
Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. 2020. Federated Learning: Challenges, Methods, and Future Directions. IEEE Signal Processing Magazine 37, 3 (2020), 50--60.
[25]
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2020. Federated Optimization in Heterogeneous Networks. In Proceedings of Machine Learning and Systems. 429--450.
[26]
Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. 2019. On the Convergence of Fedavg on Non-IID Data. arXiv preprint arXiv:1907.02189 (2019).
[27]
Tao Lin, Lingjing Kong, Sebastian U. Stich, and Martin Jaggi. 2020. Ensemble Distillation for Robust Model Fusion in Federated Learning. arXiv preprint arXiv:2006.07242 (2020).
[28]
Kangyang Luo, Xiang Li, Yunshi Lan, and Ming Gao. 2023. GradMA: A Gradient- Memory-based Accelerated Federated Learning with Alleviated Catastrophic Forgetting. In Computer Vision and Pattern Recognition. 3708--3717.
[29]
Mi Luo, Fei Chen, Dapeng Hu, Yifan Zhang, Jian Liang, and Jiashi Feng. 2021. No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data. Neural Information Processing Systems 34 (2021), 5972--5984.
[30]
Yuting Ma, Yuanzhi Yao, and Xiaohua Xu. 2023. PPIDSG: A Privacy-Preserving Image Distribution Sharing Scheme with GAN in Federated Learning. In AAAI Conference on Artificial Intelligence.
[31]
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-Efficient Learning of Deep Networks from Decentralized Data. In Artificial Intelligence and Statistics. PMLR, 1273--1282.
[32]
Umberto Michelucci. 2022. An Introduction to Autoencoders. arXiv preprint arXiv:2201.03898 (2022).
[33]
Xutong Mu, Yulong Shen, Ke Cheng, Xueli Geng, Jiaxuan Fu, Tao Zhang, and Zhiwei Zhang. 2023. FedProc: Prototypical Contrastive Federated Learning on Non-IID Data. Future Generation Computer Systems 143 (2023), 93--104.
[34]
Nguyen Ngoc-Hieu, Nguyen Hung-Quang, The-Anh Ta, Thanh Nguyen-Tang, Khoa D Doan, and Hoang Thanh-Tung. 2023. A Cosine Similarity-based Method for Out-of-Distribution Detection. arXiv preprint arXiv:2306.14920 (2023).
[35]
Amirhossein Reisizadeh, Isidoros Tziotis, Hamed Hassani, Aryan Mokhtari, and Ramtin Pedarsani. 2022. Straggler-Resilient Federated Learning: Leveraging the Interplay between Statistical Accuracy and System Heterogeneity. IEEE Journal on Selected Areas in Information Theory 3, 2 (2022), 197--205.
[36]
Xinyi Shang, Yang Lu, Gang Huang, and Hanzi Wang. 2023. Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Features. arXiv preprint arXiv:2204.13399 (2023).
[37]
Yujun Shi, Jian Liang, Wenqing Zhang, Vincent YF Tan, and Song Bai. 2022. Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning. arXiv preprint arXiv:2210.00226 (2022).
[38]
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. 2014. Striving for Simplicity: The All Convolutional Net. arXiv preprint arXiv:1412.6806 (2014).
[39]
Tao Sun, Dongsheng Li, and BaoWang. 2022. Decentralized Federated Averaging. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 4 (2022), 4289-- 4301.
[40]
Dmitrii Usynin, Alexander Ziller, Marcus R. Makowski, Rickmer F. Braren, Daniel Rueckert, Ben Glocker, Georgios Kaissis, and Jonathan Passerat-Palmbach. 2021. Adversarial Tnterference and its Mitigations in Privacy-preserving Collaborative Machine Learning. Nature Machine Intelligence 3, 9 (2021), 749--758.
[41]
Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, and Yasaman Khazaeni. 2020. Federated Learning with Matched Averaging. arXiv preprint arXiv:2002.06440 (2020).
[42]
Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H Vincent Poor. 2020. Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization. Neural Information Processing Systems 33 (2020), 7611--7623.
[43]
Shiqiang Wang and Mingyue Ji. 2023. A Lightweight Method for Tackling Unknown Participation Probabilities in Federated Averaging. arXiv preprint arXiv:2306.03401 (2023).
[44]
Gal Yona and Daniel Greenfeld. 2021. Revisiting Sanity Checks for Saliency Maps. arXiv preprint arXiv:2110.14297 (2021).
[45]
Mikhail Yurochkin, Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Nghia Hoang, and Yasaman Khazaeni. 2019. Bayesian Nonparametric Federated Learning of Neural Networks. In International Conference on Machine Learning. PMLR, 7252--7261.
[46]
Lin Zhang, Li Shen, Liang Ding, Dacheng Tao, and Ling-Yu Duan. 2022. Fine- Tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning. In Computer Vision and Pattern Recognition. 10174--10183.
[47]
Xinyi Zhang, Qichen Wang, Cheng Xu, Yun Peng, and Jianliang Xu. 2024. Fed- KNN: Secure Federated k-Nearest Neighbor Search. ACM on Management of Data 2, 1 (2024), 1--26.
[48]
Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. 2018. Federated Learning with Non-IID Data. arXiv preprint arXiv:1806.00582 (2018).
Index Terms
- FedSLS: Exploring Federated Aggregation in Saliency Latent Space
Recommendations
HFedCWA: heterogeneous federated learning algorithm based on contribution-weighted aggregation: HFedCWA: heterogeneous federated learning algorithm...
AbstractThe aim of heterogeneous federated learning (HFL) is to address the issues of data heterogeneity, computational resource disparity, and model generalizability and security in federated learning (FL). To facilitate the collaborative training of ...
Discovering Latent Topics With Saliency-Weighted LDA for Image Scene Understanding
In the latent Dirichlet allocation (LDA) model, each image is represented by word distributions with their latent topics. Since the previous LDA-based models are not capable of dealing with the spatial information of visual words in images, this paper ...
Fast-convergent federated learning with class-weighted aggregation
AbstractRecently, federated learning has attracted great attention due to its advantage of enabling model training in a distributed manner. Instead of uploading data for centralized training, it allows devices to keep local data private and ...
Comments
Information & Contributors
Information
Published In
October 2024
11719 pages
ISBN:9798400706868
DOI:10.1145/3664647
- General Chairs:
- Jianfei Cai,
- Mohan Kankanhalli,
- Balakrishnan Prabhakaran,
- Susanne Boll,
- Program Chairs:
- Ramanathan Subramanian,
- Liang Zheng,
- Vivek K. Singh,
- Pablo Cesar,
- Lexing Xie,
- Dong Xu
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Conference
MM '24
Sponsor:
MM '24: The 32nd ACM International Conference on Multimedia
October 28 - November 1, 2024
Melbourne VIC, Australia
Acceptance Rates
MM '24 Paper Acceptance Rate 1,150 of 4,385 submissions, 26%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 66Total Downloads
- Downloads (Last 12 months)66
- Downloads (Last 6 weeks)11
Reflects downloads up to 11 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in