Cited By
View all- Wazid MNautiyal SDas AShetty SIslam S(2025)A Secure Authenticated Healthcare Data Analysis Mechanism in IoMT‐Enabled HealthcareSecurity and Privacy10.1002/spy2.4688:1Online publication date: 12-Jan-2025
Secure Aggregation in Heterogeneous Federated Learning for Digital Ecosystems
Authors: 
Jiao Zhang, Xiong Li, Ke Gu, Wei Liang, Kuanching LiAuthors Info & Claims
Jiao Zhang, Xiong Li, Ke Gu, Wei Liang, Kuanching LiAuthors Info & ClaimsPages 1995 - 2003
Abstract
Privacy-preserving federated learning (PPFL) is vital for Industry 5.0 digital ecosystems due to the increasing number of interconnected devices and the volume of shared sensitive data. Secure aggregation (SA) protocols are essential components to fulfill the privacy properties of PPFL. However, there are still fundamental challenges to be tackled. For example, statistical and model heterogeneous characteristics across terminal devices, communication bottlenecks, and the requirement of inputting integers rather than real values in cryptographic operations. To overcome these problems, we propose RF-HFL, a novel secure aggregation scheme for PPFL in digital ecosystems with the capability to act on non-independent and identically distributed (non-IID) data. The scheme distills Representative Factors from each edge device through a self-attention-based neural network and transfers the average of these factors to the server for aggregation. The central model is then sent back to the devices for regularizing decentralized training models. The data transmitted between the devices and server are greatly reduced, henceforth significantly decreasing the communication overhead in PPFL. Theoretical analyses are provided for correctness, security and convergence. We set a benchmark for comparing our proposed scheme with several state-of-the-art SA protocols or algorithms in heterogeneous PPFL. Results demonstrate the effectiveness and efficiency of RF-HFL on multiple datasets.
References
[1]
H. B. McMahan, E. Moore, D. Ramage, and B. A. Y. Arcas, “Federated learning of deep networks using model averaging,” 2016, arXiv:1602.05629.
[2]
D. C. Nguyen, M. Ding, P. N. Pathirana, A. Seneviratne, J. Li, and H. V. Poor, “Federated learning for Internet of Things: A comprehensive survey,” IEEE Commun. Surveys Tuts., vol. 23, no. 3, pp. 1622–1658, 3rd Quart., 2021.
[3]
S. Yu and L. Cui, Security and Privacy in Federated Learning. Singapore: Springer, 2023.
[4]
M. Songet al., “Analyzing user-level privacy attack against federated learning,” IEEE J. Sel. Areas Commun., vol. 38, no. 10, pp. 2430–2444, Oct. 2020.
[5]
X. Yin, Y. Zhu, and J. Hu, “A comprehensive survey of privacypreserving federated learning: A taxonomy, review, and future directions,” ACM Comput. Surveys, vol. 54, no. 6, pp. 1–36, 2021.
[6]
H. Zheng, H. Hu, and Z. Han, “Preserving user privacy for machine learning: Local differential privacy or federated machine learning?” IEEE Intell. Syst., vol. 35, no. 4, pp. 5–14, Jul./Aug. 2020.
[7]
Y. Li, Y. Zhou, A. Jolfaei, D. Yu, G. Xu, and X. Zheng, “Privacypreserving federated learning framework based on chained secure multiparty computing,” IEEE Internet Things J., vol. 8, no. 8, pp. 6178–6186, Apr. 2021.
[8]
J. Ma, S.-A. Naas, S. Sigg, and X. Lyu, “Privacy-preserving federated learning based on multi-key homomorphic encryption,” Int. J. Intell. Syst., vol. 37, no. 9, pp. 5880–5901, 2022.
[9]
G. Xuet al., “Hercules: Boosting the performance of privacy-preserving federated learning,” IEEE Trans. Depend. Secure Comput., vol. 20, no. 5, pp. 4418–4433, Sep./Oct. 2023.
[10]
J. So, B. Guler, and A. S. Avestimehr, “Turbo-aggregate: Breaking the quadratic aggregation barrier in secure federated learning,” IEEE J. Sel. Areas Inf. Theory, vol. 2, no. 1, pp. 479–489, Mar. 2021.
[11]
B. Choi, J.-Y. Sohn, D.-J. Han, and J. Moon, “Communicationcomputation efficient secure aggregation for federated learning,” 2021, arXiv:12012.05433.
[12]
K. Bonawitzet al., “Practical secure aggregation for privacy-preserving machine learning,” in Proc. ACM SIGSAC Conf. Comput. Commun. Security, 2017, pp. 1175–1191.
[13]
X. Guoet al., “VeriFL: Communication-efficient and fast verifiable aggregation for federated learning,” IEEE Trans. Inf. Forensics Security, vol. 16, pp. 1736–1751, 2021.
[14]
A. R. Elkordy and A. S. Avestimehr, “HeteroSAg: Secure aggregation with heterogeneous quantization in federated learning,” IEEE Trans. Commun., vol. 70, no. 4, pp. 2372–2386, Apr. 2022.
[15]
Y. Zheng, S. Lai, Y. Liu, X. Yuan, X. Yi, and C. Wang, “Aggregation service for federated learning: An efficient, secure, and more resilient realization,” IEEE Trans. Depend. Secure Comput., vol. 20, no. 2, pp. 988–1001, Mar./Apr. 2023.
[16]
A. Shamir, “How to share a secret,” Commun. ACM, vol. 22, no. 11, pp. 612–613, 1979.
[17]
X. Ma, J. Zhu, Z. Lin, S. Chen, and Y. Qin, “A state-of-the-art survey on solving non-iid data in federated learning,” Future Gener. Comput. Syst., vol. 135, pp. 244–258, Oct. 2022.
[18]
L. Collins, H. Hassani, A. Mokhtari, and S. Shakkottai, “Exploiting shared representations for personalized federated learning,” in Proc. Int. Conf. Mach. Learn., 2021, pp. 2089–2099.
[19]
Q. Ma, Y. Xu, H. Xu, Z. Jiang, L. Huang, and H. Huang, “FedSA: A semi-asynchronous federated learning mechanism in heterogeneous edge computing,” IEEE J. Sel. Areas Commun., vol. 39, no. 12, pp. 3654–3672, Dec. 2021.
[20]
X. Zhang, M. Hong, S. Dhople, W. Yin, and Y. Liu, “FedPD: A federated learning framework with adaptivity to non-IID data,” IEEE Trans. Signal Process., vol. 69, pp. 6055–6070, Oct. 2021.
[21]
J. Wuet al., “Hierarchical personalized federated learning for user modeling,” in Proc. Web Conf., 2021, pp. 957–968.
[22]
A. Elgabli, J. Park, A. S. Bedi, C. B. Issaid, M. Bennis, and V. Aggarwal, “Q-gadmm: Quantized group ADMM for communication efficient decentralized machine learning,” IEEE Trans. Commun., vol. 69, no. 1, pp. 164–181, Jan. 2021.
[23]
A. Vaswaniet al., “Attention is all you need,” in Proc. Adv. Neural Inf. Process. Syst., vol. 30, 2017, pp. 5998–6008.
[24]
A. Dosovitskiyet al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in Proc. Int. Conf. Learn. Represent., 2020, pp. 1–21.
[25]
W. Diffie and M. Hellman, “New directions in cryptography,” IEEE Trans. Inf. Theory, vol. 22, no. 6, pp. 644–654, Nov. 1976.
[26]
P. Rogaway, “Authenticated-encryption with associated-data,” in Proc. 9th ACM Conf. Comput. Commun. Security, 2002, pp. 98–107.
[27]
J. Håstad, R. Impagliazzo, L. A. Levin, and M. Luby, “A pseudorandom generator from any one-way function,” SIAM J. Comput., vol. 28, no. 4, pp. 1364–1396, 1999.
[28]
A. T. Suresh, X. Y. Felix, S. Kumar, and H. B. McMahan, “Distributed mean estimation with limited communication,” in Proc. Int. Conf. Mach. Learn., 2017, pp. 3329–3337.
[29]
J. Wang, Q. Liu, H. Liang, G. Joshi, and H. V. Poor, “Tackling the objective inconsistency problem in heterogeneous federated optimization,” in Proc. Adv. Neural Inf. Process. Syst., 2020, pp. 7611–7623.
[30]
A. Paszkeet al., “Pytorch: An imperative style, high-performance deep learning library,” in Proc. Adv. Neural Inf. Process. Syst., 2019, pp. 8024–8035.
[31]
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proc. IEEE, vol. 86, no. 11, pp. 2278–2324, Nov. 1998.
[32]
S. Caldaset al., “LEAF: A benchmark for federated settings,” 2018, arXiv:1812.01097.
[33]
A. Krizhevsky and G. Hinton, Learning Multiple Layers of Features From Tiny Images, Univ. Toronto, Toronto, ON, Canada, 2009.
Recommendations
A Flexible and Scalable Malicious Secure Aggregation Protocol for Federated Learning
Secure aggregation becomes a major solution to providing privacy for federated learning. Secure aggregation for mobile devices typically relies on Shamir secret sharing (SSS) to achieve dropout robustness, but limits the system’s corruption and ...
Verifiable Secure Aggregation Protocol Under Federated Learning
Artificial Intelligence Security and PrivacyAbstractFederated learning is a new machine learning paradigm used for collaborative training models among multiple devices. In federated learning, multiple clients participate in model training locally and use decentralized learning methods to ensure the ...
DealSecAgg: Efficient Dealer-Assisted Secure Aggregation for Federated Learning
ARES '24: Proceedings of the 19th International Conference on Availability, Reliability and SecurityFederated learning eliminates the necessity of transferring private training data and instead relies on the aggregation of model updates. Several publications on privacy attacks show how these individual model updates are vulnerable to the extraction of ...
Comments
Information & Contributors
Information
Published In
0098-3063 © 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Press
Publication History
Published: 01 February 2024
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 09 Feb 2025
Other Metrics
Citations
Cited By
View all- Wazid MNautiyal SDas AShetty SIslam S(2025)A Secure Authenticated Healthcare Data Analysis Mechanism in IoMT‐Enabled HealthcareSecurity and Privacy10.1002/spy2.4688:1Online publication date: 12-Jan-2025