Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

EneA-FL: : Energy-aware orchestration for serverless federated learning

Published: 25 June 2024 Publication History

Abstract

Federated Learning (FL) represents the de-facto standard paradigm for enabling distributed learning over multiple clients in real-world scenarios. Despite the great strides reached in terms of accuracy and privacy awareness, the real adoption of FL in real-world scenarios, in particular in industrial deployment environments, is still an open thread. This is mainly due to privacy constraints and to the additional complexity stemming from the set of hyperparameters to tune when employing AI techniques on bandwidth-, computing-, and energy-constrained nodes. Motivated by these issues, we focus on scenarios where participating clients are characterised by highly heterogeneous computing capabilities and energy budgets proposing EneA-FL, an innovative scheme for serverless smart energy management. This novel approach dynamically adapts to optimise the training process while fostering seamless interaction between Internet of Things (IoT) devices and edge nodes. In particular, the proposed middleware provides a containerised software module that efficiently manages the interaction of each worker node with the central aggregator. By monitoring local energy budget, computational capabilities, and target accuracy, EneA-FL intelligently takes informed decisions about the inclusion of specific nodes in the subsequent training rounds, effectively balancing the tripartite trade-off between energy consumption, training time, and final accuracy. Finally, in a series of extensive experiments across diverse scenarios, our solution demonstrates impressive results, achieving between 30% and 60% lower energy consumption against popular client selection approaches available in the literature while being up to 3.5 times more efficient than standard FL solutions.

Highlights

EneA-FL targets serverless computing for FL in Fog and constrained environments.
EneA-FL introduces a resource-conscious worker selection policy in FL.
EneA-FL optimises resource consumption while achieving efficient model convergence.
EneA-FL achieves a 3x reduction in energy consumption and 50% faster convergence.
EneA-FL is flexible and can be applied to a diverse array of learning tasks.
EneA-FL uses containers and microservices to shift from the Cloud to the Edge.

References

[1]
Zhang C., Xie Y., Bai H., Yu B., Li W., Gao Y., A survey on federated learning, Knowl. Based Syst. 216 (2021),.
[2]
Lyu L., Yu H., Yang Q., Threats to federated learning: A survey, 2020, CoRR abs/2003.02133. URL: https://arxiv.org/abs/2003.02133.
[3]
Aledhari M., Razzak R., Parizi R.M., Saeed F., Federated learning: A survey on enabling technologies, protocols, and applications, IEEE Access 8 (2020) 140699–140725,.
[4]
Agiollo A., Bardhi E., Conti M., Dal Fabbro N., Lazzeretti R., Anonymous federated learning via named-data networking, Future Gener. Comput. Syst. 152 (2024) 288–303,.
[5]
Agiollo A., Rafanelli A., Magnini M., Ciatto G., Omicini A., Symbolic knowledge injection meets intelligent agents: QoS metrics and experiments, Auton. Agents Multi-Agent Syst. 37 (2023) 27:1–27:30,.
[6]
Agiollo A., Rafanelli A., Omicini A., Towards quality-of-service metrics for symbolic knowledge injection, in: WOA 2022–23rd Workshop from Objects to Agents, in: CEUR Workshop Proceedings, vol. 3261, RWTH Aachen University, Sun SITE Central Europe, 2022, pp. 30–47. URL: http://ceur-ws.org/Vol-3261/paper3.pdf.
[7]
Zhang T., Gao L., He C., Zhang M., Krishnamachari B., Avestimehr A.S., Federated learning for the internet of things: Applications, challenges, and opportunities, IEEE Internet Things Mag. 5 (2022) 24–29. URL: https://doi.org/10.1109/IOTM.004.2100182.
[8]
Agiollo A., Conti M., Kaliyar P., Lin T., Pajola L., DETONAR: Detection of routing attacks in RPL-based IoT, IEEE Trans. Netw. Serv. Manag. 18 (2021) 1178–1190. URL: https://ieeexplore.ieee.org/document/9415869.
[9]
Agiollo A., Omicini A., Load classification: A case study for applying neural networks in hyper-constrained embedded devices, Appl. Sci. 11 (2021) URL: https://www.mdpi.com/2076-3417/11/24/11957, special Issue Artificial Intelligence and Data Engineering in Engineering Applications.
[10]
Sevilla J., Heim L., Ho A., Besiroglu T., Hobbhahn M., Villalobos P., Compute trends across three eras of machine learning, in: International Joint Conference on Neural Networks, IJCNN 2022, Padua, Italy, July 18-23, 2022, IEEE, 2022, pp. 1–8. URL: https://doi.org/10.1109/IJCNN55064.2022.9891914.
[11]
Rudyy O., Garcia-Gasulla M., Mantovani F., Santiago A., Sirvent R., Vázquez M., Containers in HPC: A scalability and portability study in production biological simulations, in: 2019 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2019, Rio de Janeiro, Brazil, May 20-24, 2019, IEEE, 2019, pp. 567–577. URL: https://doi.org/10.1109/IPDPS.2019.00066.
[12]
L.P. Dewi, A. Noertjahyana, H.N. Palit, K. Yedutun, Server scalability using kubernetes, in: 2019 4th Technology Innovation Management and Engineering Science International Conference (TIMES-ICON), 2019, pp. 1–4, https://doi.org/10.1109/TIMES-iCON47539.2019.9024501.
[13]
Zhong Z., Xu M., Rodriguez M.A., Xu C., Buyya R., Machine learning-based orchestration of containers: A taxonomy and future directions, ACM Comput. Surv. 54 (2022) URL: https://doi.org/10.1145/3510415.
[14]
Rovnyagin M.M., Hrapov A.S., Guminskaia A.V., Orlov A.P., Ml-based heterogeneous container orchestration architecture, in: 2020 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus), 2020, pp. 477–481,.
[15]
Zhong Z., Xu M., Rodriguez M.A., Xu C., Buyya R., Machine learning-based orchestration of containers: A taxonomy and future directions, ACM Comput. Surv. 54 (2022) 217:1–217:35. URL: https://doi.org/10.1145/3510415.
[16]
Xu P., Shi S., Chu X., Performance evaluation of deep learning tools in docker containers, in: 2017 3rd International Conference on Big Data Computing and Communications (BIGCOM), 2017, pp. 395–403. URL: https://doi.org/10.1109/BIGCOM.2017.32.
[17]
Grafberger A., Chadha M., Jindal A., Gu J., Gerndt M., FedLess: Secure and scalable federated learning using serverless computing, in: 2021 IEEE International Conference on Big Data (Big Data), 2021, pp. 164–173,.
[18]
Singh P., Masud M., Hossain M.S., Kaur A., Muhammad G., Ghoneim A., Privacy-preserving serverless computing using federated learning for smart grids, IEEE Trans. Ind. Inform. 18 (2022) 7843–7852,.
[19]
Yi S., Li C., Li Q., A survey of fog computing: Concepts, applications and issues, in: Proceedings of the 2015 Workshop on Mobile Big Data, Mobidata ’15, Association for Computing Machinery, 2015, pp. 37–42. URL: https://doi.org/10.1145/2757384.2757397.
[20]
Puliafito C., Mingozzi E., Longo F., Puliafito A., Rana O., Fog computing for the internet of things, ACM Trans. Internet Technol. 19 (2019) 1–41. URL: https://doi.org/10.1145/3301443.
[21]
Ghobaei-Arani M., Souri A., Rahmanian A.A., Resource management approaches in fog computing: a comprehensive review, J. Grid Comput. 18 (2019) 1–42. URL: https://doi.org/10.1007/s10723-019-09491-1.
[22]
Hamdi A.M.A., Hussain F.K., Hussain O.K., Task offloading in vehicular fog computing: State-of-the-art and open issues, Future Gener. Comput. Syst. 133 (2022) 201–212. URL: https://doi.org/10.1016/j.future.2022.03.019.
[23]
Hussein M., Mousa M., Efficient task offloading for IoT-based applications in fog computing using ant colony optimization, IEEE Access 8 (2020) 37191–37201. URL: https://doi.org/10.1109/ACCESS.2020.2975741.
[24]
Zhou H., Wu T., Chen X., He S., Guo D., Wu J., Reverse auction-based computation offloading and resource allocation in mobile cloud-edge computing, IEEE Trans. Mob. Comput. 22 (2023) 6144–6159. URL: https://doi.org/10.1109/TMC.2022.3189050.
[25]
Shan F., Luo J., Jin J., Wu W., Offloading delay constrained transparent computing tasks with energy-efficient transmission power scheduling in wireless iot environment, IEEE Internet Things J. 6 (2019) 4411–4422,.
[26]
Weiner J., Agarwal N., Schatzberg D., Yang L., Wang H., Sanouillet B., Sharma B., Heo T., Jain M., Tang C., Skarlatos D., TMO: transparent memory offloading in datacenters, in: ASPLOS ’22: 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Lausanne, Switzerland, 28 February 2022 - 4 March 2022, ACM, 2022, pp. 609–621. URL: https://doi.org/10.1145/3503222.3507731.
[27]
Zhang W., Zhang Z., Chao H., Cooperative fog computing for dealing with big data in the internet of vehicles: Architecture and hierarchical resource management, IEEE Commun. Mag. 55 (2017) 60–67. URL: https://doi.org/10.1109/MCOM.2017.1700208.
[28]
Gupta H., Vahid Dastjerdi A., Ghosh S.K., Buyya R., IFogSim: A toolkit for modeling and simulation of resource management techniques in the internet of things, edge and fog computing environments, Softw. - Pract. Exp. 47 (2017) 1275–1296. URL: https://doi.org/10.1002/spe.2509.
[29]
Calheiros R.N., Ranjan R., Beloglazov A., Rose C.A.F.D., Buyya R., CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms, Softw. - Pract. Exp. 41 (2010) 23–50. URL: https://doi.org/10.1002/spe.995.
[30]
Puliafito C., Gonçalves D.M., Lopes M.M., Martins L.L., Madeira E., Mingozzi E., Rana O., Bittencourt L.F., MobFogSim: Simulation of mobility and migration for fog computing, Simul. Model. Pract. Theory 101 (2020) URL: https://www.sciencedirect.com/science/article/pii/S1569190X19301935, modeling and Simulation of Fog Computing.
[31]
Zhao X., Zhao L., Liang K., An energy consumption oriented offloading algorithm for fog computing, in: Quality, Reliability, Security and Robustness in Heterogeneous Networks - 12th International Conference, QShine 2016, Seoul, Korea, July 7-8, 2016, Proceedings, in: Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol. 199, Springer, 2016, pp. 293–301. URL: https://doi.org/10.1007/978-3-319-60717-7_29.
[32]
Meng X., Wang W., Zhang Z., Delay-constrained hybrid computation offloading with cloud and fog computing, IEEE Access 5 (2017) 21355–21367,.
[33]
Ahn S., Gorlatova M., Chiang M., Leveraging fog and cloud computing for efficient computational offloading, in: 2017 IEEE MIT Undergraduate Research Technology Conference (URTC), IEEE, 2017,.
[34]
Nguyen D.C., Ding M., Pathirana P.N., Seneviratne A., Li J., Poor H.V., Federated learning for internet of things: A comprehensive survey, IEEE Commun. Surv. Tutor. 23 (2021) 1622–1658. URL: https://doi.org/10.1109/COMST.2021.3075439.
[35]
Imteaj A., Thakker U., Wang S., Li J., Amini M.H., A survey on federated learning for resource-constrained IoT devices, IEEE Internet Things J. 9 (2022) 1–24,.
[36]
Wu S., Li G., Chen F., Shi L., Training and inference with integers in deep neural networks, in: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3 2018, Conference Track Proceedings, OpenReview.net, 2018, URL: https://openreview.net/forum?id#HJGXzmspb.
[37]
Jain P., Jain A., Nrusimha A., Gholami A., Abbeel P., Keutzer K., Stoica I., Gonzalez J., Checkmate: Breaking the memory wall with optimal tensor rematerialization, in: Proceedings of Machine Learning and Systems 2020, MLSys 2020, Austin, TX, USA, March 2-4, 2020, mlsys.org, 2020, URL: https://proceedings.mlsys.org/paper_files/paper/2020/hash/0b816ae8f06f8dd3543dc3d9ef196cab-Abstract.html.
[38]
Chen T., Xu B., Zhang C., Guestrin C., Training deep nets with sublinear memory cost, 2016, CoRR abs/1604.06174. URL: http://arxiv.org/abs/1604.06174.
[39]
Cai H., Gan C., Zhu L., Han S., TinyTL: Reduce memory, not parameters for efficient on-device learning, in: Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, Virtual, 2020, URL: https://proceedings.neurips.cc/paper/2020/hash/81f7acabd411274fcf65ce2070ed568a-Abstract.html.
[40]
Trindade S., Bittencourt L.F., da Fonseca N.L.S., Management of resource at the network edge for federated learning, 2021, CoRR abs/2107.03428. URL: https://arxiv.org/abs/2107.03428.
[41]
Yang Z., Chen M., Saad W., Hong C.S., Shikh-Bahaei M., Energy efficient federated learning over wireless communication networks, IEEE Trans. Wireless Commun. 20 (2021) 1935–1949. URL: https://doi.org/10.1109/TWC.2020.3037554.
[42]
C.W. Zaw, C.S. Hong, A decentralized game theoretic approach for energy-aware resource management in federated learning, in: IEEE International Conference on Big Data and Smart Computing, BigComp 2021, Jeju Island, South Korea, January 17-20, 2021, IEEE, 2021, pp. 133–136, URL:.
[43]
Nash Jr. J.F., Equilibrium points in n-person games, Proc. Natl. Acad. Sci. 36 (1950) 48–49.
[44]
Xu Y., Liao Y., Xu H., Ma Z., Wang L., Liu J., Adaptive control of local updating and model compression for efficient federated learning, IEEE Trans. Mob. Comput. (2022) URL: https://doi.org/10.1109/TMC.2022.3186936.
[45]
Cui Y., Cao K., Cao G., Qiu M., Wei T., Client scheduling and resource management for efficient training in heterogeneous IoT-edge federated learning, IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. 41 (2022) 2407–2420. URL: https://doi.org/10.1109/TCAD.2021.3110743.
[46]
Lai F., Zhu X., Madhyastha H.V., Chowdhury M., Oort: Efficient federated learning via guided participant selection, in: 15th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2021, July 14-16, 2021, USENIX Association, 2021, pp. 19–35. URL: https://www.usenix.org/conference/osdi21/presentation/lai.
[47]
Arouj A., Abdelmoniem A.M., Towards energy-aware federated learning on battery-powered clients, 2022, CoRR abs/2208.04505. URL: https://doi.org/10.48550/arXiv.2208.04505.
[48]
Cho Y.J., Wang J., Joshi G., Client selection in federated learning: Convergence analysis and power-of-choice selection strategies, 2020, CoRR abs/2010.01243. URL: https://arxiv.org/abs/2010.01243.
[49]
Kim Y.G., Wu C., AutoFL: Enabling heterogeneity-aware energy efficient federated learning, in: MICRO ’21: 54th Annual IEEE/ACM International Symposium on Microarchitecture, Virtual Event, Greece, October 18-22, 2021, ACM, 2021, pp. 183–198. URL: https://doi.org/10.1145/3466752.3480129.
[50]
Wang S., Tuor T., Salonidis T., Leung K.K., Makaya C., He T., Chan K., Adaptive federated learning in resource constrained edge computing systems, IEEE J. Sel. Areas Commun. 37 (2019) 1205–1221,.
[51]
He C., Ceyani E., Balasubramanian K., Annavaram M., Avestimehr S., SpreadGNN: Serverless multi-task federated learning for graph neural networks, 2021, CoRR abs/2106.02743. URL: https://arxiv.org/abs/2106.02743.
[52]
Rabenstein B., Volz J., Prometheus: A next-generation monitoring system (talk), in: SRECon15 Europe, USENIX Association, Dublin, 2015, URL: https://www.usenix.org/conference/srecon15europe/program/presentation/rabenstein.
[53]
Lua E.K., Crowcroft J., Pias M., Sharma R., Lim S., A survey and comparison of peer-to-peer overlay network schemes, IEEE Commun. Surv. Tutor. 7 (2005) 72–93.
[54]
Bellavista P., Giannelli C., Mamei M., Mendula M., Picone M., Application-driven network-aware digital twin management in industrial edge environments, IEEE Trans. Ind. Inform. 17 (2021) 7791–7801,.
[55]
Meneghello F., Calore M., Zucchetto D., Polese M., Zanella A., Iot: Internet of threats? A survey of practical security vulnerabilities in real iot devices, IEEE Internet Things J. 6 (2019) 8182–8201. URL: https://doi.org/10.1109/JIOT.2019.2935189.
[56]
Wu H., Wang P., Node selection toward faster convergence for federated learning on non-IID data, IEEE Trans. Netw. Sci. Eng. 9 (2022) 3099–3111. URL: https://doi.org/10.1109/TNSE.2022.3146399.
[57]
Cho Y.J., Wang J., Joshi G., Client selection in federated learning: Convergence analysis and power-of-choice selection strategies, 2020, CoRR abs/2010.01243. URL: https://arxiv.org/abs/2010.01243.
[58]
Wu H., Wang P., Fast-convergent federated learning with adaptive weighting, IEEE Trans. Cognit. Commun. Netw. 7 (2021) 1078–1088,.
[59]
Rjoub G., Wahab O.A., Bentahar J., Bataineh A., Trust-driven reinforcement selection strategy for federated learning on IoT devices, Computing (2022) URL: https://doi.org/10.1007/s00607-022-01078-1.
[60]
Mazzocca C., Romandini N., Mendula M., Montanari R., Bellavista P., TruFLaaS: Trustworthy federated learning as a service, IEEE Internet Things J. (2023) 1,.
[61]
Shen X., Li Z., Chen X., Node selection strategy design based on reputation mechanism for hierarchical federated learning, in: 18th International Conference on Mobility, Sensing and Networking, MSN 2022, Guangzhou, China, December (2022) 14-16, IEEE, 2022, pp. 718–722. URL: https://doi.org/10.1109/MSN57253.2022.00117.
[62]
Aloqaily M., Ridhawi I.A., Guizani M., Energy-aware blockchain and federated learning-supported vehicular networks, IEEE Trans. Intell. Transp. Syst. 23 (2022) 22641–22652. URL: https://doi.org/10.1109/TITS.2021.3103645.
[63]
Mendula M., Bellavista P., Energy-aware edge federated learning for enhanced reliability and sustainability, in: 7th IEEE/ACM Symposium on Edge Computing, SEC 2022, Seattle, WA, USA, December 5-8, 2022, IEEE, 2022, pp. 349–354,.
[64]
McMahan B., Moore E., Ramage D., Hampson S., y Arcas B.A., Communication-efficient learning of deep networks from decentralized data, in: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017 20-22 April 2017, Fort Lauderdale, FL, USA, in: Proceedings of Machine Learning Research, vol. 54, PMLR, 2017, pp. 1273–1282. URL: http://proceedings.mlr.press/v54/mcmahan17a.html.
[65]
Pillutla K., Kakade S.M., Harchaoui Z., Robust aggregation for federated learning, IEEE Trans. Signal Process. 70 (2022) 1142–1154. URL: https://doi.org/10.1109/TSP.2022.3153135.
[66]
Ma X., Zhang J., Guo S., Xu W., Layer-wised model aggregation for personalized federated learning, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, la, USA, June 18-24, 2022, IEEE, 2022, pp. 10082–10091. URL: https://doi.org/10.1109/CVPR52688.2022.00985.
[67]
Caldas S., Wu P., Li T., Konečný J., McMahan H.B., Smith V., Talwalkar A., LEAF: A benchmark for federated settings, 2018, CoRR abs/1812.01097. URL: http://arxiv.org/abs/1812.01097.
[68]
Meidan Y., Bohadana M., Mathov Y., Mirsky Y., Shabtai A., Breitenbacher D., Elovici Y., N-baIoT - network-based detection of IoT botnet attacks using deep autoencoders, IEEE Pervasive Comput. 17 (2018) 12–22,.
[69]
Tallarida R.J., Murray R.B., Area under a curve: Trapezoidal and simpson’s rules, in: Manual of Pharmacologic Calculations, Springer, 1987, pp. 77–81,.
[70]
Paszke A., Gross S., Massa F., Lerer A., Bradbury J., Chanan G., Killeen T., Lin Z., Gimelshein N., Antiga L., Desmaison A., Köpf A., Yang E.Z., DeVito Z., Raison M., Tejani A., Chilamkurthy S., Steiner B., Fang L., Bai J., Chintala S., Pytorch: An imperative style, high-performance deep learning library, in: Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, 2019, pp. 8024–8035. URL: https://proceedings.neurips.cc/paper/2019/hash/bdbca288fee7f92f2bfa9f7012727740-Abstract.html.
[71]
Agiollo A., Omicini A., GNN2GNN: Graph neural networks to generate neural networks, in: Uncertainty in Artificial Intelligence, in: Proceedings of Machine Learning Research, vol. 180, ML Research Press, 2022, pp. 32–42. URL: https://proceedings.mlr.press/v180/agiollo22a.html, proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, UAI 2022 1-5 August 2022, Eindhoven, The Netherlands.
[72]
Agiollo A., Bardhi E., Conti M., Lazzeretti R., Losiouk E., Omicini A., GNN4ifa: Interest flooding attack detection with graph neural networks, in: 2023 IEEE 8th European Symposium on Security and Privacy (EuroS & P), IEEE Computer Society, 2023, pp. 615–630. URL: https://www.computer.org/csdl/proceedings-article/eurosp/2023/651200a615.
[73]
Huong T.T., Ta P.B., Kieu H., Hoang N.V., Hoang N.X., Hung N.T., Tran K.P., Federated learning-based explainable anomaly detection for industrial control systems, IEEE Access 10 (2022) 53854–53872,.
[74]
Agiollo A., Siebert L.C., Murukannaiah P.K., Omicini A., The quarrel of local post-hoc explainers for moral values classification in natural language processing, in: Calvaresi D., Najjar A., Omicini A., Aydoǧan R., Carli R., Ciatto G., Mualla Y., Främling K. (Eds.), Explainable and Transparent AI and Multi-Agent Systems, in: Lecture Notes in Computer Science, vol. 14127, Springer, 2023, pp. 97–115,.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Future Generation Computer Systems
Future Generation Computer Systems  Volume 154, Issue C
May 2024
491 pages

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 25 June 2024

Author Tags

  1. Serverless
  2. Federated learning
  3. Energy management
  4. Internet of things
  5. Resource-constrained learning

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 0
    Total Downloads
  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 12 Nov 2024

Other Metrics

Citations

View Options

View options

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media