research-article

Open access

Experimenting with Emerging RISC-V Systems for Decentralised Machine Learning

Authors:

Gianluca Mittone,

Iacopo Colonnelli,

Doriana Medić,

Andrea Bartolini,

Roberto Esposito,

Emanuele Parisi,

Francesco Beneventi,

Massimo Torquati,

Marco AldinucciAuthors Info & Claims

CF '23: Proceedings of the 20th ACM International Conference on Computing Frontiers

Pages 73 - 83

https://doi.org/10.1145/3587135.3592211

Published: 04 August 2023 Publication History

Abstract

Decentralised Machine Learning (DML) enables collaborative machine learning without centralised input data. Federated Learning (FL) and Edge Inference are examples of DML. While tools for DML (especially FL) are starting to flourish, many are not flexible and portable enough to experiment with novel processors (e.g., RISC-V), non-fully connected network topologies, and asynchronous collaboration schemes. We overcome these limitations via a domain-specific language allowing us to map DML schemes to an underlying middleware, i.e. the FastFlow parallel programming library. We experiment with it by generating different working DML schemes on x86-64 and ARM platforms and an emerging RISC-V one. We characterise the performance and energy efficiency of the presented schemes and systems. As a byproduct, we introduce a RISC-V porting of the PyTorch framework, the first publicly available to our knowledge.

References

[1]

Robert Adolf, Saketh Rama, Brandon Reagen, Gu-Yeon Wei, and David Brooks. 2016. Fathom: Reference workloads for modern deep learning methods. In 2016 IEEE International Symposium on Workload Characterization (IISWC). IEEE, Providence, RI, USA, 1--10. https://doi.org/10.1109/IISWC.2016.7581275

[2]

Marco Aldinucci, Sonia Campa, Marco Danelutto, Peter Kilpatrick, and Massimo Torquati. 2013. Design patterns percolating to parallel programming framework implementation. International Journal of Parallel Programming 42, 6 (26 September 2013), 1012--1031. https://doi.org/10.1007/s10766-013-0273-6

Digital Library

[3]

Marco Aldinucci, Marco Danelutto, Peter Kilpatrick, Massimiliano Meneghin, and Massimo Torquati. 2012. An Efficient Unbounded Lock-Free Queue for Multi-core Systems. In Euro-Par 2012 Parallel Processing, Vol. 7484. Springer Berlin Heidelberg, Berlin, Heidelberg, 662--673. https://doi.org/10.1007/978-3-642-32820-6_65

Digital Library

[4]

Marco Aldinucci, Marco Danelutto, Peter Kilpatrick, and Massimo Torquati. 2017. Fastflow: High-Level and Efficient Streaming on Multicore. Wiley-Blackwell, 261--280.

[5]

The TensorFlow Federated Authors. 2019. TensorFlow Federated. Google. https://github.com/tensorflow/federated

[6]

Andrea Bartolini, Federico Ficarelli, Emanuele Parisi, Francesco Beneventi, Francesco Barchi, Daniele Gregori, Fabrizio Magugliani, Marco Cicala, Cosimo Gianfreda, Daniele Cesarini, et al. 2022. Monte Cimone: Paving the Road for the First Generation of RISC-V High-Performance Computers. In 2022 IEEE 35th International System-on-Chip Conference (SOCC). IEEE, Belfast, United Kingdom, 1--6. https://doi.org/10.1109/SOCC56010.2022.9908096

[7]

Enrique Tomás Martínez Beltrán, Mario Quiles Pérez, Pedro Miguel Sánchez Sánchez, Sergio López Bernal, Gérôme Bovet, Manuel Gil Pérez, Gregorio Martínez Pérez, and Alberto Huertas Celdrán. 2022. Decentralized Federated Learning: Fundamentals, State-of-the-art, Frameworks, Trends, and Challenges. arXiv preprint arXiv:2211.08413 (15 November 2022). https://doi.org/10.48550/ARXIV.2211.08413

[8]

Daniel J Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Titouan Parcollet, Pedro PB de Gusmão, and Nicholas D Lane. 2020. Flower: A friendly federated learning research framework. arXiv preprint arXiv:2007.14390 (28 July 2020). https://doi.org/10.48550/ARXIV.2007.14390

[9]

Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konečny, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. 2018. Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097 (3 December 2018). https://doi.org/10.48550/ARXIV.1812.01097

[10]

Jihong Chen and Jiabin Sun. 2021. Understanding the Chinese Data Security Law. International Cybersecurity Law Review 2, 2 (12 October 2021), 209--221. https://doi.org/10.1365/s43439-021-00038-3

[11]

Kewei Cheng, Tao Fan, Yilun Jin, Yang Liu, Tianjian Chen, Dimitrios Papadopoulos, and Qiang Yang. 2021. Secureboost: A lossless federated learning framework. IEEE Intelligent Systems 36, 6 (25 May 2021), 87--98. https://doi.org/10.1109/MIS.2021.3082561

Digital Library

[12]

Bryce Goodman and Seth Flaxman. 2017. European Union Regulations on Algorithmic Decision-Making and a "Right to Explanation". AI Magazine 38, 3 (2 Oct 2017), 50--57. https://doi.org/10.1609/aimag.v38i3.2741

Digital Library

[13]

Shane W. Grant and Randolph Voorhies. 2013. Cereal a C++11 library for serialization. USCiLab. https://github.com/USCiLab/cereal

[14]

Chaoyang He, Songze Li, Jinhyun So, Xiao Zeng, Mi Zhang, Hongyi Wang, Xiaoyang Wang, Praneeth Vepakomma, Abhishek Singh, Hang Qiu, et al. 2020. Fedml: A research library and benchmark for federated machine learning. arXiv preprint arXiv:2007.13518 (27 July 2020). https://doi.org/10.48550/ARXIV.2007.13 518

[15]

Li Li, Yuxi Fan, Mike Tse, and Kuo-Yi Lin. 2020. A review of applications in federated learning. Computers & Industrial Engineering 149 (November 2020), 106854. https://doi.org/10.1016/j.cie.2020.106854

[16]

Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, and Soumith Chintala. 2020. Pytorch distributed: Experiences on accelerating data parallel training. arXiv preprint arXiv:2006.15704 (28 June 2020). https://doi.org/10.48550/ARXIV.2006.15704

[17]

Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. 2017. Communication-Efficient Learning of Deep Networks from Decentralized Data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics AISTATS (Proceedings of Machine Learning Research, Vol. 54), Aarti Singh and Xiaojin (Jerry) Zhu (Eds.). PMLR, Fort Lauderdale, FL, USA, 1273--1282.

[18]

Stuart L Pardau. 2018. The California consumer privacy act: towards a European-style privacy regime in the United States. J. Tech. L. & Pol'y 23, 1 (2018), 68--114.

[19]

AdamPaszke, SamGross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d'Alché-Buc, Emily B. Fox, and Roman Garnett (Eds.), Vol. 32. Vancouver Convention Center, Vancouver CANADA, 8024--8035.

[20]

G. Anthony Reina, Alexey Gruzdev, Patrick Foley, Olga Perepelkina, Mansi Sharma, Igor Davidyuk, Ilya Trushkin, Maksim Radionov, Aleksandr Mokrov, Dmitry Agapov, Jason Martin, Brandon Edwards, Micah J. Sheller, Sarthak Pati, Prakash Narayana Moorthy, Shih-han Wang, Prashant Shah, and Spyridon Bakas. 2021. OpenFL: An open-source framework for Federated Learning. arXiv preprint arXiv:2105.06413 (13 May 2021). https://doi.org/10.48550/arXiv.2105.06413

[21]

Nicolò Tonci, Massimo Torquati, Gabriele Mencagli, and Marco Danelutto. 2023. Distributed-memory FastFlow Building Blocks. International of Parallel Programming 51 (2 December 2023), 1--21. https://doi.org/10.1007/s10766-022-00750-5

Digital Library

[22]

Massimo Torquati. 2019. Harnessing Parallelism in Multi/Many-Cores with Streams and Parallel Patterns. Ph. D. Dissertation. University of Pisa.

[23]

Carole-Jean Wu, David Brooks, Kevin Chen, Douglas Chen, Sy Choudhury, Marat Dukhan, Kim Hazelwood, Eldad Isaac, Yangqing Jia, Bill Jia, Tommer Leyvand, Hao Lu, Yang Lu, Lin Qiao, Brandon Reagen, Joe Spisak, Fei Sun, Andrew Tulloch, Peter Vajda, Xiaodong Wang, Yanghan Wang, Bram Wasti, Yiming Wu, Ran Xian, Sungjoo Yoo, and Peizhao Zhang. 2019. Machine learning at facebook: Understanding inference at the edge. In 2019 IEEE international symposium on high performance computer architecture (HPCA). IEEE, Washington, DC, USA, 331--344. https://doi.org/10.1109/HPCA.2019.00048

[24]

Yuexiang Xie, Zhen Wang, Daoyuan Chen, Dawei Gao, Liuyi Yao, Weirui Kuang, Yaliang Li, Bolin Ding, and Jingren Zhou. 2022. Federatedscope: A comprehensive and flexible federated learning platform via message passing. arXiv preprint arXiv:2204.05011 (11 April 2022). https://doi.org/10.48550/ARXIV.2204.05011

[25]

Alexander Ziller, Andrew Trask, Antonio Lopardo, Benjamin Szymkow, Bobby Wagner, Emma Bluemke, Jean-Mickael Nounahon, Jonathan Passerat-Palmbach, Kritika Prakash, Nick Rose, et al. 2021. Pysyft: A library for easy federated learning. Federated Learning Systems: Towards Next-Generation AI 965 (12 June 2021), 111--139. https://doi.org/10.1007/978-3-030-70604-3_5

Cited By

Iacussi LChiariotti PCigada A(2024)AI-Enhanced IoT System for Assessing Bridge Deflection in Drive-By ConditionsSensors10.3390/s2501015825:1(158)Online publication date: 30-Dec-2024
https://doi.org/10.3390/s25010158
Tan YYang LLi WWu YSchiavoni VEdinger JCao JJin Z(2024)RoleML: a Role-Oriented Programming Model for Customizable Distributed Machine Learning on EdgesProceedings of the 25th International Middleware Conference10.1145/3652892.3700765(279-291)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3652892.3700765
Colonnelli IBirke RMalenza GMittone GMulone AGaljaard JChen LBassini SScipione GMartinovič JVondrák VAldinucci M(2024)Cross-Facility Federated LearningProcedia Computer Science10.1016/j.procs.2024.07.003240:C(3-12)Online publication date: 18-Oct-2024
https://dl.acm.org/doi/10.1016/j.procs.2024.07.003
Show More Cited By

Index Terms

Experimenting with Emerging RISC-V Systems for Decentralised Machine Learning
1. Computing methodologies
  1. Distributed computing methodologies
    1. Distributed algorithms
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Hardware
  1. Emerging technologies
    1. Analysis and design of emerging devices and systems
      1. Emerging architectures

Recommendations

Performance Evaluation of Various RISC Processor Systems: A Case Study on ARM, MIPS and RISC-V
Cloud Computing – CLOUD 2021
Abstract
RISC-V is a new instruction set architecture (ISA) that has emerged in recent years. Compared with previous computer instruction architectures, RISC-V has outstanding features such as simple instructions, modular instruction set and supporting ...
AIRV: Enabling Deep Learning Inference on RISC-V
Benchmarking, Measuring, and Optimizing
Abstract
Recently the emerging RISC-V instruction set architecture (ISA) has been widely adopted by both academia and industry. Meanwhile, various artificial intelligence (AI) applications have been extensively deployed in cloud, edge, mobile and IoT ...
Evaluating and Enhancing Performance through Macro-Op Fusion Optimization with RISC-V
ICPP Workshops '24: Workshop Proceedings of the 53rd International Conference on Parallel Processing

RISC-V, renowned for its simplicity and extensibility, has garnered significant attention in the realm of high-performance processor construction. However, its streamlined design necessitates more instructions for tasks compared to established ISAs like ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CF '23: Proceedings of the 20th ACM International Conference on Computing Frontiers

May 2023

419 pages

ISBN:9798400701405

DOI:10.1145/3587135

General Chairs:
Andrea Bartolini
Università di Bologna, IT
,
Kristian Rietveld
Leiden University, NL
,
Program Chairs:
Catherine Schuman
University of Tennessee, US
,
Jose Moreira
IBM, US

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMICRO: ACM Special Interest Group on Microarchitectural Research and Processing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 August 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

European Union - EuPILOT
European Union - NextGenerationEU - Spoke Spoke FutureHPC & BigData? of the ICSC

Conference

CF '23

Sponsor:

SIGMICRO

CF '23: 20th ACM International Conference on Computing Frontiers

May 9 - 11, 2023

Bologna, Italy

Acceptance Rates

CF '23 Paper Acceptance Rate 24 of 66 submissions, 36%;

Overall Acceptance Rate 273 of 785 submissions, 35%

Upcoming Conference

CF '25

Sponsor:
sigmicro

22nd ACM International Conference on Computing Frontiers

May 28 - 30, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
568
Total Downloads

Downloads (Last 12 months)405
Downloads (Last 6 weeks)22

Reflects downloads up to 12 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Iacussi LChiariotti PCigada A(2024)AI-Enhanced IoT System for Assessing Bridge Deflection in Drive-By ConditionsSensors10.3390/s2501015825:1(158)Online publication date: 30-Dec-2024
https://doi.org/10.3390/s25010158
Tan YYang LLi WWu YSchiavoni VEdinger JCao JJin Z(2024)RoleML: a Role-Oriented Programming Model for Customizable Distributed Machine Learning on EdgesProceedings of the 25th International Middleware Conference10.1145/3652892.3700765(279-291)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3652892.3700765
Colonnelli IBirke RMalenza GMittone GMulone AGaljaard JChen LBassini SScipione GMartinovič JVondrák VAldinucci M(2024)Cross-Facility Federated LearningProcedia Computer Science10.1016/j.procs.2024.07.003240:C(3-12)Online publication date: 18-Oct-2024
https://dl.acm.org/doi/10.1016/j.procs.2024.07.003
Martínez Beltrán EPerales Gómez ÁFeng CSánchez Sánchez PLópez Bernal SBovet GGil Pérez MMartínez Pérez GHuertas Celdrán A(2024)FedstellarExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122861242:COnline publication date: 16-May-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122861
Mittone GBirke RAldinucci M(2024)Efficiently Distributed Federated LearningEuro-Par 2023: Parallel Processing Workshops10.1007/978-3-031-48803-0_40(321-326)Online publication date: 14-Apr-2024
https://doi.org/10.1007/978-3-031-48803-0_40
Prokić IGhilezan SKašterović SPopovic MPopovic MKaštelan I(2023)Correct Orchestration of Federated Learning Generic Algorithms: Formalisation and Verification in CSPEngineering of Computer-Based Systems10.1007/978-3-031-49252-5_25(274-288)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1007/978-3-031-49252-5_25
Fornaciari WReghenzani FTerraneo FBaroffio DMetra COmana MCondia JReorda MBirke RColonnelli IMittone GAldinucci MMencagli GIannone FPalombi FZummo GCesarini DTesser F(2023)RISC-V-Based Platforms for HPC: Analyzing Non-functional Properties for Future HPC and Big-Data ClustersEmbedded Computer Systems: Architectures, Modeling, and Simulation10.1007/978-3-031-46077-7_26(395-410)Online publication date: 2-Jul-2023
https://dl.acm.org/doi/10.1007/978-3-031-46077-7_26

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten