research-article

Evaluation of a Rack-Scale Disaggregated Memory Prototype for Cloud Data Centers

Authors:

Josue V. Quiroga,

Marti Torrents,

Dimitris Theodoropoulos,

Ferad Zyulkyarov,

Mario NemirovskyAuthors Info & Claims

RSP '19: Proceedings of the 30th International Workshop on Rapid System Prototyping (RSP'19)

Pages 15 - 21

https://doi.org/10.1145/3339985.3358496

Published: 17 October 2019 Publication History

Abstract

Disaggregated data centers propose a modular architecture where memory and compute resources are utilized in a finer granularity, aiming for a better optimized capacity and power consumption. In contrast to a classic compute node in a data center, memory is not required to be co-located with the processor, disaggregation can enhance hardware elasticity, improve virtual machine (VM) migration, and reduce the total cost of ownership (TCO), when compared to current data center solutions.

This paper presents an evaluation of a disaggregated memory system prototype implemented on two modern FPGA SoC boards; one as a compute node and the second one as memory node. Using the PARSEC and Splash2x benchmark suites, we characterize our disaggregated prototype running on a remote memory node. In order to understand the design trade offs, we utilize data collected from AXI Performance Monitors and break down the executions into three principal stages: (i) disaggregated memory address translation, (ii) network and (iii) memory access. This helps us to better evaluate the trade offs of our prototype and to give better insight towards the next generation of disaggregated data centers.

References

[1]

Nikolaos Alachiotis, Andreas Andronikakis, Orion Papadakis, Dimitris Theodoropoulos, Dionisios Pnevmatikatos, Dimitris Syrivelis, Andrea Reale, Kostas Katrinis, George Zervas, Vaibhawa Mishra, et al. 2019. dReDBox: A Disaggregated Architectural Perspective for Data Centers. In Hardware Accelerators in Data Centers. Springer, 35--56.

[2]

Maciej Bielski, Christian Pinto, Daniel Raho, and Renaud Pacalet. 2016. Survey on Memory and Devices Disaggregation Solutions for HPC Systems. In Computational Science and Engineering (CSE) and IEEE Intl Conference on Embedded and Ubiquitous Computing (EUC) and 15th Intl Symposium on Distributed Computing and Applications for Business Engineering (DCABES), 2016 IEEE Intl Conference on. IEEE, 197--204.

[3]

Maciej Bielski, Ilias Syrigos, Kostas Katrinis, Dimitris Syrivelis, Andrea Reale, Dimitris Theodoropoulos, Nikolaos Alachiotis, D Pnevmatikatos, EH Pap, George Zervas, et al. 2018. dReDBox: materializing a full-stack rack-scale system prototype of a next-generation disaggregated datacenter. In 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, 1093--1098.

[4]

Christian Bienia, Sanjeev Kumar, Jaswinder Pal Singh, and Kai Li. 2008. The PARSEC benchmark suite: Characterization and architectural implications. In Proceedings of the 17th international conference on Parallel architectures and compilation techniques. ACM, 72--81.

Digital Library

[5]

Dhruva R Chakrabarti, Hans-J Boehm, and Kumud Bhandari. 2014. Atlas: Leveraging locks for non-volatile memory consistency. In ACM SIGPLAN Notices, Vol. 49. ACM, 433--452.

[6]

Paolo Faraboschi, Kimberly Keeton, Tim Marsland, and Dejan S Milojicic. 2015. Beyond Processor-centric Operating Systems. In HotOS.

[7]

Engin Ipek, Onur Mutlu, José F Martínez, and Rich Caruana. 2008. Self-optimizing memory controllers: A reinforcement learning approach. In ACM SIGARCH Computer Architecture News, Vol. 36. IEEE Computer Society, 39--50.

Digital Library

[8]

Kostas Katrinis, Dimitris Syrivelis, D Pnevmatikatos, Georgios Zervas, Dimitris Theodoropoulos, Iordanis Koutsopoulos, K Hasharoni, Daniel Raho, Christian Pinto, F Espina, et al. 2016. Rack-scale disaggregated cloud data centers: The dReDBox project vision. In Proceedings of the 2016 Conference on Design, Automation & Test in Europe. EDA Consortium, 690--695.

[9]

Kimberly Keeton. 2015. The machine: An architecture for memory-centric computing. In Workshop on Runtime and Operating Systems for Supercomputers (ROSS).

Digital Library

[10]

Kevin Lim, Jichuan Chang, Trevor Mudge, Parthasarathy Ranganathan, Steven K Reinhardt, and Thomas F Wenisch. 2009. Disaggregated memory for expansion and sharing in blade servers. In ACM SIGARCH Computer Architecture News, Vol. 37. ACM, 267--278.

Digital Library

[11]

Kevin Lim, Yoshio Turner, Jose Renato Santos, Alvin AuYoung, Jichuan Chang, Parthasarathy Ranganathan, and Thomas F Wenisch. 2012. System-level implications of disaggregated memory. In High Performance Computer Architecture (HPCA), 2012 IEEE 18th International Symposium on. IEEE, 1--12.

Digital Library

[12]

Hugo Meyer, Jose Carlos Sancho, Josue V Quiroga, Ferad Zyulkyarov, Damian Roca, and Mario Nemirovsky. 2017. Disaggregated computing. an evaluation of current trends for datacentres. Procedia Computer Science 108 (2017), 685--694.

[13]

Stanko Novakovic, Paolo Faraboschi, Kimberly Keeton, Rob Schreiber, Edouard Bugnion, and Babak Falsafi. 2015. NeVer Mind networking: Using shared nonvolatile memory in scale-out software. In presentation, 2ndIntâĂ&Zacute;l Workshop Rack-Scale Computing (WRSC 15).

[14]

Omar Sefraoui, Mohammed Aissaoui, and Mohsine Eleuldj. 2012. OpenStack: toward an open-source solution for cloud computing. International Journal of Computer Applications 55, 3 (2012), 38--42.

[15]

D. Theodoropoulos, A. Reale, D. Syrivelis, M. Bielski, N. Alachiotis, and D. Pnevmatikatos. 2018. REMAP: Remote mEmory Manager for disAggregated Platforms. In 2018 IEEE 29th International Conference on Application-specific Systems, Architectures and Processors (ASAP). 1--8.

[16]

Marti Torrents, Josue Quiroga, Tugberk Arkose, Ferad Zyulkyarov, and Mario Nemirovsky. [n. d.]. Resource utilization analysis in a disaggregated datacenter architecture. ([n. d.]).

[17]

Steven Cameron Woo, Moriyoshi Ohara, Evan Torrie, Jaswinder Pal Singh, and Anoop Gupta. 1995. The SPLASH-2 programs: Characterization and methodological considerations. In ACM SIGARCH computer architecture news, Vol. 23. ACM, 24--36.

[18]

Xilinx. [n. d.]. ZCU102 Evaluation Board User Guide UG1182 (v1.4) October 4, 2018.

Cited By

Giles E(2024)Towards an Open, Hybrid AI Scale ArchitectureProceedings of the 2024 9th International Conference on Cloud Computing and Internet of Things10.1145/3704304.3704319(110-117)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1145/3704304.3704319
Puri ABellamkonda KNarreddy KJose JVenkatesh T(2023)A Practical Approach For Workload-Aware Data Movement in Disaggregated Memory Systems2023 IEEE 35th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)10.1109/SBAC-PAD59825.2023.00017(78-88)Online publication date: 17-Oct-2023
https://doi.org/10.1109/SBAC-PAD59825.2023.00017
Guo XXue XYan FPan BExarchakos GCalabretta N(2021)DACON: a reconfigurable application-centric optical network for disaggregated data center infrastructures [Invited]Journal of Optical Communications and Networking10.1364/JOCN.43895014:1(A69)Online publication date: 17-Nov-2021
https://doi.org/10.1364/JOCN.438950

Recommendations

Clio: a hardware-software co-designed disaggregated memory system
ASPLOS '22: Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

Memory disaggregation has attracted great attention recently because of its benefits in efficient memory utilization and ease of management. So far, memory disaggregation research has all taken one of two approaches: building/emulating memory nodes using ...
Memory sharing for handling memory overload on physical machines in cloud data centers
Abstract
Over-committing computing resources is a widely adopted strategy for increased cluster utilization in Infrastructure as a Service (IaaS) cloud data centers. A potential consequence of over-committing computing resources is memory overload of ...
Disaggregated Cloud Memory with Elastic Block Management
With the growing importance of in-memory data processing, cloud service providers have launched large memory virtual machine services to accommodate memory intensive workloads. Such large memory services using low volume scaled-up machines are far less ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

RSP '19: Proceedings of the 30th International Workshop on Rapid System Prototyping (RSP'19)

October 2019

80 pages

ISBN:9781450368476

DOI:10.1145/3339985

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

CEDA: Council on Electronic Design Automation
SIGBED: ACM Special Interest Group on Embedded Systems
SIGDA: ACM Special Interest Group on Design Automation
IEEE CAS
SIGMICRO: ACM Special Interest Group on Microarchitectural Research and Processing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ESWEEK '19

Sponsor:

ESWEEK '19: Fifteenth Embedded Systems Week

October 17 - 18, 2019

NY, New York, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
291
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)1

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Giles E(2024)Towards an Open, Hybrid AI Scale ArchitectureProceedings of the 2024 9th International Conference on Cloud Computing and Internet of Things10.1145/3704304.3704319(110-117)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1145/3704304.3704319
Puri ABellamkonda KNarreddy KJose JVenkatesh T(2023)A Practical Approach For Workload-Aware Data Movement in Disaggregated Memory Systems2023 IEEE 35th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)10.1109/SBAC-PAD59825.2023.00017(78-88)Online publication date: 17-Oct-2023
https://doi.org/10.1109/SBAC-PAD59825.2023.00017
Guo XXue XYan FPan BExarchakos GCalabretta N(2021)DACON: a reconfigurable application-centric optical network for disaggregated data center infrastructures [Invited]Journal of Optical Communications and Networking10.1364/JOCN.43895014:1(A69)Online publication date: 17-Nov-2021
https://doi.org/10.1364/JOCN.438950

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten