short-paper

Is Memory Disaggregation Feasible?: A Case Study with Spark SQL

Authors:

Pramod Subba Rao,

George PorterAuthors Info & Claims

ANCS '16: Proceedings of the 2016 Symposium on Architectures for Networking and Communications Systems

Pages 75 - 80

https://doi.org/10.1145/2881025.2881030

Published: 17 March 2016 Publication History

Abstract

This paper explores the feasibility of entirely disaggregated memory from compute and storage for a particular, widely deployed workload, Spark SQL analytics queries. We measure the empirical rate at which records are processed and calculate the effective memory bandwidth utilized based on the sizes of the columns accessed in the query. Our findings contradict conventional wisdom: not only is memory disaggregation possible under this workload, but achievable with already available, commercial network technology. Beyond this finding, we also recommend changes that can be made to Spark SQL to improve its ability to support memory disaggregation.

References

[1]

Apache Spark version 1.3.0. https://github.com/apache/spark/tree/branch-1.3.

[2]

Best Practice Guidelines for ProLiant Servers with the Intel Xeon 5500 processor series Engineering Whitepaper, 1st Edition, Figure 6. ftp://ftp.hp.com/pub/c-products/servers/options/Memory-Config-Recommendations-for-Intel-Xeon-5500-Series-Servers-Rev1.pdf.

[3]

Big Data Benchmark. https://amplab.cs.berkeley.edu/benchmark/.

[4]

Intel Performance Counter Monitor - A better way to measure CPU utilization. https://software.intel.com/en-us/articles/intel-performance-counter-monitor/.

[5]

Introducing Yosemite: the first open source modular chassis for high-powered microservers. https://code.facebook.com/posts/1616052405274961/introducing-yosemite-the-first-open-source-modular-chassis-for-high-powered-microservers-/.

[6]

Spark CSV. https://github.com/databricks/spark-csv.

[7]

stream-scaling. https://github.com/gregs1104/stream-scaling.

[8]

Amplab big data benchmark. https://amplab.cs.berkeley.edu/benchmark/.

[9]

M. Armbrust, R. S. Xin, C. Lian, Y. Huai, D. Liu, J. K. Bradley, X. Meng, T. Kaftan, M. J. Franklin, A. Ghodsi, and M. Zaharia. Spark sql: Relational data processing in spark. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, SIGMOD '15, pages 1383--1394, New York, NY, USA, 2015. ACM.

Digital Library

[10]

IEEE P802.3bs 400 Gb/s Ethernet Task Force. http://www.ieee802.org/3/bs/index.html.

[11]

Introducing data center fabric, the next-generation facebook data center network. https://code.facebook.com/posts/360346274145943/introducing-data-center-fabric-the-next-generation-facebook-data-center-network/.

[12]

S. Han, N. Egi, A. Panda, S. Ratnasamy, G. Shi, and S. Shenker. Network support for resource disaggregation in next-generation datacenters. In Proceedings of the Twelfth ACM Workshop on Hot Topics in Networks, HotNets-XII, pages 10:1--10:7, New York, NY, USA, 2013. ACM.

Digital Library

[13]

Impala. http://impala.io/.

[14]

K. Lim, J. Chang, T. Mudge, P. Ranganathan, S. K. Reinhardt, and T. F. Wenisch. Disaggregated memory for expansion and sharing in blade servers. In Proceedings of the 36th Annual International Symposium on Computer Architecture, ISCA '09, pages 267--278, New York, NY, USA, 2009. ACM.

Digital Library

[15]

K. Lim, Y. Turner, J. R. Santos, A. AuYoung, J. Chang, P. Ranganathan, and T. F. Wenisch. System-level implications of disaggregated memory. In Proceedings of the 2012 IEEE 18th International Symposium on High-Performance Computer Architecture, HPCA '12, pages 1--12, Washington, DC, USA, 2012. IEEE Computer Society.

Digital Library

[16]

J. D. McCalpin. Stream: Sustainable memory bandwidth in high performance computers. https://www.cs.virginia.edu/stream/, 1995.

[17]

ConnectX-4 single/dual-port adapter supporting 100 Gb/s with VPI. http://www.mellanox.com/page/products_dyn?product_family=201&mtag=connectx_4_vpi_card.

[18]

K. Ousterhout, R. Rasti, S. Ratnasamy, S. Shenker, and B.-G. Chun. Making sense of performance in data analytics frameworks. In Proceedings of the 12th USENIX Conference on Networked Systems Design and Implementation, NSDI'15, pages 293--307, Berkeley, CA, USA, 2015. USENIX Association.

Digital Library

[19]

Amazon AWS RedShift. http://aws.amazon.com/redshift/.

[20]

Apache Tez. https://tez.apache.org/.

[21]

W. A. Wulf and S. A. McKee. Hitting the memory wall: Implications of the obvious. SIGARCH Comput. Archit. News, 23(1):20--24, Mar. 1995.

Digital Library

[22]

M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica. Spark: Cluster computing with working sets. In Proceedings of the 2Nd USENIX Conference on Hot Topics in Cloud Computing, HotCloud'10, pages 10--10, Berkeley, CA, USA, 2010. USENIX Association.

Digital Library

Cited By

Wang QLu YShu J(2023)Building Write-Optimized Tree Indexes on Disaggregated MemoryACM SIGMOD Record10.1145/3604437.360444852:1(45-52)Online publication date: 8-Jun-2023
https://dl.acm.org/doi/10.1145/3604437.3604448
Zhong KCui WChen XLi QYang ZLu YYan XLuo SYuan QHuang K(2023)Revisiting Swapping in User-Space With Lightweight ThreadingIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2023.327495342:11(4205-4218)Online publication date: Dec-2023
https://doi.org/10.1109/TCAD.2023.3274953
Katsaragakis MMasouros DPapadopoulos LCatthoor FSoudris D(2023)On the Implications of Heterogeneous Memory Tiering on Spark In-Memory Analytics2023 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)10.1109/IPDPSW59300.2023.00157(945-952)Online publication date: May-2023
https://doi.org/10.1109/IPDPSW59300.2023.00157
Show More Cited By

Index Terms

Is Memory Disaggregation Feasible?: A Case Study with Spark SQL
1. Networks

Recommendations

Rcmp: Reconstructing RDMA-Based Memory Disaggregation via CXL
Memory disaggregation is a promising architecture for modern datacenters that separates compute and memory resources into independent pools connected by ultra-fast networks, which can improve memory utilization, reduce cost, and enable elastic scaling of ...
Poster: Designing a Memory Disaggregation System for Cloud
ACM SIGCOMM '23: Proceedings of the ACM SIGCOMM 2023 Conference

Memory disaggregation is a new datacenter paradigm separating compute and memory nodes. While memory disaggregation improves memory utilization and scalability, it poses challenges for cloud applications, particularly in terms of high tail latency. ...
Efficient memory disaggregation with INFINISWAP
NSDI'17: Proceedings of the 14th USENIX Conference on Networked Systems Design and Implementation

Memory-intensive applications suffer large performance loss when their working sets do not fully fit in memory. Yet, they cannot leverage otherwise unused remote memory when paging out to disks even in the presence of large imbalance in memory ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ANCS '16: Proceedings of the 2016 Symposium on Architectures for Networking and Communications Systems

March 2016

148 pages

ISBN:9781450341837

DOI:10.1145/2881025

General Chair:
Patrick Crowley
Washington University in St. Louis, USA
,
Program Chairs:
Luigi Rizzo
Università di Pisa, Italy
,
Laurent Mathy
Université de Liège, Belgium

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 March 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Short-paper

Funding Sources

National Science Foundation

Conference

ANCS '16

Sponsor:

ANCS '16: Symposium on Architectures for Networking and Communications Systems

March 17 - 18, 2016

California, Santa Clara, USA

Acceptance Rates

ANCS '16 Paper Acceptance Rate 12 of 58 submissions, 21%;

Overall Acceptance Rate 88 of 314 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
512
Total Downloads

Downloads (Last 12 months)32
Downloads (Last 6 weeks)1

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang QLu YShu J(2023)Building Write-Optimized Tree Indexes on Disaggregated MemoryACM SIGMOD Record10.1145/3604437.360444852:1(45-52)Online publication date: 8-Jun-2023
https://dl.acm.org/doi/10.1145/3604437.3604448
Zhong KCui WChen XLi QYang ZLu YYan XLuo SYuan QHuang K(2023)Revisiting Swapping in User-Space With Lightweight ThreadingIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2023.327495342:11(4205-4218)Online publication date: Dec-2023
https://doi.org/10.1109/TCAD.2023.3274953
Katsaragakis MMasouros DPapadopoulos LCatthoor FSoudris D(2023)On the Implications of Heterogeneous Memory Tiering on Spark In-Memory Analytics2023 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)10.1109/IPDPSW59300.2023.00157(945-952)Online publication date: May-2023
https://doi.org/10.1109/IPDPSW59300.2023.00157
Zhang YLi GWan JWang JLi JYao TWu HWang D(2023)DoW-KV: A DPU-offloaded and Write-optimized Key-Value Store on Disaggregated Persistent Memory2023 IEEE International Conference on Cluster Computing (CLUSTER)10.1109/CLUSTER52292.2023.00030(271-283)Online publication date: 31-Oct-2023
https://doi.org/10.1109/CLUSTER52292.2023.00030
Ewais MChow P(2023)Disaggregated Memory in the Datacenter: A SurveyIEEE Access10.1109/ACCESS.2023.325040711(20688-20712)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3250407
Wang QLu YShu JIves ZBonifati AEl Abbadi A(2022)Sherman: A Write-Optimized Distributed B+Tree Index on Disaggregated MemoryProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3517824(1033-1048)Online publication date: 10-Jun-2022
https://dl.acm.org/doi/10.1145/3514221.3517824
Mehra PCoughlin T(2022)Taming Memory With DisaggregationComputer10.1109/MC.2022.318784755:9(94-98)Online publication date: Oct-2022
https://doi.org/10.1109/MC.2022.3187847
Sun XZhang NToonen BAllcock B(2020)Performance Modeling and Evaluation of a Production Disaggregated Memory SystemProceedings of the International Symposium on Memory Systems10.1145/3422575.3422795(223-232)Online publication date: 28-Sep-2020
https://dl.acm.org/doi/10.1145/3422575.3422795
Zacarias FNishtala RCarpenter PPalesi MPalermo GGraves CArima E(2020)Contention-aware application performance prediction for disaggregated memory systemsProceedings of the 17th ACM International Conference on Computing Frontiers10.1145/3387902.3392625(49-59)Online publication date: 11-May-2020
https://dl.acm.org/doi/10.1145/3387902.3392625
Pinto CSyrivelis DGazzetti MKoutsovasilis PReale AKatrinis KHofstee H(2020)ThymesisFlow: A Software-Defined, HW/SW co-Designed Interconnect Stack for Rack-Scale Memory Disaggregation2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO50266.2020.00075(868-880)Online publication date: Oct-2020
https://doi.org/10.1109/MICRO50266.2020.00075
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents