research-article

Efficient in-memory data management: an analysis

Authors:

Hao Zhang,

Bogdan Marius Tudor,

Gang Chen,

Beng Chin OoiAuthors Info & Claims

Proceedings of the VLDB Endowment, Volume 7, Issue 10

Pages 833 - 836

https://doi.org/10.14778/2732951.2732956

Published: 01 June 2014 Publication History

Get Access

Abstract

This paper analyzes the performance of three systems for in-memory data management: Memcached, Redis and the Resilient Distributed Datasets (RDD) implemented by Spark. By performing a thorough performance analysis of both analytics operations and fine-grained object operations such as set/get, we show that neither system handles efficiently both types of workloads. For Memcached and Redis the CPU and I/O performance of the TCP stack are the bottlenecks -- even when serving in-memory objects within a single server node. RDD does not support efficient get operation for random objects, due to a large startup cost of the get job. Our analysis reveals a set of features that a system must support in order to achieve efficient in-memory data management.

References

[1]

Aredis java redis client. http://aredis.sourceforge.net/.

Google Scholar

[2]

Memcached. http://memcached.org.

Google Scholar

[3]

Redis. http://redis.io.

Google Scholar

[4]

Spymemcached memcached client. https://code.google.com/p/spymemcached/.

Google Scholar

[5]

Stanford large network dataset collection. https://snap.stanford.edu/data/.

Google Scholar

[6]

K. Lim, D. Meisner, A. G. Saidi, P. Ranganathan, and T. F. Wenisch. Thin Servers with Smart Pipes: Designing SoC Accelerators for Memcached. In ISCA, 2013.

Digital Library

Google Scholar

[7]

M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauley, M. J. Franklin, S. Shenker, and I. Stoica. Resilient Distributed Datasets: A Fault-tolerant Abstraction for In-memory Cluster Computing. In NSDI, 2012.

Digital Library

Google Scholar

Cited By

View all

Miranda AOña PMendoza KChango W(2024)NoSQL Database Benchmarking: Key/value in an Application-building EnvironmentESPOCH Congresses: The Ecuadorian Journal of S.T.E.A.M.10.18502/espoch.v4i1.158133:2(129-142)Online publication date: 18-Apr-2024
https://doi.org/10.18502/espoch.v4i1.15813
Aoba HKawashima RMatsuo H(2023)Accelerating Key-Value Store with Layer-2 Transparent Proxy Cache Using DPDK2023 Eleventh International Symposium on Computing and Networking Workshops (CANDARW)10.1109/CANDARW60564.2023.00056(298-302)Online publication date: 27-Nov-2023
https://doi.org/10.1109/CANDARW60564.2023.00056
Denninnart CSalehi M(2022)Harnessing the Potential of Function-Reuse in Multimedia Cloud SystemsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2021.309791133:3(617-629)Online publication date: 1-Mar-2022
https://doi.org/10.1109/TPDS.2021.3097911
Show More Cited By

Efficient in-memory data management: an analysis

Recommendations

Big Data Management: Advanced Issues and Approaches

The objective of this article is to provide the advanced issues and approaches of big data management. The literature review indicates the overview of big data management; the aspects of Big Data Analytics BDA; the importance of big data management; the ...
A survey of big data management

The rapid growth of emerging applications and the evolution of cloud computing technologies have significantly enhanced the capability to generate vast amounts of data. Thus, it has become a great challenge in this big data era to manage such voluminous ...
Big data technologies and Management

The era of big data has resulted in the development and applications of technologies and methods aimed at effectively using massive amounts of data to support decision-making and knowledge discovery activities. In this paper, the five Vs of big data, ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the VLDB Endowment

Proceedings of the VLDB Endowment Volume 7, Issue 10

June 2014

146 pages

ISSN:2150-8097

Editors:
H. V. Jagadish
University of Michigan
,
Aoying Zhou
East Normal University, China

Issue’s Table of Contents

Publisher

VLDB Endowment

Publication History

Published: 01 June 2014

Published in PVLDB Volume 7, Issue 10

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
523
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Miranda AOña PMendoza KChango W(2024)NoSQL Database Benchmarking: Key/value in an Application-building EnvironmentESPOCH Congresses: The Ecuadorian Journal of S.T.E.A.M.10.18502/espoch.v4i1.158133:2(129-142)Online publication date: 18-Apr-2024
https://doi.org/10.18502/espoch.v4i1.15813
Aoba HKawashima RMatsuo H(2023)Accelerating Key-Value Store with Layer-2 Transparent Proxy Cache Using DPDK2023 Eleventh International Symposium on Computing and Networking Workshops (CANDARW)10.1109/CANDARW60564.2023.00056(298-302)Online publication date: 27-Nov-2023
https://doi.org/10.1109/CANDARW60564.2023.00056
Denninnart CSalehi M(2022)Harnessing the Potential of Function-Reuse in Multimedia Cloud SystemsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2021.309791133:3(617-629)Online publication date: 1-Mar-2022
https://doi.org/10.1109/TPDS.2021.3097911
Hemmatpour MMontrucchio BRebaudengo MSadoghi M(2022)Analyzing In-Memory NoSQL LandscapeIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2020.300290834:4(1628-1643)Online publication date: 1-Apr-2022
https://doi.org/10.1109/TKDE.2020.3002908
Tang SHe BYu CLi YLi K(2022)A Survey on Spark Ecosystem: Big Data Processing Infrastructure, Machine Learning, and ApplicationsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2020.297565234:1(71-91)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1109/TKDE.2020.2975652
DAVID AMIHAI DMIHAILESCU MCARABAS MTAPUS N(2022)Scalability through Distributed Deployment for Moodle Learning Management SystemProcedia Computer Science10.1016/j.procs.2022.11.145214(34-41)Online publication date: 2022
https://doi.org/10.1016/j.procs.2022.11.145
Wu PWang SFeng THuang YHuang TLiu Y(2021)NCA: Accelerating Network Caching with eXpress Data Path2021 4th International Conference on Hot Information-Centric Networking (HotICN)10.1109/HotICN53262.2021.9680837(141-146)Online publication date: 25-Nov-2021
https://doi.org/10.1109/HotICN53262.2021.9680837
Amini Salehi MLi XSalehi MLi X(2021)Service Reuse in Multimedia CloudsMultimedia Cloud Computing Systems10.1007/978-3-030-88451-2_6(97-123)Online publication date: 13-Sep-2021
https://doi.org/10.1007/978-3-030-88451-2_6
Amini Salehi MLi XSalehi MLi X(2021)Future of Multimedia Streaming and Cloud TechnologyMultimedia Cloud Computing Systems10.1007/978-3-030-88451-2_10(179-187)Online publication date: 13-Sep-2021
https://doi.org/10.1007/978-3-030-88451-2_10
Cai QZhang HGuo WChen GOoi BTan KWong W(2019)MemepiC: Towards a Unified In-Memory Big Data Management SystemIEEE Transactions on Big Data10.1109/TBDATA.2017.27892865:1(4-17)Online publication date: 1-Mar-2019
https://doi.org/10.1109/TBDATA.2017.2789286
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Recommendations

Big Data Management: Advanced Issues and Approaches

A survey of big data management

Big data technologies and Management

Comments

Information

Published In

Publisher

Publication History

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations