research-article

Free access

HyperDex: a distributed, searchable key-value store

Authors:

Robert Escriva,

Emin Gün SirerAuthors Info & Claims

SIGCOMM '12: Proceedings of the ACM SIGCOMM 2012 conference on Applications, technologies, architectures, and protocols for computer communication

Pages 25 - 36

https://doi.org/10.1145/2342356.2342360

Published: 13 August 2012 Publication History

Abstract

Distributed key-value stores are now a standard component of high-performance web services and cloud computing applications. While key-value stores offer significant performance and scalability advantages compared to traditional databases, they achieve these properties through a restricted API that limits object retrieval---an object can only be retrieved by the (primary and only) key under which it was inserted. This paper presents HyperDex, a novel distributed key-value store that provides a unique search primitive that enables queries on secondary attributes. The key insight behind HyperDex is the concept of hyperspace hashing in which objects with multiple attributes are mapped into a multidimensional hyperspace. This mapping leads to efficient implementations not only for retrieval by primary key, but also for partially-specified secondary attribute searches and range queries. A novel chaining protocol enables the system to achieve strong consistency, maintain availability and guarantee fault tolerance. An evaluation of the full system shows that HyperDex is 12-13x faster than Cassandra and MongoDB for finding partially specified objects. Additionally, HyperDex achieves 2-4x higher throughput for get/put operations.

Supplementary Material

JPG File (sigcomm-i-03-hyperdexadistributedsearchablekeyvaluestore.jpg)

Download
21.26 KB

MP4 File (sigcomm-i-03-hyperdexadistributedsearchablekeyvaluestore.mp4)

Download
64.61 MB

References

[1]

M. K. Aguilera, W. M. Golab, and M. A. Shah. A Practical Scalable Distributed B-Tree. In PVLDB, 1(1), 2008.

Digital Library

[2]

D. Altínbüken and E. G. Sirer. Commodifying Replicated State Machines with OpenReplica. Computing and Information Science, Cornell University, Technical Report 1813--29009, 2012.

[3]

S. Anand. http://www.infoq.com/presentations/NoSQL-Netflix/.

[4]

D. G. Andersen, J. Franklin, M. Kaminsky, A. Phanishayee, L. Tan, and V. Vasudevan. FAWN: A Fast Array of Wimpy Nodes. In Proc. of SOSP, Big Sky, MT, Oct. 2009.

Digital Library

[5]

R. Bayer. The Universal B-Tree for Multidimensional Indexing: General Concepts. In Proc. of WWCA, Tsukuba, Japan, Mar. 1997.

Digital Library

[6]

R. E. Bellman. Dynamic Programming. Princeton University Press, 1957.

[7]

J. L. Bentley. Multidimensional Binary Search Trees Used for Associative Searching. In CACM, 18(9), 1975.

Digital Library

[8]

A. R. Bharambe, M. Agrawal, and S. Seshan. Mercury: Supporting Scalable Multi-Attribute Range Queries. In Proc. of SIGCOMM, Portland, OR, Aug. 2004.

Digital Library

[9]

M. Burrows. The Chubby Lock Service for Loosely-Coupled Distributed Systems. In Proc. of OSDI, Seattle, WA, Nov. 2006.

Digital Library

[10]

M. Cai, M. R. Frank, J. Chen, and P. A. Szekely. MAAN: A Multi-Attribute Addressable Network for Grid Information Services. In Proc. of GRID Workshop, Phoenix, AZ, Nov. 2003.

Digital Library

[11]

F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. Gruber. BigTable: A Distributed Storage System for Structured Data. In Proc. of OSDI, Seattle, WA, Nov. 2006.

Digital Library

[12]

J. Cipar, G. R. Ganger, K. Keeton, C. B. M. III, C. A. N. Soules, and A. C. Veitch. LazyBase: Trading Freshness for Performance in a Scalable Database. In Proc. of EuroSys, Bern, Switzerland, Apr. 2012.

Digital Library

[13]

A. T. Clements, D. R. K. Ports, and D. R. Karger. Arpeggio: Metadata Searching and Content Sharing with Chord. In Proc. of IPTPS Workshop, La Jolla, CA, Feb. 2005.

Digital Library

[14]

B. F. Cooper, R. Ramakrishnan, U. Srivastava, A. Silberstein, P. Bohannon, H.-A. Jacobsen, N. Puz, D. Weaver, and R. Yerneni. PNUTS: Yahoo!'s Hosted Data Serving Platform. In PVLDB, 1(2), 2008.

Digital Library

[15]

B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. Benchmarking Cloud Serving Systems with YCSB. In Proc. of SoCC, Indianapolis, IN, June 2010.

Digital Library

[16]

CouchDB. http://couchdb.apache.org/.

[17]

A. Crainiceanu, P. Linga, J. Gehrke, and J. Shanmugasundaram. Querying Peer-to-Peer Networks Using P-Trees. In Proc. of WebDB Workshop, Paris, France, June 2004.

Digital Library

[18]

J. Dean. Designs, Lessons, and Advice from Building Large Distributed Systems. Keynote. In Proc. of LADIS, Big Sky, MT, Oct. 2009.

[19]

G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: Amazon's Highly Available Key-Value Store. In Proc. of SOSP, Stevenson, WA, Oct. 2007.

Digital Library

[20]

R. Escriva, B. Wong, and E. G. Sirer. Hyperdex. http://hyperdex.org/.

[21]

P. Ganesan, B. Yang, and H. Garcia-Molina. One Torus to Rule Them All: Multidimensional Queries in P2P Systems. In Proc. of WebDB Workshop, Paris, France, June 2004.

Digital Library

[22]

S. Gilbert and N. A. Lynch. Brewer's Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services. In SIGACT News, 33(2), 2002.

Digital Library

[23]

S. D. Gribble. A Design Framework and a Scalable Storage Platform to Simplify Internet Service Construction. U.C. Berkeley, 2000.

Digital Library

[24]

A. Guttman. R-Trees: A Dynamic Index Structure for Spatial Searching. In Proc. of SIGMOD, 1984.

Digital Library

[25]

HBase. http://hbase.apache.org/.

[26]

M. Herlihy and J. M. Wing. Linearizability: A Correctness Condition for Concurrent Objects. In ACM ToPLaS, 12(3), 1990.

Digital Library

[27]

P. Hunt, M. Konar, F. P. Junqueira, and B. Reed. ZooKeeper: Wait-Free Coordination for Internet-Scale Systems. In Proc. of USENIX, Boston, MA, June 2010.

Digital Library

[28]

Hypertable. http://http://hypertable.org/.

[29]

D. R. Karger, E. Lehman, F. T. Leighton, R. Panigrahy, M. S. Levine, and D. Lewin. Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web. In Proc. of STOC, El Paso, TX, May 1997.

Digital Library

[30]

F. B. Kashani and C. Shahabi. SWAM: A Family of Access Methods for Similarity-Search in Peer-to-Peer Data Networks. In Proc. of CIKM, Washington, D.C., Nov. 2004.

Digital Library

[31]

A. Klinger. Patterns and Search Statistics. In Optimizing Methods in Statistics, Academic Press, 1971.

[32]

A. Lakshman and P. Malik. Cassandra: A Decentralized Structured Storage System. In Proc. of LADIS, Big Sky, MT, Oct. 2009.

[33]

H. Lim, B. Fan, D. G. Andersen, and M. Kaminsky. SILT: A Memory-Efficient, High-Performance Key-Value Store. In Proc. of SOSP, Cascais, Portugal, Oct. 2011.

Digital Library

[34]

W. Lloyd, M. J. Freedman, M. Kaminsky, and D. G. Andersen. Don't Settle for Eventual: Scalable Causal Consistency for Wide-Area Storage with COPS. In Proc. of SOSP, Cascais, Portugal, Oct. 2011.

Digital Library

[35]

Y. Mao, E. Kohler, and R. T. Morris. Cache Craftiness for Fast Multicore Key-Value Storage. In Proc. of EuroSys, Bern, Switzerland, Apr. 2012.

Digital Library

[36]

Memcached. http://memcached.org/.

[37]

MongoDB. http://www.mongodb.org/.

[38]

Neo4j. http://neo4j.org/.

[39]

J. Nievergelt, H. Hinterberger, and K. C. Sevcik. The Grid File: An Adaptable, Symmetric Multikey File Structure. In ACM ToDS, 9(1), 1984.

Digital Library

[40]

D. Ongaro, S. M. Rumble, R. Stutsman, J. K. Ousterhout, and M. Rosenblum. Fast Crash Recovery in RAMCloud. In Proc. of SOSP, Cascais, Portugal, Oct. 2011.

Digital Library

[41]

J. A. Orenstein and T. H. Merrett. A Class of Data Structures for Associative Searching. In Proc. of PODS, 1984.

Digital Library

[42]

L. Peterson, A. Bavier, and S. Bhatia. VICCI: A Programmable Cloud-Computing Research Testbed. Princeton, Technical Report TR-912--11, 2011.

[43]

D. R. K. Ports, A. T. Clements, I. Zhang, S. Madden, and B. Liskov. Transactional Consistency and Automatic Management in an Application Data Cache. In Proc. of OSDI, Vancouver, Canada, Oct. 2010.

Digital Library

[44]

D. Pritchett. BASE: An ACID Alternative. In ACM Queue, 6(3), 2008.

Digital Library

[45]

Project Voldemort. http://project-voldemort.com/.

[46]

J. Rao, E. J. Shekita, and S. Tata. Using Paxos to Build a Scalable, Consistent, and Highly Available Datastore. In PVLDB, 4(4), 2011.

Digital Library

[47]

S. Ratnasamy, P. Francis, M. Handley, R. M. Karp, and S. Shenker. A Scalable Content-Addressable Network. In Proc. of SIGCOMM, San Diego, CA, Aug. 2001.

Digital Library

[48]

Redis. http://redis.io/.

[49]

Riak. http://basho.com/.

[50]

A. I. T. Rowstron and P. Druschel. Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems. In Proc. of ICDSP, volume 2218, 2001.

Digital Library

[51]

H. Samet. Spatial Data Structures. In Modern Database Systems: The Object Model, Interoperability, and Beyond, Addison Wesley/ACM Press, 1995.

Digital Library

[52]

C. Schmidt and M. Parashar. Enabling Flexible Queries with Guarantees in P2P Systems. In Internet Computing Journal, 2004.

Digital Library

[53]

Y. Shu, B. C. Ooi, K.-L. Tan, and A. Zhou. Supporting Multi-Dimensional Range Queries in Peer-to-Peer Systems. In Proc. of IEEE International Conference on Peer-to-Peer Computing, Konstanz, Germany, Aug. 2005.

Digital Library

[54]

R. P. Spillane, P. J. Shetty, E. Zadok, S. Dixit, and S. Archak. An Efficient Multi-Tier Tablet Server Storage Architecture. In Proc. of SoCC, 2011.

Digital Library

[55]

I. Stoica, R. Morris, D. R. Karger, M. F. Kaashoek, and H. Balakrishnan. Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications. In Proc. of SIGCOMM, San Diego, CA, Aug. 2001.

Digital Library

[56]

D. B. Terry, M. Theimer, K. Petersen, A. J. Demers, M. Spreitzer, and C. Hauser. Managing Update Conflicts in Bayou, a Weakly Connected Replicated Storage System. In Proc. of SOSP, Copper Mountain, CO, Dec. 1995.

Digital Library

[57]

R. van Renesse and F. B. Schneider. Chain Replication for Supporting High Throughput and Availability. In Proc. of OSDI, San Francisco, CA, Dec. 2004.

Digital Library

[58]

C. Zhang, A. Krishnamurthy, and R. Y. Wang. SkipIndex: Towards a Scalable Peer-to-Peer Index Service for High Dimensional Data. Princeton, Technical Report TR-703-04, 2004.

Cited By

Yu QGuo CZhuang JThakkar VWang JCao Z(2024)CaaS-LSM: Compaction-as-a-Service for LSM-based Key-Value Stores in Storage Disaggregated InfrastructureProceedings of the ACM on Management of Data10.1145/36549272:3(1-28)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3654927
Mao ZSrinivasan KKhandelwal A(2024)Trinity: A Fast Compressed Multi-attribute Data StoreProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3650072(405-420)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3627703.3650072
Miller dHassan APalmieri RAgrawal KPetrank E(2024)Brief Announcement: LIT: Lookup Interlocked Table for Range QueriesProceedings of the 36th ACM Symposium on Parallelism in Algorithms and Architectures10.1145/3626183.3660266(69-71)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3626183.3660266
Show More Cited By

Index Terms

HyperDex: a distributed, searchable key-value store
1. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems
    2. Software system structures
      1. Distributed systems organizing principles

Recommendations

KVell: the design and implementation of a fast persistent key-value store
SOSP '19: Proceedings of the 27th ACM Symposium on Operating Systems Principles

Modern block-addressable NVMe SSDs provide much higher bandwidth and similar performance for random and sequential access. Persistent key-value stores (KVs) designed for earlier storage devices, using either Log-Structured Merge (LSM) or B trees, do not ...
HyperDex: a distributed, searchable key-value store
Special october issue SIGCOMM '12

Distributed key-value stores are now a standard component of high-performance web services and cloud computing applications. While key-value stores offer significant performance and scalability advantages compared to traditional databases, they achieve ...
A Read-Optimized Index Structure for Distributed Log-Structured Key-Value Store
COMPSAC '15: Proceedings of the 2015 IEEE 39th Annual Computer Software and Applications Conference - Volume 03

Recently, Big Data processing is becoming a necessary technique to efficiently store, manage, and analyze massive data obtained by social media contents. NoSQL is one of databases that efficiently handle Big Data compared to the traditional database ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGCOMM '12: Proceedings of the ACM SIGCOMM 2012 conference on Applications, technologies, architectures, and protocols for computer communication

August 2012

474 pages

ISBN:9781450314190

DOI:10.1145/2342356

General Chairs:
Lars Eggert
NetApp, Germany
,
Jörg Ott
Aalto University, Finland
,
Program Chairs:
Venkat Padmanabhan
Microsoft Research, India
,
George Varghese
UCSD, USA

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 August 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGCOMM '12

Sponsor:

SIGCOMM

SIGCOMM '12: ACM SIGCOMM 2012 Conference

August 13 - 17, 2012

Helsinki, Finland

Acceptance Rates

Overall Acceptance Rate 462 of 3,389 submissions, 14%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

99
Total Citations
View Citations
1,791
Total Downloads

Downloads (Last 12 months)273
Downloads (Last 6 weeks)27

Reflects downloads up to 10 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yu QGuo CZhuang JThakkar VWang JCao Z(2024)CaaS-LSM: Compaction-as-a-Service for LSM-based Key-Value Stores in Storage Disaggregated InfrastructureProceedings of the ACM on Management of Data10.1145/36549272:3(1-28)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3654927
Mao ZSrinivasan KKhandelwal A(2024)Trinity: A Fast Compressed Multi-attribute Data StoreProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3650072(405-420)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3627703.3650072
Miller dHassan APalmieri RAgrawal KPetrank E(2024)Brief Announcement: LIT: Lookup Interlocked Table for Range QueriesProceedings of the 36th ACM Symposium on Parallelism in Algorithms and Architectures10.1145/3626183.3660266(69-71)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3626183.3660266
Duffy CShim JKim SKim J(2023)Dotori: A Key-Value SSD Based KV StoreProceedings of the VLDB Endowment10.14778/3583140.358316716:6(1560-1572)Online publication date: 1-Feb-2023
https://dl.acm.org/doi/10.14778/3583140.3583167
Guo ZZhang HZhao CBai YSwift MLiu MSchulzrinne HKohler EMaltz DMisra V(2023)LEED: A Low-Power, Fast Persistent Key-Value Store on SmartNIC JBOFsProceedings of the ACM SIGCOMM 2023 Conference10.1145/3603269.3604880(1012-1027)Online publication date: 10-Sep-2023
https://dl.acm.org/doi/10.1145/3603269.3604880
Wang ZJia ZZheng SZhang ZFu XNg TWang YDruschel PKaufmann AMace JFlinn JSeltzer M(2023)GEMINI: Fast Failure Recovery in Distributed Training with In-Memory CheckpointsProceedings of the 29th Symposium on Operating Systems Principles10.1145/3600006.3613145(364-381)Online publication date: 23-Oct-2023
https://dl.acm.org/doi/10.1145/3600006.3613145
Qi SWang JMiao MZhang MChen X(2023)TinyEnc: Enabling Compressed and Encrypted Big Data Stores With Rich Query SupportIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2021.312933220:1(176-192)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TDSC.2021.3129332
Yuan YHuang JSun YWang TNelson JPorts DWang YWang RTai CKim N(2023)Rambda: RDMA-driven Acceleration Framework for Memory-intensive µs-scale Datacenter Applications2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA)10.1109/HPCA56546.2023.10071127(499-515)Online publication date: Feb-2023
https://doi.org/10.1109/HPCA56546.2023.10071127
Singh BYe QHu HXiao B(2023)Efficient and lightweight indexing approach for multi-dimensional historical data in blockchainFuture Generation Computer Systems10.1016/j.future.2022.09.002139(210-223)Online publication date: Feb-2023
https://doi.org/10.1016/j.future.2022.09.002
Burckhardt SChandramouli BGillum CJusto DKallas KMcMahon CMeiklejohn CZhu X(2022)NetheriteProceedings of the VLDB Endowment10.14778/3529337.352934415:8(1591-1604)Online publication date: 22-Jun-2022
https://dl.acm.org/doi/10.14778/3529337.3529344
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents