abstract

Ludo Hashing: Compact, Fast, and Dynamic Key-value Lookups for Practical Network Systems

Authors:

SIGMETRICS '20: Abstracts of the 2020 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer Systems

Pages 57 - 58

https://doi.org/10.1145/3393691.3394198

Published: 08 June 2020 Publication History

Get Access

Abstract

Key-value lookup engines running in fast memory are crucial components of many networked and distributed systems such as packet forwarding, virtual network functions, content distribution networks, distributed storage, and cloud/edge computing. These lookup engines must be memory-efficient because fast memory is small and expensive. This work presents a new key-value lookup design, called Ludo Hashing, which costs the least space (3.76 + 1.05 ι bits per key-value item for ι-bit values) among known compact lookup solutions including the recently proposed partial-key Cuckoo and Bloomier perfect hashing. In addition to its space efficiency, Ludo Hashing works well with most practical systems by supporting fast lookups, fast updates, and concurrent writing/reading. We implement Ludo Hashing and evaluate it with both micro-benchmark and two network systems deployed in CloudLab. The results show that in practice Ludo Hashing saves 40% to 80%+ memory cost compared to existing dynamic solutions. It costs only a few GB memory for 1 billion key-value items and achieves high lookup throughput: over 65 million queries per second on a single node with multiple threads.

Supplementary Material

MP4 File (3393691.3394198.mp4)

Video for "Ludo Hashing: Compact, Fast, and Dynamic Key-value Lookups for Practical Network Systems" in SIGMETRICS 2020. ID No: sigmfpa24\r\n

Download
307.30 MB

References

[1]

Implementation of Ludo Hashing. https://github.com/QianLabUCSC/Ludo.

Google Scholar

[2]

Belazzougui, D., and Botelho, F. C. Hash, displace, and compress. In Proc. of Algorithms-ESA (2009).

Crossref

Google Scholar

[3]

Charles, D., and Chellapilla, K. Bloomier Filters: A Second Look. In Proc. of European Symposium on Algorithms (2008).

Digital Library

Google Scholar

[4]

Chazelle, B., Kilian, J., Rubinfeld, R., and Tal, A. The Bloomier Filter: An Efficient Data Structure for Static Support Lookup Tables. In Proc. of ACM SODA (2004), pp. 30--39.

Google Scholar

[5]

Esposito, E., Graf, T. M., and Vigna, S. Recsplit: Minimal perfect hashing via recursive splitting. Tech. rep., 2019.

Google Scholar

[6]

Fan, B., Andersen, D. G., Kaminsky, M., and Mitzenmacher, M. D. Cuckoo filter: Practically better than bloom. In Proceedings of the 10th ACM International on Conference on emerging Networking Experiments and Technologies (2014), ACM.

Digital Library

Google Scholar

[7]

Fan, B., Zhou, D., Lim, H., Kaminsky, M., and Andersen, D. G. When cycles are cheap, some tables can be huge. In Proc. of USENIX HotOS (2013).

Google Scholar

[8]

Lim, H., Fan, B., Andersen, D. G., and Kaminsky, M. SILT: A Memory-Efficient, High-Performance Key-Value Store. In Proc. of ACM SOSP (2011).

Digital Library

Google Scholar

[9]

Pagh, R., and Rodler, F. F. Cuckoo hashing. Journal of Algorithms (2004).

Google Scholar

[10]

Shi, S., and Qian, C. Ludo hashing: Compact, fast, and dynamic key-value lookups for practical network systems. Proceedings of the ACM on Measurement and Analysis of Computing Systems (POMACS) (2020).

Digital Library

Google Scholar

[11]

Shi, S., Qian, C., and Wang, M. Re-designing Compact-structure based Forwarding for Programmable Networks. In Proc. of IEEE ICNP (2019).

Crossref

Google Scholar

[12]

Wang, M., Zhou, M., Shi, S., and Qian, C. Vacuum filters: more space-efficient and faster replacement for bloom and cuckoo filters. Proceedings of the VLDB Endowment (2019).

Digital Library

Google Scholar

[13]

Yu, Y., Belazzougui, D., Qian, C., and Zhang, Q. Memory-efficient and Ultra-fast Network Lookup and Forwarding using Othello Hashing. IEEE/ACM Transactions on Networking (2018).

Google Scholar

[14]

Zhou, D., Fan, B., Lim, H., Andersen, D. G., Kaminsky, M., Mitzenmacher, M., Wang, R., and Singh, A. Scaling up clustered network appliances with scalebricks. In SIGCOMM (2015).

Google Scholar

Cited By

View all

Ashok SPartap ATahir A(2022)Fast and efficient lookups via data-driven FIB designsProceedings of the ACM SIGCOMM Workshop on Future of Internet Routing & Addressing10.1145/3527974.3545728(66-71)Online publication date: 22-Aug-2022
https://dl.acm.org/doi/10.1145/3527974.3545728
Shi XShi SWang MKaunisto JQian CKim YKim JVigna GShi E(2021)On-device IoT Certificate Revocation Checking with Small Memory and Low LatencyProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security10.1145/3460120.3484580(1118-1134)Online publication date: 12-Nov-2021
https://dl.acm.org/doi/10.1145/3460120.3484580
Qian CShi SShi XWang MZhao BZheng HMadhyastha HPadmanabhan V(2020)Don't Work on Individual Data Plane Algorithms. Put Them Together!Proceedings of the 19th ACM Workshop on Hot Topics in Networks10.1145/3422604.3425932(60-66)Online publication date: 4-Nov-2020
https://dl.acm.org/doi/10.1145/3422604.3425932
Show More Cited By

Index Terms

Ludo Hashing: Compact, Fast, and Dynamic Key-value Lookups for Practical Network Systems
1. Networks
  1. Network algorithms
    1. Data path algorithms
2. Theory of computation
  1. Design and analysis of algorithms
    1. Streaming, sublinear and near linear time algorithms
      1. Bloom filters and hashing

Recommendations

Ludo Hashing: Compact, Fast, and Dynamic Key-value Lookups for Practical Network Systems
SIGMETRICS

Key-value lookup engines running in fast memory are crucial components of many networked and distributed systems such as packet forwarding, virtual network functions, content distribution networks, distributed storage, and cloud/edge computing. These ...
Ludo Hashing: Compact, Fast, and Dynamic Key-value Lookups for Practical Network Systems

Ludo Hashing is a key-value lookup design for networked and distributed systems such as packet forwarding and distributed storage. Ludo costs the least space (3.76 + 1.05l bits per key-value item for l -bit values) among known compact lookup solutions ...
Smash: Flexible, Fast, and Resource-efficient Placement and Lookup of Distributed Storage
SIGMETRICS '23

Smash is a new placement and lookup method for distributed storage systems. It achieves full placement flexibility and low DRAM cost to store ID-to-location mappings, two desired features that could not be achieved simultaneously by any prior method.

Comments

Information & Contributors

Information

Published In

SIGMETRICS '20: Abstracts of the 2020 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer Systems

June 2020

124 pages

ISBN:9781450379854

DOI:10.1145/3393691

General Chair:
Edmund Yeh
Northeastern University, USA
,
Program Chairs:
Athina Markopoulou
University of California, Irvine, USA
,
Y.C. Tay
National University of Singapore, Singapore

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 June 2020

Check for updates

Author Tags

Qualifiers

Abstract

Funding Sources

National Science Foundation

Conference

SIGMETRICS '20

Sponsor:

SIGMETRICS

SIGMETRICS '20: ACM SIGMETRICS / International Conference on Measurement and Modeling of Computer Systems

June 8 - 12, 2020

MA, Boston, USA

Acceptance Rates

Overall Acceptance Rate 459 of 2,691 submissions, 17%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
115
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Ashok SPartap ATahir A(2022)Fast and efficient lookups via data-driven FIB designsProceedings of the ACM SIGCOMM Workshop on Future of Internet Routing & Addressing10.1145/3527974.3545728(66-71)Online publication date: 22-Aug-2022
https://dl.acm.org/doi/10.1145/3527974.3545728
Shi XShi SWang MKaunisto JQian CKim YKim JVigna GShi E(2021)On-device IoT Certificate Revocation Checking with Small Memory and Low LatencyProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security10.1145/3460120.3484580(1118-1134)Online publication date: 12-Nov-2021
https://dl.acm.org/doi/10.1145/3460120.3484580
Qian CShi SShi XWang MZhao BZheng HMadhyastha HPadmanabhan V(2020)Don't Work on Individual Data Plane Algorithms. Put Them Together!Proceedings of the 19th ACM Workshop on Hot Topics in Networks10.1145/3422604.3425932(60-66)Online publication date: 4-Nov-2020
https://dl.acm.org/doi/10.1145/3422604.3425932
Shi SQian C(2020)Ludo HashingACM SIGMETRICS Performance Evaluation Review10.1145/3410048.341008148:1(57-58)Online publication date: 9-Jul-2020
https://dl.acm.org/doi/10.1145/3410048.3410081
Shi SQian CYeh EMarkopoulou ATay Y(2020)Ludo HashingAbstracts of the 2020 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer Systems10.1145/3393691.3394198(57-58)Online publication date: 8-Jun-2020
https://dl.acm.org/doi/10.1145/3393691.3394198

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations