Article

Succinct Data Structures for Retrieval and Approximate Membership (Extended Abstract)

Authors:

Martin Dietzfelbinger,

Rasmus PaghAuthors Info & Claims

ICALP '08: Proceedings of the 35th international colloquium on Automata, Languages and Programming - Volume Part I

Pages 385 - 396

https://doi.org/10.1007/978-3-540-70575-8_32

Published: 07 July 2008 Publication History

Abstract

The retrieval problemis the problem of associatingdata with keys in a set. Formally, the data structure must store afunction $f\colon U\to \{0,1\}^r$ that has specified values on theelements of a given set S⊆ U, |S|= n, but may have any value on elements outsideS. All known methods (e. g. those based on perfect hashfunctions), induce a space overhead of θ(n)bits over the optimum, regardless of the evaluation time. We showthat for any k, query time O(k) can beachieved using space that is within a factor 1 + e^-kof optimal, asymptotically forlarge n. The time to construct the data structure isO(n), expected. If we allow logarithmicevaluation time, the additive overhead can be reduced toO(loglogn) bits whp. A general reductiontransfers the results on retrieval into analogous results onapproximate membership, a problem traditionally addressedusing Bloom filters. Thus we obtain space bounds arbitrarily closeto the lower bound for this problem as well. The evaluationprocedures of our data structures are extremely simple. For theresults stated above we assume free access to fully random hashfunctions. This assumption can be justified using spaceo(n) to simulate full randomness on a RAM.

References

[1]

Alstrup, S., Brodal, G.S., Rauhe, T.: Optimal static range reporting in one dimension. In: Proc. 33rd ACM STOC, pp. 476-482 (2001)

Digital Library

[2]

Bloom, B.H.: Space/time trade-offs in hash coding with allowable errors. Commun. ACM 13(7), 422-426 (1970)

Digital Library

[3]

Botelho, F.C., Pagh, R., Ziviani, N.: Simple and space-efficient minimal perfect hash functions. In: Dehne, F., Sack, J.-R., Zeh, N. (eds.) WADS 2007. LNCS, vol. 4619, pp. 139-150. Springer, Heidelberg (2007)

Digital Library

[4]

Broder, A.Z., Mitzenmacher, M.: Network applications of Bloom filters: A survey. In: Proc. 40th Annual Allerton Conference on Communication, Control, and Computing, pp. 636-646. ACM Press, New York (2002)

[5]

Cain, J.A., Sanders, P., Wormald, N.C.: The random graph threshold for k- orientiability and a fast algorithm for optimal multiple-choice allocation. In: Proc. 18th ACM-SIAM SODA, pp. 469-476 (2007)

Digital Library

[6]

Calkin, N.J.: Dependent sets of constant weight binary vectors. Combinatorics, Probability and Computing 6(3), 263-271 (1997)

Digital Library

[7]

Carter, L., Floyd, R.W., Gill, J., Markowsky, G., Wegman, M.N.: Exact and approximate membership testers. In: Proc. 10th ACM STOC, pp. 59-65 (1978)

Digital Library

[8]

Chazelle, B., Kilian, J., Rubinfeld, R., Tal, A.: The Bloomier filter: an efficient data structure for static support lookup tables. In: Proc. 15th ACM-SIAM SODA, pp. 30-39 (2004)

Digital Library

[9]

Cooper, C.: On the rank of random matrices. Random Struct. Algorithms 16(2), 209-232 (2001)

Digital Library

[10]

Czumaj, A., Riley, C., Scheideler, C.: Perfectly Balanced Allocation. In: Arora, S., Jansen, K., Rolim, J.D.P., Sahai, A. (eds.) RANDOM 2003 and APPROX 2003. LNCS, vol. 2764, pp. 240-251. Springer, Heidelberg (2003)

[11]

Dietzfelbinger, M.: Design strategies for minimal perfect hash functions. In: Proc. 4th Int. Symp. on Stochastic Algorithms: Foundations and Applications (SAGA). LNCS, vol. 4665, pp. 2-17. Springer, Heidelberg (2007)

[12]

Dietzfelbinger, M., Pagh, R.: Succinct data structures for retrieval and approximate membership, Technical Report, arXiv:0803.3693v1 [cs.DS] (March 26, 2008)

[13]

Dietzfelbinger, M., Weidling, C.: Balanced allocation and dictionaries with tightly packed constant size bins. Theoret. Comput. Sci. 380(1-2), 47-68 (2007)

Digital Library

[14]

Fernholz, D., Ramachandran, V.: The k-orientability thresholds for G_n,p. In: Proc. 18th ACM-SIAM SODA, pp. 459-468 (2007)

Digital Library

[15]

Fotakis, D., Pagh, R., Sanders, P., Spirakis, P.G.: Space efficient hash tables with worst case constant access time. Theory Comput. Syst. 38(2), 229-248 (2005)

[16]

Hagerup, T., Tholey, T.: Efficient minimal perfect hashing in nearly minimal space. In: Ferreira, A., Reichel, H. (eds.) STACS 2001. LNCS, vol. 2010, pp. 317-326. Springer, Heidelberg (2001)

Digital Library

[17]

Majewski, B.S., Wormald, N.C., Havas, G., Czech, Z.J.: A family of perfect hashing methods. Computer J. 39(6), 547-554 (1996)

[18]

Mitzenmacher, M.: Compressed Bloom filters. IEEE/ACM Transactions on Networking 10(5), 604-612 (2002)

Digital Library

[19]

Mortensen, C.W., Pagh, R., Pătraşcu, M.: On dynamic range reporting in one dimension. In: Proc. 37th ACM STOC, pp. 104-111 (2005)

Digital Library

[20]

Pagh, R., Rodler, F.F.: Cuckoo Hashing. J. Algorithms 51, 122-144 (2004)

Digital Library

[21]

Panigrahy, R.: Efficient hashing with lookups in two memory accesses. In: Proc. 16th ACM-SIAM SODA, pp. 830-839 (2005)

Digital Library

[22]

Porat, E.: An optimal Bloom filter replacement based on matrix solving, Technical Report, arXiv:0804.1845v1 [cs.DS] (April 11, 2008)

[23]

Seiden, S.S., Hirschberg, D.S.: Finding succinct ordered minimal perfect hash functions. Inf. Process. Lett. 51(6), 283-288 (1994)

Digital Library

[24]

Zukowski, M., Heman, S., Boncz, P.A.: Architecture-conscious hashing. In: Proc. Int. Workshop on Data Management on New Hardware (DaMoN), Chicago, 8 pages, Article No. 6 (2006)

Digital Library

Cited By

Kuszmaul WWalzer SMohar BShinkar IO'Donnell R(2024)Space Lower Bounds for Dynamic Filters and Value-Dynamic RetrievalProceedings of the 56th Annual ACM Symposium on Theory of Computing10.1145/3618260.3649649(1153-1164)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3618260.3649649
Blass ENoubir G(2023)Forward Security with Crash Recovery for Secure LogsACM Transactions on Privacy and Security10.1145/363152427:1(1-28)Online publication date: 3-Nov-2023
https://dl.acm.org/doi/10.1145/3631524
Wang JSun SLi TQi SChen XYin HStavrou ACremers CShi E(2022)Practical Volume-Hiding Encrypted Multi-Maps with Optimal Overhead and BeyondProceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security10.1145/3548606.3559345(2825-2839)Online publication date: 7-Nov-2022
https://dl.acm.org/doi/10.1145/3548606.3559345
Show More Cited By

Index Terms

Succinct Data Structures for Retrieval and Approximate Membership (Extended Abstract)

Index terms have been assigned to the content through auto-classification.

Recommendations

Squeezing succinct data structures into entropy bounds
SODA '06: Proceedings of the seventeenth annual ACM-SIAM symposium on Discrete algorithm

Consider a sequence S of n symbols drawn from an alphabet A = {1, 2,. . .,σ}, stored as a binary string of nlog σ bits. A succinct data structure on S supports a given set of primitive operations on S using just f (n) = o(n log σ) extra bits. We present ...
Statistical encoding of succinct data structures
CPM'06: Proceedings of the 17th Annual conference on Combinatorial Pattern Matching

In recent work, Sadakane and Grossi [SODA 2006] introduced a scheme to represent any sequence S=s₁s₂...s_n, over an alphabet of size σ, using $nH_k(S)+O(\frac{n}{\log_\sigma n} (k \log \sigma + \log\log n))$ bits of space, where H_k(S) is the k-th order ...
Bit-probe lower bounds for succinct data structures
STOC '09: Proceedings of the forty-first annual ACM symposium on Theory of computing

We prove lower bounds on the redundancy necessary to represent a set S of objects using a number of bits close to the information-theoretic minimum log₂ |S|, while answering various queries by probing few bits. Our main results are: To represent n ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

ICALP '08: Proceedings of the 35th international colloquium on Automata, Languages and Programming - Volume Part I

July 2008

892 pages

ISBN:3540705740

Editors:
Luca Aceto
School of Computer Science, Reykjavik University, Reykjavík, Iceland 103
,
Ivan Damgård
Department of Computer Science, IT-Parken, University of Aarhus, ÅrhusN, Denmark 8200
,
Leslie Ann Goldberg
Department of Computer Science, University of Liverpool, Liverpool, UK L69 3BX
,
Magnús M. Halldórsson
School of Computer Science, Reykjavik University, Reykjavik, Iceland 103
,
Anna Ingólfsdóttir
Department of Computer Science, Reykjavik University, Reykjavík, Iceland 103
,
Igor Walukiewicz
Université de Bordeaux-1, LaBRI, Talence cedex, France 33405

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 07 July 2008

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kuszmaul WWalzer SMohar BShinkar IO'Donnell R(2024)Space Lower Bounds for Dynamic Filters and Value-Dynamic RetrievalProceedings of the 56th Annual ACM Symposium on Theory of Computing10.1145/3618260.3649649(1153-1164)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3618260.3649649
Blass ENoubir G(2023)Forward Security with Crash Recovery for Secure LogsACM Transactions on Privacy and Security10.1145/363152427:1(1-28)Online publication date: 3-Nov-2023
https://dl.acm.org/doi/10.1145/3631524
Wang JSun SLi TQi SChen XYin HStavrou ACremers CShi E(2022)Practical Volume-Hiding Encrypted Multi-Maps with Optimal Overhead and BeyondProceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security10.1145/3548606.3559345(2825-2839)Online publication date: 7-Nov-2022
https://dl.acm.org/doi/10.1145/3548606.3559345
Graf TLemire D(2022)Binary Fuse Filters: Fast and Smaller Than Xor FiltersACM Journal of Experimental Algorithmics10.1145/351044927(1-15)Online publication date: 4-Mar-2022
https://dl.acm.org/doi/10.1145/3510449
Walzer SMarx D(2021)Peeling close to the orientability thresholdProceedings of the Thirty-Second Annual ACM-SIAM Symposium on Discrete Algorithms10.5555/3458064.3458195(2194-2211)Online publication date: 10-Jan-2021
https://dl.acm.org/doi/10.5555/3458064.3458195
Schmidt TBandle MGiceva J(2021)A four-dimensional analysis of partitioned approximate filtersProceedings of the VLDB Endowment10.14778/3476249.347628614:11(2355-2368)Online publication date: 27-Oct-2021
https://dl.acm.org/doi/10.14778/3476249.3476286
Kipf AChromejko DHall ABoncz PAndersen D(2020)Cuckoo indexProceedings of the VLDB Endowment10.14778/3424573.342457713:13(3559-3572)Online publication date: 27-Oct-2020
https://dl.acm.org/doi/10.14778/3424573.3424577
Duggal AJenkins FShilane PChinthekindi RShah RKamat MDan TDahlia M(2019)Data domain cloud tierProceedings of the 2019 USENIX Conference on Usenix Annual Technical Conference10.5555/3358807.3358862(647-660)Online publication date: 10-Jul-2019
https://dl.acm.org/doi/10.5555/3358807.3358862
Clayton DPatton CShrimpton TCavallaro LKinder JWang XKatz J(2019)Probabilistic Data Structures in Adversarial EnvironmentsProceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security10.1145/3319535.3354235(1317-1334)Online publication date: 6-Nov-2019
https://dl.acm.org/doi/10.1145/3319535.3354235
Naor MEylon Y(2019)Bloom Filters in Adversarial EnvironmentsACM Transactions on Algorithms10.1145/330619315:3(1-30)Online publication date: 7-Jun-2019
https://dl.acm.org/doi/10.1145/3306193
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents