research-article

Constructions and Applications for Accurate Counting of the Bloom Filter False Positive Free Zone

Authors:

Ori Rottenstreich,

Pedro Reviriego,

S. MuthukrishnanAuthors Info & Claims

SOSR '20: Proceedings of the Symposium on SDN Research

Pages 135 - 145

https://doi.org/10.1145/3373360.3380845

Published: 04 March 2020 Publication History

Abstract

Bloom filters are used in many networking applications to answer set membership queries at low cost but suffer from false positives. We study Bloom filter constructions that when representing a set of size up to d taken from a finite universe of size n, completely avoid false positives. We suggest memory-efficient Bloom filters constructions with a false positive free zone to allow representations of larger sets through linear memory dependency in the set size. Our first construction relies on Orthogonal Latin Square (OLS) codes and the second relies on the representation of elements through values of polynomials defined modulo primes. Beyond Bloom filters supporting set membership, we also consider sketches allowing a more general functionality such as flow size estimation. In particular, we show the applicability of the false positive free zone for accurate size estimation in the famous Count-Min sketch. We compare the new constructions to existing approaches through analytical and experimental evaluations for showing their superiority.

References

[1]

Zainab Abaid, Mohamed Ali Kâafar, and Sanjay Jha. 2017. Early detection of in-the-wild botnet attacks by exploiting network communication uniformity: An empirical study. In IFIP Networking.

[2]

Omid Alipourfard, Masoud Moshref, Yang Zhou, Tong Yang, and Minlan Yu. 2018. A Comparison of Performance and Accuracy of Measurement Algorithms in Software. In ACM SOSR.

[3]

B. Bloom. 1970. Space/Time Tradeoffs in Hash Coding with Allowable Errors. Commun. ACM 13, 7 (1970).

[4]

Flavio Bonomi, Michael Mitzenmacher, Rina Panigrahy, Sushil Singh, and George Varghese. 2006. An Improved Construction for Counting Bloom Filters. In ESA.

[5]

Pat Bosshart, Dan Daly, Glen Gibb, Martin Izzard, Nick McKeown, Jennifer Rexford, Cole Schlesinger, Dan Talayco, Amin Vahdat, George Varghese, and David Walker. 2014. P4: Programming Protocol-independent Packet Processors. ACM SIGCOMM Comput. Commun. Rev. 44, 3 (2014), 87--95.

Digital Library

[6]

Andrei Z. Broder and Michael Mitzenmacher. 2003. Survey: Network Applications of Bloom Filters: A Survey. Internet Mathematics 1, 4 (2003), 485--509.

[7]

Tao Chen, Deke Guo, Yuan He, Honghui Chen, Xue Liu, and Xueshan Luo. 2013. A Bloom filters based dissemination protocol in wireless sensor networks. Ad Hoc Networks 11, 4 (2013), 1359--1371.

Digital Library

[8]

Graham Cormode and S. Muthukrishnan. 2005. An improved data stream summary: The Count-Min sketch and its applications. J. Algorithms 55, 1 (2005), 58--75.

Digital Library

[9]

Haipeng Dai, Yuankun Zhong, Alex X. Liu, Wei Wang, and Meng Li. 2016. Noisy Bloom Filters for Multi-Set Membership Testing. In ACM SIGMETRICS.

[10]

Niv Dayan, Manos Athanassoulis, and Stratos Idreos. 2017. Monkey: Optimal Navigable Key-Value Store. In ACM SIGMOD.

Digital Library

[11]

Sarang Dharmapurikar, Praveen Krishnamurthy, and David E. Taylor. 2006. Longest prefix matching using Bloom filters. IEEE/ACM Trans. Netw. 14, 2 (2006), 397--409.

Digital Library

[12]

Benoit Donnet, Bruno Baynat, and Timur Friedman. 2006. Retouched Bloom filters: Allowing networked applications to trade off selected false positives against false negatives. In ACM CoNEXT.

[13]

Benoit Donnet, Bamba Gueye, and Mohamed Ali Kaafar. 2012. Path similarity evaluation using Bloom filters. Computer Networks 56, 2 (2012), 858--869.

Digital Library

[14]

David Eppstein, Michael T. Goodrich, and Daniel S. Hirschberg. 2007. Improved Combinatorial Group Testing Algorithms for Real-World Problem Sizes. SIAM J. Comput. 36, 5 (2007), 1360--1375.

Digital Library

[15]

Bin Fan, Dave G. Andersen, Michael Kaminsky, and Michael D. Mitzenmacher. 2014. Cuckoo filter: Practically better than Bloom. In ACM CoNEXT.

Digital Library

[16]

Richard W Hamming. 1950. Error detecting and error correcting codes. Bell System technical journal 29, 2 (1950), 147--160.

[17]

Maurice Herlihy, Nir Shavit, and Moran Tzafrir. 2008. Hopscotch Hashing. In International Symposium Distributed Computing (DISC).

[18]

J. Hill, M. Aloserij, and P. Grosso. 2018. Tracking Network Flows with P4. In IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS).

[19]

MY Hsiao, DC Bossen, and RT Chien. 1970. Orthogonal Latin square codes. IBM Journal of Research and Development 14, 4 (1970), 390--394.

Digital Library

[20]

A Donald Keedwell and József Dénes. 2015. Latin squares and their applications. Elsevier.

[21]

Sándor Z. Kiss, Éva Hosszu, János Tapolcai, Lajos Rónyai, and Ori Rottenstreich. 2018. Bloom Filter with a False Positive Free Zone. In IEEE INFOCOM.

[22]

Donald Knuth. 1973. The Art of Computer Programming. Part 3, Sorting and Searching. Addison-Wesley.

[23]

Rafael P. Laufer, Pedro B. Velloso, and Otto Carlos Muniz Bandeira Duarte. 2011. A Generalized Bloom Filter to Secure Distributed Network Applications. Computer Networks 55, 8 (2011), 1804--1819.

Digital Library

[24]

Yuliang Li, Rui Miao, Changhoon Kim, and Minlan Yu. 2016. FlowRadar: A Better NetFlow for Data Centers. In USENIX NSDI.

[25]

Shu Lin and Daniel J Costello. 2004. Error Control Coding. 2nd Edition. Englewood Cliffs, NJ: Prentice-Hall (2004).

Digital Library

[26]

Zaoxing Liu, Antonis Manousis, Gregory Vorsanger, Vyas Sekar, and Vladimir Braverman. 2016. One Sketch to Rule Them All: Rethinking Network Flow Monitoring with UnivMon. In ACM SIGCOMM.

[27]

Lailong Luo, Deke Guo, Richard T. B. Ma, Ori Rottenstreich, and Xueshan Luo. 2019. Optimizing Bloom Filter: Challenges, Solutions, and Comparisons. IEEE Communications Surveys and Tutorials 21, 2 (2019), 1912--1949.

[28]

Lailong Luo, Deke Guo, Jie Wu, Ori Rottenstreich, Qian He, Yudong Qin, and Xueshan Luo. 2017. Efficient Multiset Synchronization. IEEE/ACM Trans. Netw. 25, 2 (2017), 1190--1205.

Digital Library

[29]

Robert MacDavid, Rüdiger Birkner, Ori Rottenstreich, Arpit Gupta, Nick Feamster, and Jennifer Rexford. 2017. Concise Encoding of Flow Attributes in SDN Switches. In ACM SOSR.

[30]

Michael D. Mitzenmacher, Salvatore Pontarelli, and Pedro Reviriego. 2018. Adaptive Cuckoo Filters. In Workshop on Algorithm Engineering and Experiments (ALENEX).

[31]

Rasmus Pagh and Flemming Friche Rodler. 2004. Cuckoo hashing. J. Algorithms 51, 2 (2004), 122--144.

Digital Library

[32]

F. Pereira, N. Neves, and F. M. V. Ramos. 2017. Secure network monitoring using programmable data planes. In IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN).

[33]

Salvatore Pontarelli, Pedro Reviriego, and Juan Antonio Maestro. 2016. Improving Counting Bloom Filter Performance with Fingerprints. Inform. Process. Lett. 116, 4 (2016), 304--309.

Digital Library

[34]

Jaikumar Radhakrishnan, Smit Shah, and Saswata Shannigrahi. 2010. Data structures for storing small sets in the bitprobe model. In Springer ESA.

[35]

Pedro Reviriego, Salvatore Pontarelli, Alfonso Sánchez-Macián, and Juan Antonio Maestro. 2014. A Method to Extend Orthogonal Latin Square Codes. IEEE Trans. VLSI Syst. 22, 7 (2014), 1635--1639.

[36]

Ori Rottenstreich, Yossi Kanizo, and Isaac Keslassy. 2014. The Variable-Increment Counting Bloom Filter. IEEE/ACM Trans. Netw. 22, 4 (2014), 1092--1105.

Digital Library

[37]

Mikko Särelä, Christian Esteve Rothenberg, Tuomas Aura, András Zahemszky, Pekka Nikander, and Jörg Ott. 2011. Forwarding anomalies in Bloom filter-based multicast. In IEEE INFOCOM.

[38]

Vibhaalakshmi Sivaraman, Srinivas Narayana, Ori Rottenstreich, S. Muthukrishnan, and Jennifer Rexford. 2017. Heavy-Hitter Detection Entirely in the Data Plane. In ACM SOSR.

[39]

Sasu Tarkoma, Christian Esteve Rothenberg, and Eemil Lagerspetz. 2012. Theory and practice of Bloom filters for distributed systems. IEEE Communications Surveys & Tutorials 14, 1 (2012), 131--155.

[40]

Jeffrey Scott Vitter. 1982. Implementations for Coalesced Hashing. Commun. ACM 25, 12 (1982), 911--926.

Digital Library

[41]

Tong Yang, Jie Jiang, Peng Liu, Qun Huang, Junzhi Gong, Yang Zhou, Rui Miao, Xiaoming Li, and Steve Uhlig. 2018. Elastic sketch: Adaptive and fast network-wide measurements. In ACM SIGCOMM.

Cited By

Yi WWang CXie QZhao YJia J(2023)PSBF: p-adic Integer Scalable Bloom FilterSensors10.3390/s2318777523:18(7775)Online publication date: 9-Sep-2023
https://doi.org/10.3390/s23187775
Kasu PHamandawana PChung T(2023)CRBF: Cross-Referencing Bloom-Filter-Based Data Integrity Verification Framework for Object-Based Big Data Transfer SystemsApplied Sciences10.3390/app1313783013:13(7830)Online publication date: 3-Jul-2023
https://doi.org/10.3390/app13137830
Alsuhibany SAlsuhaibani MKhan RQamar A(2022)Performance Analysis of Bloom Filter for Big Data AnalyticsComputational Intelligence and Neuroscience10.1155/2022/24146052022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/2414605
Show More Cited By

Index Terms

Constructions and Applications for Accurate Counting of the Bloom Filter False Positive Free Zone
1. Networks
  1. Network algorithms

Recommendations

False Negative Problem of Counting Bloom Filter

Bloom filter is effective, space-efficient data structure for concisely representing a data set and supporting approximate membership queries. Traditionally, researchers often believe that it is possible that a Bloom filter returns a false positive, but ...
Improving retouched Bloom filter for trading off selected false positives against false negatives

Where distributed agents must share voluminous set membership information, Bloom filters provide a compact, though lossy, way for them to do so. Numerous recent networking papers have examined the trade-offs between the bandwidth consumed by the ...
Bloom Filter with a False Positive Free Zone
IEEE INFOCOM 2018 - IEEE Conference on Computer Communications
Bloom filters and their variants are widely used as space efficient probabilistic data structures for representing set systems and are very popular in networking applications. They support fast element insertion and deletion, along with membership queries ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SOSR '20: Proceedings of the Symposium on SDN Research

March 2020

151 pages

ISBN:9781450371018

DOI:10.1145/3373360

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 March 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SOSR '20

Sponsor:

SIGCOMM

SOSR '20: Symposium on SDN Research

March 3, 2020

CA, San Jose, USA

Acceptance Rates

Overall Acceptance Rate 7 of 43 submissions, 16%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
357
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)2

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yi WWang CXie QZhao YJia J(2023)PSBF: p-adic Integer Scalable Bloom FilterSensors10.3390/s2318777523:18(7775)Online publication date: 9-Sep-2023
https://doi.org/10.3390/s23187775
Kasu PHamandawana PChung T(2023)CRBF: Cross-Referencing Bloom-Filter-Based Data Integrity Verification Framework for Object-Based Big Data Transfer SystemsApplied Sciences10.3390/app1313783013:13(7830)Online publication date: 3-Jul-2023
https://doi.org/10.3390/app13137830
Alsuhibany SAlsuhaibani MKhan RQamar A(2022)Performance Analysis of Bloom Filter for Big Data AnalyticsComputational Intelligence and Neuroscience10.1155/2022/24146052022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/2414605
Vairam PKumar PRebeiro CKamakoti V(2022)FadingBF: A Bloom Filter With Consistent Guarantees for Online ApplicationsIEEE Transactions on Computers10.1109/TC.2020.303642471:1(40-52)Online publication date: 1-Jan-2022
https://doi.org/10.1109/TC.2020.3036424
Patgiri RBorah M(2022)HEX-BLOOM: An Efficient Method for Authenticity and Integrity Verification in Privacy-preserving Computing2022 IEEE International Performance, Computing, and Communications Conference (IPCCC)10.1109/IPCCC55026.2022.9894352(397-403)Online publication date: 11-Nov-2022
https://doi.org/10.1109/IPCCC55026.2022.9894352
Reviriego PPontarelli SCarle GOtt J(2021)Perfect cuckoo filtersProceedings of the 17th International Conference on emerging Networking EXperiments and Technologies10.1145/3485983.3494852(205-211)Online publication date: 2-Dec-2021
https://dl.acm.org/doi/10.1145/3485983.3494852
Rottenstreich OReviriego PPorat EMuthukrishnan S(2021)Avoiding Flow Size Overestimation in Count-Min Sketch With Bloom Filter ConstructionsIEEE Transactions on Network and Service Management10.1109/TNSM.2021.306860418:3(3662-3676)Online publication date: Sep-2021
https://doi.org/10.1109/TNSM.2021.3068604
Saha SSiva Kumar D(2021)Enhancement of Precision on Cloud Data using Multi-Layered Bloom Filter2021 IEEE 18th India Council International Conference (INDICON)10.1109/INDICON52576.2021.9691668(1-7)Online publication date: 19-Dec-2021
https://doi.org/10.1109/INDICON52576.2021.9691668
Rottenstreich O(2021)Sketches for Blockchains2021 International Conference on COMmunication Systems & NETworkS (COMSNETS)10.1109/COMSNETS51098.2021.9352944(254-262)Online publication date: 5-Jan-2021
https://doi.org/10.1109/COMSNETS51098.2021.9352944
Wang RDu HShen ZJia Z(2021)DAP-Sketch: An accurate and effective network measurement sketch with Deterministic Admission PolicyComputer Networks10.1016/j.comnet.2021.108155194(108155)Online publication date: Jul-2021
https://doi.org/10.1016/j.comnet.2021.108155
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten