research-article

Counter braids: a novel counter architecture for per-flow measurement

Authors:

Yi Lu,

Andrea Montanari,

Balaji Prabhakar,

Sarang Dharmapurikar,

Abdul KabbaniAuthors Info & Claims

SIGMETRICS '08: Proceedings of the 2008 ACM SIGMETRICS international conference on Measurement and modeling of computer systems

Pages 121 - 132

https://doi.org/10.1145/1375457.1375472

Published: 02 June 2008 Publication History

Get Access

Abstract

Fine-grained network measurement requires routers and switches to update large arrays of counters at very high link speed (e.g. 40 Gbps). A naive algorithm needs an infeasible amount of SRAM to store both the counters and a flow-to-counter association rule, so that arriving packets can update corresponding counters at link speed. This has made accurate per-flow measurement complex and expensive, and motivated approximate methods that detect and measure only the large flows.

This paper revisits the problem of accurate per-flow measurement. We present a counter architecture, called Counter Braids, inspired by sparse random graph codes. In a nutshell, Counter Braids "compresses while counting". It solves the central problems (counter space and flow-to-counter association) of per-flow measurement by "braiding" a hierarchy of counters with random graphs. Braiding results in drastic space reduction by sharing counters among flows; and using random graphs generated on-the-fly with hash functions avoids the storage of flow-to-counter association.

The Counter Braids architecture is optimal (albeit with a complex decoder) as it achieves the maximum compression rate asymptotically. For implementation, we present a low-complexity message passing decoding algorithm, which can recover flow sizes with essentially zero error. Evaluation on Internet traces demonstrates that almost all flow sizes are recovered exactly with only a few bits of counter space per flow.

References

[1]

http://www.cisco.com/warp/public/732/Tech/netflow.

Google Scholar

[2]

Juniper networks solutions for network accounting. www.juniper.net/techcenter/appnote/350003.html.

Google Scholar

[3]

B. Bloom. Space/time trade-offs in hash coding with allowable errors. Comm. ACM, 13, July 1970.

Digital Library

Google Scholar

[4]

J. W. Byers, M. Luby, M. Mitzenmacher, and A. Rege. A digital fountain approach to reliable distribution of bulk data. In SIGCOMM, pages 56--67, 1998.

Digital Library

Google Scholar

[5]

G. Caire, S. Shamai, and S. Verdú. Noiseless data compression with low density parity check codes. In DIMACS, New York, 2004.

Crossref

Google Scholar

[6]

E. Candès and T. Tao. Near optimal signal recovery from random projections and universal encoding strategies. IEEE Trans. Inform. Theory, 2004.

Google Scholar

[7]

G. Cormode and S. Muthukrishnan. An improved data stream summary: the count-min sketch and its applications. Journal of Algorithms, 55(1), April 2005.

Digital Library

Google Scholar

[8]

T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley, New York, 1991.

Digital Library

Google Scholar

[9]

M. Crovella and A. Bestavros. Self-similarity in world wide web traffic: Evidence and possible causes. IEEE/ACM Trans. Networking, 1997.

Digital Library

Google Scholar

[10]

S. Dharmapurikar and V. Paxson. Robust tcp stream reassembly in the presence of adversaries. 14th USENIX Security Symposium, 2005.

Digital Library

Google Scholar

[11]

D. Donoho. Compressed sensing. IEEE Trans. Inform. Theory, 52(4), April 2006.

Digital Library

Google Scholar

[12]

C. Estan and G. Varghese. New directions in traffic measurement and accounting. Proc. ACM SIGCOMM Internet Measurement Workshop, pages 75--80, 2001.

Digital Library

Google Scholar

[13]

R. G. Gallager. Low-Density Parity-Check Codes. MIT Press, Cambridge, Massachussetts.

Google Scholar

[14]

M. Grossglauser and J. Rexford. Passive traffic measurement for ip operations. The Internet as a Large-Scale Complex System, 2002.

Google Scholar

[15]

F. Kschischang, B. Frey, and H.-A. Loeliger. Factor graphs and the sum-product algorithm. IEEE Trans. Inform. Theory, 47:498--519, 2001.

Digital Library

Google Scholar

[16]

A. Kumar, M. Sung, J. J. Xu, and J. Wang. Data streaming algorithms for efficient and accurate estimation of flow size distribution. Proceedings of ACM SIGMETRICS, 2004.

Digital Library

Google Scholar

[17]

Y. Lu, A. Montanari, and B. Prabhakar. Detailed network measurements using sparse graph counters: The theory. Proceedings of the Allerton Conference, September 2007.

Google Scholar

[18]

M. Luby, M. Mitzenmacher, A. Shokrollahi, D. A. Spielman, and V. Stemann. Practical loss-resilient codes. In Proceedings of the 29th annual ACM Symposium on Theory of Computing, pages 150--159, 1997.

Digital Library

Google Scholar

[19]

M. Mézard and A. Montanari. Constraint satisfaction networks in Physics and Computation. In Preparation.

Google Scholar

[20]

S. Ramabhadran and G. Varghese. Efficient implementation of a statistics counter architecture. Proc. ACM SIGMETRICS, pages 261--271, 2003.

Digital Library

Google Scholar

[21]

T. Richardson and R. Urbanke. Modern Coding Theory. Cambridge University Press, 2007.

Digital Library

Google Scholar

[22]

D. Shah, S. Iyer, B. Prabhakar, and N. McKeown. Analysis of a statistics counter architecture. Proc. IEEE HotI 9.

Digital Library

Google Scholar

[23]

Q. G. Zhao, J. J. Xu, and Z. Liu. Design of a novel statistics counter architecture with optimal space and time efficiency. SIGMetrics/Performance, June 2006.

Digital Library

Google Scholar

Cited By

View all

Xiao QCai XQin YTang ZChen SLiu Y(2023)Universal and Accurate Sketch for Estimating Heavy Hitters and Moments in Data StreamsIEEE/ACM Transactions on Networking10.1109/TNET.2022.321602531:5(1919-1934)Online publication date: Oct-2023
https://doi.org/10.1109/TNET.2022.3216025
Wang HLin HZhong ZYang TShahzad M(2023)Enhanced Machine Learning Sketches for Network MeasurementsIEEE Transactions on Computers10.1109/TC.2022.318556072:4(957-970)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TC.2022.3185560
Li HChen QZhang YYang TCui B(2022)Stingy sketchProceedings of the VLDB Endowment10.14778/3523210.352322015:7(1426-1438)Online publication date: 22-Jun-2022
https://dl.acm.org/doi/10.14778/3523210.3523220
Show More Cited By

Recommendations

Counter braids: a novel counter architecture for per-flow measurement
SIGMETRICS '08

Fine-grained network measurement requires routers and switches to update large arrays of counters at very high link speed (e.g. 40 Gbps). A naive algorithm needs an infeasible amount of SRAM to store both the counters and a flow-to-counter association ...
Performance evaluation of Westwood+ TCP congestion control
Internet performance symposium (IPS 2002)

Westwood+ TCP is a sender-side only modification of the classic Tahoe/Reno TCP that has been recently proposed to improve fairness and efficiency of TCP. The key idea of Westwood+ TCP is to perform an end-to-end estimate of the bandwidth available for a ...
On evaluating the differences of TCP and ICMP in network measurement

Network measurement is an important approach to understand network behaviors, which has been widely studied. Both TCP and ICMP are applied in network measurement, while investigating the differences between the measured results of these two protocols is ...

Reviews

Reviewer: Jeff Ellis Smith

Counter braids (CBs) incrementally compress network flow counts, "braiding" a hierarchy of counters with random graphs, allowing exact, and zero error, counting with high link rates in embedded router architectures. The result is a technique for fast, small footprint, fine-grained, accurate per-flow network measurement. As a motivation, related previous exact/approximate counting, compressed sensing, and sparse random graph code research is objectively compared, describing the measurement error, algorithm speed, and hardware cost compromises of previous attempts. The CB algorithm is first presented and analyzed as a one-and two-layer hierarchy, with easy-to-follow diagrams and pseudocode describing the random hash mappings and the update algorithm at the heart of the procedure. A multi-layer generalization and design is then described. An evaluation is given, using randomly generated and real Internet traces, comparing one- and two-layer CBs, as well as trace simulation of an OC-48 link through a simulated router. The paper concludes with implementation notes for on-chip updates, and predicts the computation cost of a decoding CB. The authors have accomplished their objective of describing, analyzing, and simulating an efficient, minimum space architecture for high-speed network flow counting, required for accurate billing, data collection, and network design. The paper is targeted to an audience interested in the improved network measurement techniques required by burgeoning larger-scale networks. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

SIGMETRICS '08: Proceedings of the 2008 ACM SIGMETRICS international conference on Measurement and modeling of computer systems

June 2008

486 pages

ISBN:9781605580050

DOI:10.1145/1375457

General Chair:
Zhen Liu
IBM T. J. Watson Research Center, USA
,
Program Chairs:
Vishal Misra
Columbia University, USA
,
Prashant Shenoy
University of Massachusetts, USA

ACM SIGMETRICS Performance Evaluation Review Volume 36, Issue 1
SIGMETRICS '08
June 2008
469 pages
ISSN:0163-5999
DOI:10.1145/1384529
Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 June 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGMETRICS08

Sponsor:

SIGMETRICS08: ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems

June 2 - 6, 2008

MD, Annapolis, USA

Acceptance Rates

Overall Acceptance Rate 459 of 2,691 submissions, 17%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

225
Total Citations
View Citations
908
Total Downloads

Downloads (Last 12 months)40
Downloads (Last 6 weeks)3

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Xiao QCai XQin YTang ZChen SLiu Y(2023)Universal and Accurate Sketch for Estimating Heavy Hitters and Moments in Data StreamsIEEE/ACM Transactions on Networking10.1109/TNET.2022.321602531:5(1919-1934)Online publication date: Oct-2023
https://doi.org/10.1109/TNET.2022.3216025
Wang HLin HZhong ZYang TShahzad M(2023)Enhanced Machine Learning Sketches for Network MeasurementsIEEE Transactions on Computers10.1109/TC.2022.318556072:4(957-970)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TC.2022.3185560
Li HChen QZhang YYang TCui B(2022)Stingy sketchProceedings of the VLDB Endowment10.14778/3523210.352322015:7(1426-1438)Online publication date: 22-Jun-2022
https://dl.acm.org/doi/10.14778/3523210.3523220
Lei YYu LLiu VXu MKuipers FOrda A(2022)PrintQueueProceedings of the ACM SIGCOMM 2022 Conference10.1145/3544216.3544257(516-529)Online publication date: 22-Aug-2022
https://dl.acm.org/doi/10.1145/3544216.3544257
Zheng HTian CYang TLin HLiu CZhang ZDou WChen GKuipers FOrda A(2022)FlyMonProceedings of the ACM SIGCOMM 2022 Conference10.1145/3544216.3544239(486-502)Online publication date: 22-Aug-2022
https://dl.acm.org/doi/10.1145/3544216.3544239
Ma CChen SZhang YXiao QOdegbile O(2022)Super Spreader Identification Using Geometric-Min FilterIEEE/ACM Transactions on Networking10.1109/TNET.2021.310803330:1(299-312)Online publication date: Feb-2022
https://doi.org/10.1109/TNET.2021.3108033
Dao DJang RJung CMohaisen DNyang D(2022)Minimizing Noise in HyperLogLog-Based Spread Estimation of Multiple Flows2022 52nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)10.1109/DSN53405.2022.00042(331-342)Online publication date: Jun-2022
https://doi.org/10.1109/DSN53405.2022.00042
Fu YAn LShen SChen KBarlet-Ros P(2021)Jellyfish: Locality-Sensitive Subflow SketchingIEEE INFOCOM 2021 - IEEE Conference on Computer Communications10.1109/INFOCOM42981.2021.9488847(1-10)Online publication date: 10-May-2021
https://doi.org/10.1109/INFOCOM42981.2021.9488847
Wang YWang HMa CChen SXia Y(2021)Supporting Real-Time ${T}$-Queries on Network Traffic with a Cloud-Based Offloading Model2021 IEEE 14th International Conference on Cloud Computing (CLOUD)10.1109/CLOUD53861.2021.00031(179-188)Online publication date: Sep-2021
https://doi.org/10.1109/CLOUD53861.2021.00031
Demianiuk VGorinsky SNikolenko SKogan K(2020)Robust Distributed Monitoring of Traffic FlowsIEEE/ACM Transactions on Networking10.1109/TNET.2020.3034890(1-14)Online publication date: 2020
https://doi.org/10.1109/TNET.2020.3034890
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Cited By

Index Terms

Recommendations