research-article

UA-Sketch: An Accurate Approach to Detect Heavy Flow based on Uninterrupted Arrival

Authors:

Jiawei HuangAuthors Info & Claims

ICPP '22: Proceedings of the 51st International Conference on Parallel Processing

August 2022

Article No.: 57, Pages 1 - 11

https://doi.org/10.1145/3545008.3545017

Published: 13 January 2023 Publication History

Abstract

Heavy flow detection in enormous network traffic is a critical task for network measurement. Due to the limited memory size and high link capacity, accurate detection of heavy flows becomes challenging in large-scale networks. Almost all existing approaches of detecting heavy flows use single-dimension statistics of flow size to make flow-replacement decisions. However, under the mass number of small flows, the heavy flows are prone to be frequently and mistakenly replaced, resulting in unsatisfactory accuracy. To solve this problem, we reveal that the number of uninterrupted arrival packets is a useful metric in identifying flow types. We further propose UA-Sketch that expels small flows and protects heavy ones according to the multiple-dimension statistics including both estimated flow size and number of uninterrupted arrival packets. The test results of trace-driven simulations and OVS experiments show that, even under small memory, UA-Sketch achieves higher accuracy than the existing works, with the F1 Score by up to 2.1 ×.

References

[1]

[1]A. Sivaraman, S. Subramanian, A. Agrawal, S. Chole, S. Chuang, T. Edsall, M. Alizadeh, H. Balakrishnan, S. Katti and N. McKeown. Programmable packet scheduling at line rate. In Proc. ACM SIGCOMM, 2016.

Digital Library

[2]

[2]A. Feldmann, A. Greenberg, C. Lund, N. Reingold, J. Rexford, and F. True. Deriving Traffic Demands for Operational IP Networks: Methodology and Experience. In Proc. SIGCOMM, 2000.

Digital Library

[3]

[3]A. Lakhina, M. Crovella, and C. Diot. Characterization of Network-Wide Anomalies in Traffic Flows. In Proc. ACM IMC, 2004.

Digital Library

[4]

[4]Y. Li, R. Miao, C. Kim, and M. Yu. FlowRadar: A Better NetFlow for Data Centers. In Proc. NSDI, 2016.

[5]

[5]E. Demaine, A. López-Ortiz, and J. Munro. Frequency Estimation of Internet Packet Streams with Limited Space. In Proc. AlgorithmsESA, 2002.

[6]

[6]G. Manku and R. Motwani. Approximate Frequency Counts over Data Streams. In Proc. VLDB, 2002.

Digital Library

[7]

[7]A. Metwally, D. Agrawal, and A. E. Abbadi. Efficient Computation of Frequent and Top-k Elements in Data Streams. In Proc. Springer ICDT, 2005.

[8]

[8]Z. Li, F. Xiao, S. Wang, T. Pei, and J. Li. Achievable rate maximization for cognitive hybrid satellite-terrestrial networks with af-relays. IEEE Journal on Selected Areas in Communications 36, 2 (2018), 304–313, 2018.

[9]

[9]Y. Zhang, M. Roughan, W. Willinger, and L. Qiu. Spatio-temporal compressive sensing and internet traffic matrices. ACM SIGCOMM Computer Communication Review 39, 4 (2009), 267–278, 2009.

Digital Library

[10]

[10]G. Cormode and S. Muthukrishnan. An improved data stream summary: the count-min sketch and its applications. Journal of Algorithms 55, 1 (2005), 58–75, 2005.

Digital Library

[11]

[11]C. Estan and G. Varghese. New Directions in Traffic Measurement and Accounting: Focusing on the Elephants, Ignoring the Mice. ACM Transactions on Computer Systems 21, 3 (2003), 270–313, 2003.

Digital Library

[12]

[12]M. Charikar, K. Chen, and M. Farach-Colton. Finding Frequent Items in Data Streams. In Proc. ICALP, 2002.

Digital Library

[13]

[13]D. Powers. Applications and explanations of Zipf’s law. In Proc. NeMLaP3/CoNLL, 1998.

[14]

[14]L. Tang, Q. Huang, and P. Lee. MV-Sketch: A Fast and Compact Invertible Sketch for Heavy Flow Detection in Network Data Streams. In Proc. IEEE INFOCOM, 2019.

Digital Library

[15]

[15]T. Yang, J. Jiang, P. Liu, Q. Huang, J. Gong, Y. Zhou, R. Miao, X. Li, and S. Uhlig. Elastic Sketch: Adaptive and Fast Network-Wide Measurements. In Proc. ACM SIGCOMM, 2018.

Digital Library

[16]

[16]The caida anonymized 2016 internet traces. http://www.caida.org/data/overview/.

[17]

[17]MAWI Working Group Traffic Archive. http://mawi.wide.ad.jp/mawi/.

[18]

[18]A. Appleby. https://github.com/aappleby/smhasher.

[19]

[19]R. Basat, G. Einziger, R. Friedman, and Y. Kassner. Randomized admission policy for efficient top-k and frequency estimation. In Proc. IEEE INFOCOM, 2017.

[20]

[20]R. Basat, G. Einziger, R. Friedman, and Y. Kassner. Heavy hitters in streams and sliding windows. In Proc. IEEE INFOCOM, 2016.

Digital Library

[21]

[21]D. Ting. Data Sketches for Disaggregated Subset Sum and Frequent Item Estimation. In Proc. ACM SIGMOD, 2018.

[22]

[22]T. Yang, Y. Zhou, H. Jin, S. Chen, and X. Li. Pyramid Sketch: a Sketch Framework for Frequency Estimation of Data Streams. VLDB Endowment 10, 11 (2017), 1442–1453, 2017.

[23]

[23]T. Yang, S. Gao, Z. Sun, Y. Wang, Y. Shen, and X. Li. Diamond Sketch: Accurate Per-flow Measurement for Big Streaming Data. IEEE Transactions on Parallel and Distributed Systems 30, 12 (2019), 2650–2662, 2019.

[24]

[24]J. Gong, T. Yang, H. Zhang, H. Li, S. Uhlig, S. Chen, L. Uden, and X. Li. Heavykeeper: An Accurate Algorithm for Finding Top-k Elephant Flows. In Proc. USENIX ATC, 2018.

[25]

[25]J. Li, Z. Li, Y. Xu, S. Jiang, T. Yang, B. Cui, Y. Dai, and G. Zhang. WavingSketch: An Unbiased and Generic Sketch for Finding Top-k Items in Data Streams. In Proc. ACM SIGKDD, 2020.

Digital Library

[26]

[26]P. Roy, A. Khan, and G. Alonso. Augmented Sketch: Faster and More Accurate Stream Processing. In Proc. ACM SIGMOD, 2016.

Digital Library

[27]

[27]Q. Huang, P. Lee, and Y. Bao. Sketchlearn: Relieving User Burdens in Approximate Measurement with Automated Statistical Inference. In Proc. ACM SIGCOMM, 2018.

Digital Library

[28]

[28]Y. Zhang, Z. Liu, R. Wang, T. Yang, J. Li, R. Miao, P. Liu, R. Zhang, J. Jiang. CocoSketch: High-Performance Sketch-Based Measurement over Arbitrary Partial Key Query. In Proc. ACM SIGCOMM, 2021.

Cited By

Zhang BHuang HSun YDu YWang D(2024)Jigsaw-Sketch: a fast and accurate algorithm for finding top-k elephant flows in high-speed networksScience China Information Sciences10.1007/s11432-022-3794-167:4Online publication date: 20-Mar-2024
https://doi.org/10.1007/s11432-022-3794-1

Index Terms

UA-Sketch: An Accurate Approach to Detect Heavy Flow based on Uninterrupted Arrival
1. Networks
  1. Network services
    1. Network management
    2. Network monitoring

Recommendations

MCSketch: An Accurate Sketch for Heavy Flow Detection and Heavy Flow Frequency Estimation
Web and Big Data
Abstract
Accurately finding heavy flows in data streams is challenging owing to limited memory availability. Prior algorithms have focused on accuracy in heavy flow detection but cannot provide the frequency of a heavy flow exactly. In this paper, we ...
Read More
An effective and accurate flow size measurement using funnel-shaped sketch
Abstract
Accurate measurement of network traffic is an essential part of current network management tasks. Sketch is a kind of probabilistic data structure widely accepted in network measurement. However, it is challenging for classic sketches to reduce ...
Highlights
- An efficient funnel-shaped sketch (FS) is proposed for network measurement.
- Most measurement errors are caused by the collisions among mouse flows.
- The accuracy and robustness of FS are superior to state-of-the-art schemes.
Read More
AGC Sketch: An effective and accurate per-flow measurement to adapt flow size distribution
Abstract
Network measurements are indispensable for efficient network management. As a probabilistic data structure, sketch is widely used in network measurements. To adapt to the skewed and dynamic distribution of network traffic, most of the existing ...
Highlights
- A two-stage data structure called AGC Sketch is proposed, which more adapts various flow size distribution using effective memory utilization.
- A unique update and query mechanism for AGC Sketch is presented, which can allocate ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICPP '22: Proceedings of the 51st International Conference on Parallel Processing

August 2022

976 pages

ISBN:9781450397339

DOI:10.1145/3545008

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 January 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Key Research and Development Program of Hunan
Natural Science Foundation of Hunan Province, China
Key Research and Development Program of Guangxi
the National Natural Science Foundation of China

Conference

ICPP '22

ICPP '22: 51st International Conference on Parallel Processing

August 29 - September 1, 2022

Bordeaux, France

Acceptance Rates

Overall Acceptance Rate 91 of 313 submissions, 29%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
196
Total Downloads

Downloads (Last 12 months)129
Downloads (Last 6 weeks)16

Other Metrics

View Author Metrics

Citations

Cited By

Zhang BHuang HSun YDu YWang D(2024)Jigsaw-Sketch: a fast and accurate algorithm for finding top-k elephant flows in high-speed networksScience China Information Sciences10.1007/s11432-022-3794-167:4Online publication date: 20-Mar-2024
https://doi.org/10.1007/s11432-022-3794-1

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents