Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3603269.3604847acmconferencesArticle/Chapter ViewAbstractPublication PagescommConference Proceedingsconference-collections
research-article

OmniWindow: A General and Efficient Window Mechanism Framework for Network Telemetry

Published: 01 September 2023 Publication History

Abstract

Recent network telemetry solutions typically target programmable switches to achieve high performance and in-network visibility. They partition the packet stream into windows and then apply various stream processing techniques to summarize flow-level statistics. However, existing studies focus on the measurement within each window. Window management is still a missing piece due to the resource limitation of programmable switches. In this paper, we propose OmniWindow, a general and efficient window mechanism framework. OmniWindow splits the original window into fine-grained sub-windows such that the sub-windows can be merged into various window types. To deal with the resource restriction, OmniWindow carefully designs its data plane memory layout and proposes a window synchronization method. It also employs a collaborative architecture that can collect and reset stateful data in sub-windows within a limited time. We prototype OmniWindow on Tofino. We incorporate OmniWindow into a SOTA query-driven telemetry system and eight sketch-based telemetry algorithms. Our experiments demonstrate that OmniWindow enables these telemetry solutions to achieve higher accuracy than conventional window mechanism.

References

[1]
2008. IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems. IEEE Std 1588-2008 (2008).
[2]
Pankaj K Agarwal, Graham Cormode, Zengfeng Huang, Jeff M Phillips, Zhewei Wei, and Ke Yi. 2013. Mergeable summaries. ACM Transactions on Database Systems 38, 4 (2013).
[3]
Noga Alon, Yossi Matias, and Mario Szegedy. 1996. The space complexity of approximating the frequency moments. In Proc. of ACM STOC.
[4]
Ran Ben-Basat, Gil Einziger, Roy Friedman, and Yaron Kassner. 2016. Heavy hitters in streams and sliding windows. In Proc. of IEEE INFOCOM.
[5]
Pat Bosshart, Dan Daly, Glen Gibb, Martin Izzard, Nick McKeown, Jennifer Rexford, Cole Schlesinger, Dan Talayco, Amin Vahdat, George Varghese, et al. 2014. P4: Programming protocol-independent packet processors. Proc. of ACM SIGCOMM CCR 44 (2014), 87--95.
[6]
Pat Bosshart, Glen Gibb, Hun-Seok Kim, George Varghese, Nick McKeown, Martin Izzard, Fernando Mujica, and Mark Horowitz. 2013. Forwarding Metamorphosis: Fast Programmable Match-Action Processing in Hardware for SDN. In Proc. of ACM SIGCOMM.
[7]
Qizhe Cai, Shubham Chaudhary, Midhul Vuppalapati, Jaehyun Hwang, and Rachit Agarwal. 2021. Understanding Host Network Stack Overheads. In Proc. of ACM SIGCOMM.
[8]
Caida Anonymized Internet Traces 2018 Dataset. [n. d.]. http://www.caida.org/data/passive/passive_dataset.xml. ([n. d.]).
[9]
NVIDIA Mellanox ConnectX-5 EN Card. [n. d.]. https://network.nvidia.com/files/doc-2020/pb-connectx-5-en-card.pdf. ([n. d.]).
[10]
Ho-Leung Chan, Tak-Wah Lam, Lap-Kei Lee, and Hing-Fung Ting. 2012. Continuous monitoring of distributed data streams over a time-based sliding window. Algorithmica 62, 3 (2012).
[11]
F. Chang, Wu chang Feng, and Kang Li. 2004. Approximate caches for packet classification. In Proc. of IEEE INFOCOM.
[12]
Xiang Chen, Qun Huang, Peiqiao Wang, Hongyan Liu, Yuxin Chen, Dong Zhang, Haifeng Zhou, and Chunming Wu. 2021. MTP: Avoiding Control Plane Overload with Measurement Task Placement. In Proc. of IEEE INFOCOM.
[13]
Xiang Chen, Qun Huang, Dong Zhang, Haifeng Zhou, and Chunming Wu. 2020. ApproSync: Approximate State Synchronization for Programmable Networks. In Proc. of IEEE ICNP.
[14]
Xiaoqi Chen, Shir Landau-Feibish, Mark Braverman, and Jennifer Rexford. 2020. BeauCoup: Answering Many Network Traffic Queries, One Memory Update at a Time. In Proc. of ACM SIGCOMM.
[15]
Graham Cormode and S. Muthukrishnan. 2005. An Improved Data Stream Summary: The Count-Min Sketch and its Applications. Journal of Algorithms (2005), 58--75.
[16]
Graham Cormode and Ke Yi. 2011. Tracking distributed aggregates over time-based sliding windows. In Proc. of ACM PODC.
[17]
Andrew R. Curtis, Wonho Kim, and Praveen Yalagandula. 2011. Mahout: Low-overhead datacenter traffic management using end-host-based elephant detection. In Proc. of IEEE INFOCOM.
[18]
Andrew R. Curtis, Jeffrey C. Mogul, Jean Tourrilhes, Praveen Yalagandula, Puneet Sharma, and Sujata Banerjee. 2011. DevoFlow: Scaling Flow Management for High-Performance Networks. In Proc. of ACM SIGCOMM.
[19]
Nathan Farrington, George Porter, Sivasankar Radhakrishnan, Hamid Hajabdolali Bazzaz, Vikram Subramanya, Yeshaiahu Fainman, George Papen, and Amin Vahdat. 2010. Helios: A Hybrid Electrical/Optical Switch Architecture for Modular Data Centers. In Proc. of ACM SIGCOMM.
[20]
Joan Feigenbaum, Sampath Kannan, Martin J Strauss, and Mahesh Viswanathan. 2002. An approximate L 1-difference algorithm for massive data streams. SIAM J. Comput. 32, 1 (2002).
[21]
Yilong Geng, Shiyu Liu, Zi Yin, Ashish Naik, Balaji Prabhakar, Mendel Rosunblum, and Amin Vahdat. 2018. Exploiting a Natural Network Effect for Scalable, FineGrained Clock Synchronization. In Proc. of USENIX NSDI.
[22]
Xiangyang Gou, Long He, Yinda Zhang, Ke Wang, Xilai Liu, Tong Yang, Yi Wang, and Bin Cui. 2020. Sliding Sketches: A Framework Using Time Zones for Data Stream Processing in Sliding Windows. In Proc. of ACM SIGKDD.
[23]
gRPC. [n. d.]. https://grpc.io/. ([n. d.]).
[24]
Arpit Gupta, Rob Harrison, Marco Canini, Nick Feamster, Jennifer Rexford, and Walter Willinger. 2018. Sonata: Query-Driven Streaming Network Telemetry. In Proc. of ACM SIGCOMM.
[25]
Stefan Heule, Marc Nunkesser, and Alexander Hall. 2013. HyperLogLog in Practice: Algorithmic Engineering of a State of The Art Cardinality Estimation Algorithm. In Proc. of EDBT.
[26]
Qun Huang, Xin Jin, Patrick P C Lee, Runhui Li, Lu Tang, Yi-Chao Chen, and Gong Zhang. 2017. SketchVisor: Robust Network Measurement for Software Packet Processing. In Proc. of ACM SIGCOMM.
[27]
Qun Huang, Patrick P. C. Lee, and Yungang Bao. 2018. SketchLearn: Relieving User Burdens in Approximate Measurement with Automated Statistical Inference. In Proc. of ACM SIGCOMM.
[28]
Qun Huang, Siyuan Sheng, Xiang Chen, Yungang Bao, Rui Zhang, Yanwei Xu, and Gong Zhang. 2021. Toward Nearly-Zero-Error Sketching via Compressive Sensing. In Proc. of USENIX NSDI.
[29]
Qun Huang, Haifeng Sun, Patrick P. C. Lee, Wei Bai, Feng Zhu, and Yungang Bao. 2020. OmniMon: Re-Architecting Network Telemetry with Resource Efficiency and Full Accuracy. In Proc. of ACM SIGCOMM.
[30]
Regant YS Hung, Lap-Kei Lee, and Hing-Fung Ting. 2010. Finding frequent items over sliding windows with constant update time. Inform. Process. Lett. 110, 7 (2010).
[31]
Mobin Javed and Vern Paxson. 2013. Detecting Stealthy, Distributed SSH Brute-Forcing. In Proc. of ACM SIGSAC.
[32]
Lavanya Jose, Minlan Yu, and Jennifer Rexford. 2011. Online Measurement of Large Traffic Aggregates on Commodity Switches. In Proc of USENIX Hot-ICE.
[33]
Jaeyeon Jung, V. Paxson, A. W. Berger, and H. Balakrishnan. 2004. Fast portscan detection using sequential hypothesis testing. In Proc. of IEEE Symposium on Security and Privacy.
[34]
Daehyeok Kim, Zaoxing Liu, Yibo Zhu, Changhoon Kim, Jeongkeun Lee, Vyas Sekar, and Srinivasan Seshan. 2020. TEA: Enabling State-Intensive Network Functions on Programmable Switches. In Proc. of ACM SIGCOMM.
[35]
Daehyeok Kim, Yibo Zhu, Changhoon Kim, Jeongkeun Lee, and Srinivasan Seshan. 2018. Generic External Memory for Switch Data Planes. In Proc. of ACM HotNets.
[36]
Data Plane Development Kit. [n. d.]. https://www.dpdk.org/. ([n. d.]).
[37]
Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. (2009).
[38]
Paolo Laffranchini, Luis Rodrigues, Marco Canini, and Balachander Krishnamurthy. 2019. Measurements As First-class Artifacts. In Proc. of IEEE INFOCOM.
[39]
Leslie Lamport. 1978. Time, Clocks, and the Ordering of Events in a Distributed System. Commun. ACM 21, 7 (1978).
[40]
Jonatan Langlet, Ran Ben-Basat, Gabriele Oliaro, Michael Mitzenmacher, Minlan Yu, and Gianni Antichi. 2023. Direct Telemetry Access. In Proc. of ACM SIGCOMM.
[41]
Jonatan Langlet, Ran Ben-Basat, Sivaramakrishnan Ramanathan, Gabriele Oliaro, Michael Mitzenmacher, Minlan Yu, and Gianni Antichi. 2021. Zero-CPU Collection with Direct Telemetry Access. In Proc. of ACM HotNets.
[42]
Ki Suh Lee, Han Wang, Vishal Shrivastav, and Hakim Weatherspoon. 2016. Globally Synchronized Time via Datacenter Networks. In Proc. of ACM SIGCOMM.
[43]
Lap-Kei Lee and HF Ting. 2006. A simpler and more efficient deterministic scheme for finding frequent items over sliding windows. In Proc. of ACM PODS.
[44]
Jakub Lemiesz. 2021. On the Algebra of Data Sketches. VLDB Endowment 14, 9 (2021).
[45]
Bojie Li, Tianyi Cui, Zibo Wang, Wei Bai, and Lintao Zhang. 2019. Socksdirect: Datacenter Sockets Can Be Fast and Compatible. In Proc. of ACM SIGCOMM.
[46]
Yuliang Li, Rui Miao, Changhoon Kim, and Minlan Yu. 2016. FlowRadar: A Better NetFlow for Data Centers. In Proc. of USENIX NSDI.
[47]
Yuliang Li, Rui Miao, Changhoon Kim, and Minlan Yu. 2016. LossRadar: Fast Detection of Lost Packets in Data Center Networks. In Proc. of ACM CoNEXT.
[48]
W. Liu, W. Qu, J. Gong, and K. Li. 2016. Detection of Superpoints Using a Vector Bloom Filter. IEEE Trans. on Information Forensics and Security 11 (2016), 514--527.
[49]
Xuemei Liu, Meral Shirazipour, Minlan Yu, and Ying Zhang. 2016. MOZART: Temporal Coordination of Measurement. In Proc. of ACM SOSR.
[50]
Zaoxing Liu, Antonis Manousis, Gregory Vorsanger, Vyas Sekar, and Vladimir Braverman. 2016. One Sketch to Rule Them All: Rethinking Network Flow Monitoring with UnivMon. In Proc. of ACM SIGCOMM.
[51]
Ziyuan Liu, Zhixiong Niu, Ran Shu, Wenxue Cheng, Peng Cheng, Yongqiang Xiong, Lihua Yuan, Jacob Nelson, and Dan RK Ports. 2022. A Disaggregate Data Collecting Approach for Loss-Tolerant Applications. In Proc. of ACM APNet.
[52]
Srinivas Narayana, Anirudh Sivaraman, Vikram Nathan, Prateesh Goyal, Venkat Arun, Mohammad Alizadeh, Vimalkumar Jeyakumar, and Changhoon Kim. 2017. Language-Directed Hardware Design for Network Performance Monitoring. In Proc. of ACM SIGCOMM.
[53]
Jelani Nelson and David P Woodruff. 2010. Fast manhattan sketches in data streams. In Proc. of ACM PODS.
[54]
Odysseas Papapetrou, Minos Garofalakis, and Antonios Deligiannakis. 2012. Sketch-based Querying of Distributed Sliding-Window Data Streams. VLDB Endowment 5, 10 (2012).
[55]
PktGen. [n. d.]. https://pktgen-dpdk.readthedocs.io. ([n. d.]).
[56]
Jeff Rasley, Brent Stephens, Colin Dixon, Eric Rozner, Wes Felter, Kanak Agarwal, John Carter, and Rodrigo Fonseca. 2014. Planck: Millisecond-scale Monitoring and Control for Commodity Networks. In Proc. of ACM SIGCOMM.
[57]
Intel Architecture Instruction Set Extensions Programming Reference. [n. d.]. https://www.intel.com/content/dam/develop/external/us/en/documents/319433-024-697869.pdf. ([n. d.]).
[58]
Intel SSE4 Programming Reference. [n. d.]. https://www.intel.com/content/dam/develop/external/us/en/documents/d9156103-705230.pdf. ([n. d.]).
[59]
Nicoló Rivetti, Yann Busnel, and Achour Mostéfaoui. 2015. Efficiently Summarizing Data Streams over Sliding Windows. In Proc. of IEEE NCA.
[60]
InfiniBand Architecture Specification Release 1.2.1 Annex A17: RoCEv2. [n. d.]. https://cw.infinibandta.org/document/dl/7781. ([n. d.]).
[61]
Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).
[62]
Vibhaalakshmi Sivaraman, Srinivas Narayana, Ori Rottenstreich, S. Muthukrishnan, and Jennifer Rexford. 2017. Heavy-Hitter Detection Entirely in the Data Plane. In Proc. of ACM SOSR.
[63]
John Sonchack, Adam J Aviv, Eric Keller, and Jonathan M Smith. 2018. Turboflow: Information rich flow record generation on commodity switches. In Proc. of ACM EuroSys.
[64]
Rajath Subramanyam, Indranil Gupta, Luke M. Leslie, and Wenting Wang. 2015. Idempotent Distributed Counters Using a Forgetful Bloom Filter. In Proc. of IEEE ICCAC.
[65]
Praveen Tammana, Rachit Agarwal, and Myungjin Lee. 2018. Distributed Network Monitoring and Debugging with SwitchPointer. In Proc. of USENIX NSDI.
[66]
Lu Tang, Qun Huang, and Patrick PC Lee. 2019. MV-Sketch: A fast and compact invertible sketch for heavy flow detection in network data streams. In Proc. of IEEE INFOCOM.
[67]
Lu Tang, Qun Huang, and Patrick PC Lee. 2020. A fast and compact invertible sketch for network-wide heavy flow detection. IEEE/ACM Transactions on Networking 28, 5 (2020), 2350--2363.
[68]
Lu Tang, Qun Huang, and Patrick P. C. Lee. 2020. SpreadSketch: Toward Invertible and Network-Wide Detection of Superspreaders. In Proc. of IEEE INFOCOM.
[69]
Tofino. [n. d.]. https://www.barefootnetworks.com/products/brief-tofino/. ([n. d.]).
[70]
Steve T. Watt, Shankar Achanta, Hamza Abubakari, Eric Sagen, Zafer Korkmaz, and Husam Ahmed. 2015. Understanding and applying precision time protocol. In 2015 Saudi Arabia Smart Grid (SASG).
[71]
Kyu-Young Whang, Brad T Vander-Zanden, and Howard M Taylor. 1990. A linear-time probabilistic counting algorithm for database applications. Proc. of ACM TODS 15 (1990), 208--229.
[72]
Tong Yang, Jie Jiang, Peng Liu, Qun Huang, Junzhi Gong, Yang Zhou, Rui Miao, Xiaoming Li, and Steve Uhlig. 2018. Elastic Sketch: Adaptive and Fast Network-wide Measurements. In Proc. of ACM SIGCOMM.
[73]
MyungKeun Yoon. 2010. Aging Bloom Filter with Two Active Buffers for Dynamic Sets. IEEE TKDE 22, 1 (2010).
[74]
Minlan Yu. 2019. Network Telemetry: Towards a Top-down Approach. ACM SIGCOMM Computer Communication Review 49, 1 (2019).
[75]
Minlan Yu, Lavanya Jose, and Rui Miao. 2013. Software Defined Traffic Measurement with OpenSketch. In Proc. of USENIX NSDI.
[76]
Yifei Yuan, Dong Lin, Ankit Mishra, Sajal Marwaha, Rajeev Alur, and Boon Thau Loo. 2017. Quantitative Network Monitoring with NetQRE. In Proc. of ACM SIGCOMM.
[77]
Ying Zhang. 2013. An Adaptive Flow Counting Method for Anomaly Detection in SDN. In Proc. of ACM CoNEXT.
[78]
Yikai Zhao, Kaicheng Yang, Zirui Liu, Tong Yang, Li Chen, Shiyi Liu, Naiqian Zheng, Ruixin Wang, Hanbo Wu, Yi Wang, et al. 2021. LightGuardian: A Full-Visibility, Lightweight, In-band Telemetry System Using Sketchlets. In Proc. of USENIX NSDI.
[79]
Hao Zheng, Chen Tian, Tong Yang, Huiping Lin, Chang Liu, Zhaochen Zhang, Wanchun Dou, and Guihai Chen. 2022. FlyMon: Enabling on-the-Fly Task Reconfiguration for Network Measurement. In Proc. of ACM SIGCOMM.
[80]
Yu Zhou, Dai Zhang, Kai Gao, Chen Sun, Jiamin Cao, Yangyang Wang, Mingwei Xu, and Jianping Wu. 2020. Newton: Intent-Driven Network Traffic Monitoring. In Proc. of CoNEXT.
[81]
Yang Zhou, Ying Zhang, Minlan Yu, Guangyu Wang, Dexter Cao, Eric Sung, and Starsky Wong. 2022. Evolvable Network Telemetry at Facebook. In Proc. of USENIX NSDI.
[82]
Yibo Zhu, Nanxi Kang, Jiaxin Cao, Albert Greenberg, Guohan Lu, Ratul Mahajan, Dave Maltz, Lihua Yuan, Ming Zhang, Ben Y. Zhao, and Haitao Zheng. 2015. Packet-Level Telemetry in Large Datacenter Networks. In Proc. of ACM SIGCOMM.

Cited By

View all
  • (2024)μMon: Empowering Microsecond-level Network Monitoring with WaveletsProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672236(274-290)Online publication date: 4-Aug-2024
  • (2024)Distributed Network Telemetry With Resource Efficiency and Full AccuracyIEEE/ACM Transactions on Networking10.1109/TNET.2023.332734532:3(1857-1872)Online publication date: Jun-2024
  • (2024)Advancing Sketch-Based Network Measurement: A General, Fine-Grained, Bit-Adaptive Sliding Window Framework2024 IEEE/ACM 32nd International Symposium on Quality of Service (IWQoS)10.1109/IWQoS61813.2024.10682923(1-10)Online publication date: 19-Jun-2024

Index Terms

  1. OmniWindow: A General and Efficient Window Mechanism Framework for Network Telemetry

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ACM SIGCOMM '23: Proceedings of the ACM SIGCOMM 2023 Conference
    September 2023
    1217 pages
    ISBN:9798400702365
    DOI:10.1145/3603269
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 01 September 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Badges

    Author Tags

    1. window mechanism
    2. network telemetry

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    ACM SIGCOMM '23
    Sponsor:
    ACM SIGCOMM '23: ACM SIGCOMM 2023 Conference
    September 10, 2023
    NY, New York, USA

    Acceptance Rates

    Overall Acceptance Rate 462 of 3,389 submissions, 14%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)493
    • Downloads (Last 6 weeks)35
    Reflects downloads up to 04 Oct 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)μMon: Empowering Microsecond-level Network Monitoring with WaveletsProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672236(274-290)Online publication date: 4-Aug-2024
    • (2024)Distributed Network Telemetry With Resource Efficiency and Full AccuracyIEEE/ACM Transactions on Networking10.1109/TNET.2023.332734532:3(1857-1872)Online publication date: Jun-2024
    • (2024)Advancing Sketch-Based Network Measurement: A General, Fine-Grained, Bit-Adaptive Sliding Window Framework2024 IEEE/ACM 32nd International Symposium on Quality of Service (IWQoS)10.1109/IWQoS61813.2024.10682923(1-10)Online publication date: 19-Jun-2024

    View Options

    Get Access

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media