Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/1993806.1993839acmconferencesArticle/Chapter ViewAbstractPublication PagespodcConference Proceedingsconference-collections
abstract

Tracking distributed aggregates over time-based sliding windows

Published: 06 June 2011 Publication History

Abstract

The area of distributed monitoring requires tracking the value of a function of distributed data as new observations are made. An important case is when attention is restricted to only a recent time period, such as the last hour of readings---the sliding window case. In this announcement, we outline a novel paradigm for handling such monitoring problems, which we dub the "forward/backward" approach. This provides clean solutions for several fundamental problems, such as counting, tracking frequent items, and maintaining order statistics. We obtain efficient protocols for these problems that improve on previous work, and are easy to implement. Specifically, we obtain optimal O(k/ε log(ε n/k)) communication per window of n updates for tracking counts and heavy hitters with accuracy ε across k sites; and near-optimal communication of O(k/epsilon log2(1/ε) log (n/k)) for quantiles.

References

[1]
C. Arackaparambil, J. Brody, and A. Chakrabarti. Functional monitoring without monotonicity. In ICALP, 2009.
[2]
C. Busch, S. Tirthapura, and B. Xu. Sketching asycnhronous streams over sliding windows. In ACM PODC, 2006.
[3]
H.-L. Chan, T.-W. Lam, L.-K. Lee, and H.-F. Ting. Continuous monitoring of distributed data streams over a time-based sliding window. In STACS, 2010.
[4]
G. Cormode and M. Garofalakis. Sketching streams through the net: Distributed approximate query tracking. In VLDB, 2005.
[5]
G. Cormode, M. Garofalakis, S. Muthukrishnan, and R. Rastogi. Holistic aggregates in a networked world: Distributed tracking of approximate quantiles. In ACM SIGMOD, 2005.
[6]
G. Cormode, S. Muthukrishnan, and K. Yi. Algorithms for distributed, functional monitoring. In ACM-SIAM SODA, 2008.
[7]
G. Cormode, S. Muthukrishnan, K. Yi, and Q. Zhang. Optimal sampling from distributed streams. In ACM PODS, 2010.
[8]
Y. Emek and A. Korman. Efficient threshold detection in a distributed environment. In ACM PODC, 2010.
[9]
P. Gibbons and S. Tirthapura. Distributed streams algorithms for sliding windows. In ACM SPAA, 2002.
[10]
R. Keralapura, G. Cormode, and J. Ramamirtham. Communication-efficient distributed monitoring of thresholded counts. In ACM SIGMOD, 2006.
[11]
F. Kuhn, T. Locher, and S. Schmid. Distributed computation of the mode. In ACM PODC, 2008.
[12]
B. Patt-Shamir. A note on efficient aggregate queries in sensor networks. In ACM PODC, 2004.
[13]
I. Sharfman, A. Schuster, and D. Keren. A geometric approach to monitoring threshold functions over distributed data streams. In ACM SIGMOD, 2006.
[14]
K. Yi and Q. Zhang. Optimal tracking of distributed heavy hitters and quantiles. In ACM PODS, 2009.

Cited By

View all
  • (2023)OmniWindow: A General and Efficient Window Mechanism Framework for Network TelemetryProceedings of the ACM SIGCOMM 2023 Conference10.1145/3603269.3604847(867-880)Online publication date: 10-Sep-2023
  • (2022)A Sketch Framework for Approximate Data Stream Processing in Sliding WindowsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.3151140(1-1)Online publication date: 2022
  • (2020)Sliding Sketches: A Framework using Time Zones for Data Stream Processing in Sliding WindowsProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3394486.3403144(1015-1025)Online publication date: 23-Aug-2020
  • Show More Cited By

Index Terms

  1. Tracking distributed aggregates over time-based sliding windows

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    PODC '11: Proceedings of the 30th annual ACM SIGACT-SIGOPS symposium on Principles of distributed computing
    June 2011
    406 pages
    ISBN:9781450307192
    DOI:10.1145/1993806

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 06 June 2011

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. data streams
    2. sliding window

    Qualifiers

    • Abstract

    Conference

    PODC '11
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 740 of 2,477 submissions, 30%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)10
    • Downloads (Last 6 weeks)2
    Reflects downloads up to 10 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)OmniWindow: A General and Efficient Window Mechanism Framework for Network TelemetryProceedings of the ACM SIGCOMM 2023 Conference10.1145/3603269.3604847(867-880)Online publication date: 10-Sep-2023
    • (2022)A Sketch Framework for Approximate Data Stream Processing in Sliding WindowsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.3151140(1-1)Online publication date: 2022
    • (2020)Sliding Sketches: A Framework using Time Zones for Data Stream Processing in Sliding WindowsProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3394486.3403144(1015-1025)Online publication date: 23-Aug-2020
    • (2016)High Fidelity Data Reduction for Big Data Security Dependency AnalysesProceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security10.1145/2976749.2978378(504-516)Online publication date: 24-Oct-2016
    • (2015)Distributed Online TrackingProceedings of the 2015 ACM SIGMOD International Conference on Management of Data10.1145/2723372.2737790(2047-2061)Online publication date: 27-May-2015
    • (2013)The continuous distributed monitoring modelACM SIGMOD Record10.1145/2481528.248153042:1(5-14)Online publication date: 1-May-2013
    • (2011)Continuous distributed monitoringProceedings of the First International Workshop on Algorithms and Models for Distributed Event Processing10.1145/2031792.2031793(1-10)Online publication date: 19-Sep-2011

    View Options

    Get Access

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media