research-article

Sampling based algorithms for quantile computation in sensor networks

Authors:

Zengfeng Huang,

Yunhao LiuAuthors Info & Claims

SIGMOD '11: Proceedings of the 2011 ACM SIGMOD International Conference on Management of data

Pages 745 - 756

https://doi.org/10.1145/1989323.1989401

Published: 12 June 2011 Publication History

Abstract

We study the problem of computing approximate quantiles in large-scale sensor networks communication-efficiently, a problem previously studied by Greenwald and Khana [12] and Shrivastava et al [21]. Their algorithms have a total communication cost of O(k log² n / ε) and O(k log u / ε), respectively, where k is the number of nodes in the network, n is the total size of the data sets held by all the nodes, u is the universe size, and ε is the required approximation error. In this paper, we present a sampling based quantile computation algorithm with O(√kh/ε) total communication (h is the height of the routing tree), which grows sublinearly with the network size except in the pathological case h=Θ(k). In our experiments on both synthetic and real data sets, this improvement translates into a 10 to 100-fold communication reduction for achieving the same accuracy in the computed quantiles. Meanwhile, the maximum individual node communication of our algorithm is no higher than that of the previous two algorithms.

References

[1]

http://cast.cse.ohio-state.edu/exscal/.

[2]

http://www.greenorbs.org/.

[3]

http://www.ncfloodmaps.com.

[4]

J. Considine, F. Li, G. Kollios, and J. Byers. Approximate aggregation techniques for sensor databases. In Proc. IEEE International Conference on Data Engineering, 2004.

Digital Library

[5]

G. Cormode, M. Garofalakis, S. Muthukrishnan, and R. Rastogi. Holistic aggregates in a networked world: Distributed tracking of approximate quantiles. In Proc. ACM SIGMOD International Conference on Management of Data, 2005.

Digital Library

[6]

G. Cormode and M. Hadjieleftheriou. Finding frequent items in data streams. Proceedings of the VLDB Endowment, 1(2):1530--1541, 2008.

Digital Library

[7]

G. Cormode, F. Korn, S. Muthukrishnan, and D. Srivastava. Space- and time-efficient deterministic algorithms for biased quantiles over data streams. In Proc. ACM Symposium on Principles of Database Systems, 2006.

Digital Library

[8]

G. Cormode and S. Muthukrishnan. An improved data stream summary: The count-min sketch and its applications. Journal of Algorithms, 55(1):58--75, 2005.

Digital Library

[9]

E. Fasolo, M. Rossi, J. Widmer, and M. Zorzi. In-network aggregation techniques for wireless sensor networks: a survey. IEEE Wireless Communications, 14(2):70--87, 2007.

Digital Library

[10]

A. C. Gilbert, Y. Kotidis, S. Muthukrishnan, and M. J. Strauss. How to summarize the universe: Dynamic maintenance of quantiles. In Proc. International Conference on Very Large Data Bases, 2002.

Digital Library

[11]

M. Greenwald and S. Khanna. Space-efficient online computation of quantile summaries. In Proc. ACM SIGMOD International Conference on Management of Data, pages 58--66, 2001.

Digital Library

[12]

M. Greenwald and S. Khanna. Power conserving computation of order-statistics over sensor networks. In Proc. ACM Symposium on Principles of Database Systems, 2004.

Digital Library

[13]

W. B. Heinzelman, A. P. Chandrakasan, and H. Balakrishnan. An application-specific protocol architecture for wireless microsensor networks. IEEE Transactions on Wireless Communications, 1(4):660--670, 2002.

Digital Library

[14]

S. Madden, M. J. Franklin, J. M. Hellerstein, and W. Hong. TAG: a tiny aggregation service for ad-hoc sensor networks. In Proc. Symposium on Operating Systems Design and Implementation, 2002.

Digital Library

[15]

A. Mainwaring, D. Culler, J. Polastre, R. Szewczyk, and J. Anderson. Wireless sensor networks for habitat monitoring. In ACM WSNA, 2002.

Digital Library

[16]

A. Manjhi, S. Nath, and P. B. Gibbons. Tributaries and deltas: efficient and robust aggregation in sensor network streams. In Proc. ACM SIGMOD International Conference on Management of Data, 2005.

Digital Library

[17]

L. Mo, Y. He, Y. Liu, J. Zhao, S. Tang, X.-Y. Li, and G. Dai. Canopy closure estimates with GreenOrbs: Sustainable sensing in the forest. In Proc. ACM SenSys, 2009.

Digital Library

[18]

J. I. Munro and M. S. Paterson. Selection and sorting with limited storage. Theoretical Computer Science, 12:315--323, 1980.

[19]

S. Nath, P. B. Gibbons, S. Seshan, and Z. R. Anderson. Synopsis diffusion for robust aggregation in sensor networks. In Proc. ACM SenSys, 2004.

Digital Library

[20]

J. Polastre, R. Szewczyk, and D. Culler. Telos: Enabling ultra-low power wireless research. In ACM/IEEE IPSN, 2005.

Digital Library

[21]

N. Shrivastava, C. Buragohain, D. Agrawal, and S. Suri. Medians and beyond: New aggregation techniques for sensor networks. In Proc. ACM SenSys, 2004.

Digital Library

[22]

A. Silberstein, K. Munagala, and J. Yang. Energy-efficient monitoring of extreme values in sensor networks. In Proc. ACM SIGMOD International Conference on Management of Data, 2006.

Digital Library

[23]

V. N. Vapnik and A. Y. Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications, 16:264--280, 1971.

[24]

J. Yang, K. Munagala, and A. Silberstein. Data aggregation in sensor networks. In Encyclopedia of Database Systems. 2009.

[25]

Y. Yao and J. Gehrke. Query processing in sensor networks. In Proc. Conference on Innovative Data Systems Research, 2003.

[26]

K. Yi and Q. Zhang. Optimal tracking of distributed heavy hitters and quantiles. In Proc. ACM Symposium on Principles of Database Systems, 2009.

Digital Library

[27]

P. Zhang, C. M. Sadler, S. A. Lyon, and M. Martonosi. Hardware design experiences in ZebraNet. In Proc. ACM SenSys, 2004.

Digital Library

[28]

Y. Zhang, X. Lin, J. Xu, F. Korn, and W. Wang. Space-efficient relative error order sketch over data streams. In Proc. IEEE International Conference on Data Engineering, 2006.

Digital Library

Cited By

Lu CYu YKarimireddy SJordan MRaskar RKrause ABrunskill ECho KEngelhardt BSabato SScarlett J(2023)Federated conformal predictors for distributed uncertainty quantificationProceedings of the 40th International Conference on Machine Learning10.5555/3618408.3619361(22942-22964)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.5555/3618408.3619361
Shahout RFriedman RBen Basat R(2023)Together is Better: Heavy Hitters Quantile EstimationProceedings of the ACM on Management of Data10.1145/35889371:1(1-25)Online publication date: 30-May-2023
https://dl.acm.org/doi/10.1145/3588937
Ivkin NLiberty ELang KKarnin ZBraverman V(2022)Streaming Quantiles Algorithms with Small Space and Update TimeSensors10.3390/s2224961222:24(9612)Online publication date: 8-Dec-2022
https://doi.org/10.3390/s22249612
Show More Cited By

Index Terms

Sampling based algorithms for quantile computation in sensor networks
1. Theory of computation
  1. Design and analysis of algorithms
    1. Data structures design and analysis
      1. Sorting and searching

Recommendations

Optimal Gossip Algorithms for Exact and Approximate Quantile Computations
PODC '18: Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing

This paper gives drastically faster gossip algorithms to compute exact and approximate quantiles.

Gossip algorithms, which allow each node to contact a uniformly random other node in each round, have been intensely studied and been adopted in many ...
A Tight Lower Bound for Comparison-Based Quantile Summaries
PODS'20: Proceedings of the 39th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems

Quantiles, such as the median or percentiles, provide concise and useful information about the distribution of a collection of items, drawn from a totally ordered universe. We study data structures, called quantile summaries, which keep track of all ...
A note on efficient aggregate queries in sensor networks

We consider a scenario where nodes in a sensor network hold numeric items, and the task is to evaluate simple functions of the distributed data. In this note we present distributed protocols for computing the median with sublinear space and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGMOD '11: Proceedings of the 2011 ACM SIGMOD International Conference on Management of data

June 2011

1364 pages

ISBN:9781450306614

DOI:10.1145/1989323

General Chair:
Timos Sellis
IMIS/RC Athena
,
Program Chair:
Renée J. Miller
University of Toronto
,
Publications Chairs:
Anastasios Kementsietsidis
IBM T.J. Watson Research Center
,
Yannis Velegrakis
University of Trento

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOD: ACM Special Interest Group on Management of Data

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 June 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGMOD/PODS '11

Sponsor:

SIGMOD

SIGMOD/PODS '11: International Conference on Management of Data

June 12 - 16, 2011

Athens, Greece

Acceptance Rates

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

42
Total Citations
View Citations
401
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)1

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lu CYu YKarimireddy SJordan MRaskar RKrause ABrunskill ECho KEngelhardt BSabato SScarlett J(2023)Federated conformal predictors for distributed uncertainty quantificationProceedings of the 40th International Conference on Machine Learning10.5555/3618408.3619361(22942-22964)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.5555/3618408.3619361
Shahout RFriedman RBen Basat R(2023)Together is Better: Heavy Hitters Quantile EstimationProceedings of the ACM on Management of Data10.1145/35889371:1(1-25)Online publication date: 30-May-2023
https://dl.acm.org/doi/10.1145/3588937
Ivkin NLiberty ELang KKarnin ZBraverman V(2022)Streaming Quantiles Algorithms with Small Space and Update TimeSensors10.3390/s2224961222:24(9612)Online publication date: 8-Dec-2022
https://doi.org/10.3390/s22249612
Li HYi LTang BLu HJensen C(2022)Efficient and error-bounded spatiotemporal quantile monitoring in edge computing environmentsProceedings of the VLDB Endowment10.14778/3538598.353860015:9(1753-1765)Online publication date: 27-Jul-2022
https://dl.acm.org/doi/10.14778/3538598.3538600
Zhu TCheng XCheng WTian ZLi Y(2022)Principal component analysis based data collection for sustainable internet of things enabled Cyber–Physical SystemsMicroprocessors & Microsystems10.1016/j.micpro.2021.10403288:COnline publication date: 1-Feb-2022
https://dl.acm.org/doi/10.1016/j.micpro.2021.104032
Aseeri AZhang R(2022)Locally Differentially Private Quantile Summary Aggregation in Wireless Sensor NetworksIntelligent Information and Database Systems10.1007/978-3-031-21743-2_29(364-376)Online publication date: 9-Dec-2022
https://doi.org/10.1007/978-3-031-21743-2_29
Xue Z(2022)An Effective Single-Pass Approach for Estimating the Φ-quantile in Data StreamsAlgorithms and Architectures for Parallel Processing10.1007/978-3-030-95384-3_48(775-789)Online publication date: 23-Feb-2022
https://doi.org/10.1007/978-3-030-95384-3_48
Shi BZhao ZPeng YLi FPhillips JLi GLi ZIdreos SSrivastava D(2021)At-the-time and Back-in-time Persistent SketchesProceedings of the 2021 International Conference on Management of Data10.1145/3448016.3452802(1623-1636)Online publication date: 9-Jun-2021
https://dl.acm.org/doi/10.1145/3448016.3452802
Aseeri AZhang R(2021)SecQSA: Secure Sampling-Based Quantile Summary Aggregation in Wireless Sensor Networks2021 17th International Conference on Mobility, Sensing and Networking (MSN)10.1109/MSN53354.2021.00074(454-461)Online publication date: Dec-2021
https://doi.org/10.1109/MSN53354.2021.00074
Liu ZZhang A(2020)Sampling for Big Data Profiling: A SurveyIEEE Access10.1109/ACCESS.2020.29881208(72713-72726)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.2988120
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents