research-article

CAM conscious integrated answering of frequent elements and top-k queries over data streams

Authors:

Divyakant Agrawal,

Amr El AbbadiAuthors Info & Claims

DaMoN '08: Proceedings of the 4th international workshop on Data management on new hardware

Pages 1 - 10

https://doi.org/10.1145/1457150.1457152

Published: 13 June 2008 Publication History

Abstract

Frequent elements and top-k queries constitute an important class of queries for data stream analysis applications. Certain applications require answers for both frequent elements and top-k queries on the same stream. In addition, the ever increasing data rates call for providing fast answers to the queries, and researchers have been looking towards exploiting specialized hardware for this purpose. Content Addressable Memory(CAM) provides an efficient way of looking up elements and hence are well suited for the class of algorithms that involve lookups. In this paper, we present a fast and efficient CAM conscious integrated solution for answering both frequent elements and top-k queries on the same stream. We call our scheme CAM conscious Space Saving with Stream Summary (CSSwSS), and it can efficiently answer continuous queries. We provide an implementation of the proposed scheme using commodity CAM chips, and the experimental evaluation demonstrates that not only does the proposed scheme outperforms existing CAM conscious techniques by an order of magnitude at query loads of about 10%, but the proposed scheme can also efficiently answer continuous queries.

References

[1]

N. Bandi, A. Metwally, D. Agrawal, and A. E. Abbadi. Fast data stream algorithms using associative memories. In SIGMOD, pages 247--256, Beijing, China, 2007.

Digital Library

[2]

N. Bandi, S. Schnieder, D. Agrawal, and A. E. Abbadi. Hardware Acceleration of Database operations using Content Addressable Memories. In DaMoN, 2005.

[3]

B. Bhattacharjee, N. Abe, K. Goldman, B. Zadrozny, V. R. Chillakuru, M. del Carpio, and C. Apte. Using secure coprocessors for privacy preserving collaborative data mining and analysis. In DaMoN '06, 2006.

Digital Library

[4]

M. Charikar, K. Chen, and M. Farach-Colton. Finding frequent items in data streams. In ICALP'02, pages 693--703, 2002.

Digital Library

[5]

J. Cieslewicz and K. A. Ross. Adaptive Aggregation on Chip Multiprocessors. In VLDB 2007, pages 339--350, 2007.

Digital Library

[6]

G. Das, D. Gunopulos, N. Koudas, and N. Sarkas. Ad-hoc top-k query answering for data streams. In VLDB, pages 183--194, 2007.

Digital Library

[7]

E. D. Demaine, A. López-Ortiz, and J. I. Munro. Frequency estimation of internet packet streams with limited space. In ESA, volume 2461, pages 348--360, 2002.

Digital Library

[8]

R. Fang, B. He, M. Lu, K. Yang, N. K. Govindaraju, Q. Luo, and P. V. Sander. Gpuqp: query co-processing using graphics processors. In SIGMOD '07, pages 1061--1063, 2007.

Digital Library

[9]

M. Fischer and S. Salzberg. Finding a majority among n votes: Solution to problem 81--5. In Journal of Algorithms 3(4), pages 376--379, 1982.

[10]

B. T. Gold, A. Ailamaki, L. Huston, and B. Falsai. Accelerating database operators using a network processor. In DAMON '05, 2005.

Digital Library

[11]

M. Greenwald and S. Khanna. Space-efficient online computation of quantile summaries. In SIGMOD, 2001.

Digital Library

[12]

N. Hardavellas, I. Pandis, R. Johnson, N. G. Mancheril, A. Ailamaki, and B. Falsafi. Database Servers on Chip Multiprocessors: Limitations and Opportunities. In CIDR, pages 79--87, 2007.

[13]

Integrated Devices Technologies, Integrated IP Co-processor IDT 75K62134. http://idt.com/?genID=75K62134&source=products_genericPart_75K62134, 2006.

[14]

Intel Internet Exchange Architecture for Network Processors. Technical report, Intel Corp., 2002.

[15]

Intel IXP 2800 Network Processor Product Brief. http://www.intel.com/design/network/prodbrf/279054.htm, 2006.

[16]

G. S. Manku and R. Motwani. Approximate frequency counts over data streams. In VLDB, 2002.

Digital Library

[17]

A. Metwally, D. Agrawal, and A. E. Abbadi. An integrated efficient solution for computing frequent and top-k elements in data streams. ACM Trans. Database Syst., 31(3):1095--1133, 2006.

Digital Library

[18]

K. Mouratidis, S. Bakiras, and D. Papadias. Continuous monitoring of top-k queries over sliding windows. In SIGMOD, pages 635--646, 2006.

Digital Library

[19]

R. Panigrahy and D. Thomas. Finding Frequent Elements in Non-Bursty Streams. In ESA, pages 53--62, October 2007.

Digital Library

[20]

D. Shah and P. Gupta. Fast updating algorithms for tcams. IEEE Micro, 21(1):36--47, 2001.

Digital Library

[21]

Teja networking systems and teja np application development platform. http://www.teja.com, 2006.

[22]

S. Venkataraman, D. Song, P. Gibbons, and A. Blum. New streaming algorithms for fast detection of superspreaders. In NDSS, 2005.

[23]

J. Zhou, J. Cieslewicz, K. A. Ross, and M. Shah. Improving database performance on simultaneous multithreading processors. In VLDB '05, pages 49--60, 2005.

Digital Library

[24]

G. K. Zipf. Human Behavior and The Principle of Least Effort. Addison-Wesley, Cambridge, MA, 1949.

[25]

M. Zukowski, S. Héman, and P. Boncz. Architecture-conscious hashing. In DaMoN '06, 2006.

Digital Library

Cited By

Zhang YFang BZhang Y(2019)Parallelizing weighted frequency counting in high-speed network monitoringComputer Communications10.1016/j.comcom.2010.04.02634:4(536-547)Online publication date: 4-Jan-2019
https://dl.acm.org/doi/10.1016/j.comcom.2010.04.026
Inoue KHoang TPham C(2018)Frequent items counter based on binary decodersIEICE Electronics Express10.1587/elex.15.2018080815:20(20180808-20180808)Online publication date: 2018
https://doi.org/10.1587/elex.15.20180808
Inoue KHoang TNguyen XNguyen HPham C(2018)VLSI Design of Frequent Items Counting Using Binary Decoders Applied to 8-bit per Item Case-study2018 14th Conference on Ph.D. Research in Microelectronics and Electronics (PRIME)10.1109/PRIME.2018.8430308(161-164)Online publication date: Jul-2018
https://doi.org/10.1109/PRIME.2018.8430308
Show More Cited By

Index Terms

CAM conscious integrated answering of frequent elements and top-k queries over data streams
1. Information systems
  1. Information systems applications

Recommendations

An integrated efficient solution for computing frequent and top-k elements in data streams

We propose an approximate integrated approach for solving both problems of finding the most popular k elements, and finding frequent elements in a data stream coming from a large domain. Our solution is space efficient and reports both frequent and top-...
An Efficient Algorithm for top-k Queries on Uncertain Data Streams
ICMLA '12: Proceedings of the 2012 11th International Conference on Machine Learning and Applications - Volume 01

We tackle the problem of answering maximum probabilistic top-k tuple set queries. We use a sliding-window model on uncertain data streams and present an efficient algorithm for processing sliding-window queries on uncertain streams. In each sliding ...
Efficient Process of Top-k Range-Sum Queries over Multiple Streams with Minimized Global Error

Due to the resource limitation in the data stream environments, it has been reported that answering user queries according to the wavelet synopsis of a stream is an essential ability of a Data Stream Management System (DSMS). In the literature, recent ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

DaMoN '08: Proceedings of the 4th international workshop on Data management on new hardware

June 2008

57 pages

ISBN:9781605581842

DOI:10.1145/1457150

Editors:
Qiong Luo
Hong Kong University of Science and Technology
,
Kenneth A. Ross
Columbia University

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOD: ACM Special Interest Group on Management of Data
Microsoft: Microsoft
Intel: Intel
IBM: IBM

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 June 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

DaMoN '08

Sponsor:

SIGMOD
Microsoft
Intel
IBM

DaMoN '08: Data Management on New Hardware

June 13, 2008

Vancouver, Canada

Acceptance Rates

Overall Acceptance Rate 94 of 127 submissions, 74%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
187
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 19 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang YFang BZhang Y(2019)Parallelizing weighted frequency counting in high-speed network monitoringComputer Communications10.1016/j.comcom.2010.04.02634:4(536-547)Online publication date: 4-Jan-2019
https://dl.acm.org/doi/10.1016/j.comcom.2010.04.026
Inoue KHoang TPham C(2018)Frequent items counter based on binary decodersIEICE Electronics Express10.1587/elex.15.2018080815:20(20180808-20180808)Online publication date: 2018
https://doi.org/10.1587/elex.15.20180808
Inoue KHoang TNguyen XNguyen HPham C(2018)VLSI Design of Frequent Items Counting Using Binary Decoders Applied to 8-bit per Item Case-study2018 14th Conference on Ph.D. Research in Microelectronics and Electronics (PRIME)10.1109/PRIME.2018.8430308(161-164)Online publication date: Jul-2018
https://doi.org/10.1109/PRIME.2018.8430308
Hoang TNguyen XNguyen HTruong NLe DInoue KPham C(2017)FPGA-based frequent items counting using matrix of equality comparators2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS)10.1109/MWSCAS.2017.8052916(285-288)Online publication date: Aug-2017
https://doi.org/10.1109/MWSCAS.2017.8052916
Zhang YSun YZhang JXu JWu Y(2013)An efficient framework for parallel and continuous frequent item monitoringConcurrency and Computation: Practice and Experience10.1002/cpe.318226:18(2856-2879)Online publication date: 5-Dec-2013
https://doi.org/10.1002/cpe.3182
Zhang Y(2012)Parallelizing the Weighted Lossy Counting Algorithm in High-speed Network MonitoringProceedings of the 2012 Second International Conference on Instrumentation, Measurement, Computer, Communication and Control10.1109/IMCCC.2012.183(757-761)Online publication date: 8-Dec-2012
https://dl.acm.org/doi/10.1109/IMCCC.2012.183
Das SAntony SAgrawal DAbbadi A(2009)CoTSProceedings of the 2009 IEEE International Conference on Data Engineering10.1109/ICDE.2009.231(1323-1326)Online publication date: 29-Mar-2009
https://dl.acm.org/doi/10.1109/ICDE.2009.231

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten