Article

Incremental maintenance of quotient cube for median

Authors:

Anthony K. H. Tung,

Shan WangAuthors Info & Claims

KDD '04: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining

Pages 226 - 235

https://doi.org/10.1145/1014052.1014079

Published: 22 August 2004 Publication History

Abstract

Data cube pre-computation is an important concept for supporting OLAP(Online Analytical Processing) and has been studied extensively. It is often not feasible to compute a complete data cube due to the huge storage requirement. Recently proposed quotient cube addressed this issue through a partitioning method that groups cube cells into equivalence partitions. Such an approach is not only useful for distributive aggregate functions such as SUM but can also be applied to the holistic aggregate functions like MEDIAN.Maintaining a data cube for holistic aggregation is a hard problem since its difficulty lies in the fact that history tuple values must be kept in order to compute the new aggregate when tuples are inserted or deleted. The quotient cube makes the problem harder since we also need to maintain the equivalence classes. In this paper, we introduce two techniques called addset data structure and sliding window to deal with this problem. We develop efficient algorithms for maintaining a quotient cube with holistic aggregation functions that takes up reasonably small storage space. Performance study shows that our algorithms are effective, efficient and scalable over large databases.

References

[1]

S. Agarwal, R. Agrawal, P. M. Deshpande, A. Gupta, J. F. Naughton, R. Ramakrishnan, and S. Sarawagi. On the computation of multidimensional aggregates. In Proc. 1996 Int. Conf. Very Large Data Bases (VLDB'96), pages 506--521, Bombay, India, Sept. 1996.

Digital Library

[2]

D. Barbara. Quasi-cubes: Exploiting approximation in multidimensional databases. SIGMOD Record, 26:12--17, 1997.

Digital Library

[3]

K. Beyer and R. Ramakrishnan. Bottom-up computation of sparse and iceberg cubes. In Proc. 1999 ACM-SIGMOD Int. Conf. Management of Data (SIGMOD'99), pages 359--370, Philadelphia, PA, June 1999.

Digital Library

[4]

A. Davey and H. A. Priestley. Introduction to Lattices and Order. Cambridge University Press, 1990.

[5]

B. Ganter, R. Wille, and C. Franzke. Formal concept analysis: mathematical foundations. Springer-verlag, 1999.

Digital Library

[6]

R. Godin, R. Missaoui, and H. Alaoui. Incremental concept formation algorithms based on galois lattices. Computational Intelligence, 11:246--267, 1991.

[7]

J. Gray, S. Chaudhuri, A. Bosworth, A. Layman, D. Reichart, M. Venkatrao, F. Pellow, and H. Pirahesh. Data cube: A relational aggregation operator generalizing group-by, cross-tab and sub-totals. Data Mining and Knowledge Discovery, 1:29--54, 1997.

Digital Library

[8]

V. Harinarayan, A. Rajaraman, and J. D. Ullman. Implementing data cubes efficiently. In Proc. 1996 ACM-SIGMOD Int. Conf. Management of Data (SIGMOD'96), pages 205--216, Montreal, Canada, June 1996.

Digital Library

[9]

W. Labio, U. Yang, Y. Cui, H. Garcia-Molina, and J. Widom. Performance issues in incremental warehouse maintenance. In Proc. of the 26st Int'l Conference on Very Large Databases (VLDB'00), 2000.

Digital Library

[10]

L. Lakshmanan, J. Pei, and J. Han. Quotient cube: How to summarize the semantics of a data cube. In Proc. 2002 Int. Conf. Very Large Data Bases (VLDB'02), 2002.

Digital Library

[11]

L. Lakshmanan, J. Pei, and Y. Zhao. Qc-trees: An efficient summary structure for semantic olap. In Proc. Of ACM-SIGMOD Int'l Conference on Management of Data, 2003.

Digital Library

[12]

I. Mumick, D. Quass, and B. Mumick. Maintaince of data cubes and summary tables in a warehouse. In Proc. Of ACM-SIGMOD Int'l Conference on Management of Data, 1997.

Digital Library

[13]

K. Ross and D. Srivastava. Fast computation of sparse datacubes. In Proc. 1997 Int. Conf. Very Large Data Bases (VLDB'97), pages 116--125, Athens, Greece, Aug. 1997.

Digital Library

[14]

K. Ross, D. Srivastava, and S. Sudarshan. Materialized view maintenance and integrity constraint checking:trading space for time. In Proc. Of ACM-SIGMOD Int'l Conference on Management of Data, 1996.

Digital Library

[15]

J. Shanmugasundaram, U. Fayyad, and P. Bradley. Compressed data cubes for olap aggregate query approximation on continuous dimensions. In Proc. Of ACM-SIGKDD Int'l Conference on Knowledge Discovery and Data Mining, 1999.

Digital Library

[16]

A. Shukla, P. Deshpande, and J. F. Naughton. Materialized view selection for multidimensional datasets. In Proc. 1998 Int. Conf. Very Large Data Bases (VLDB'98), pages 488--499, New York, NY, Aug. 1998.

Digital Library

[17]

Y. Sismanis, A. Deligiannakis, N. Roussopoulos, and Y. Kotidis. Dwarf: Shrinking the petacube. In Proc. Of ACM-SIGMOD Int'l Conference on Management of Data, 2002.

Digital Library

[18]

W. Wang, J. Feng, H. Lu, and J. Yu. Condensed cube: An effective approach to reducing data cube size. In Proc. of 2002 Int'l Conf. on Data Engineering (ICDE'02), 2002.

Digital Library

[19]

Y. Zhao, P. M. Deshpande, and J. F. Naughton. An array-based algorithm for simultaneous multidimensional aggregates. In Proc. 1997 ACM-SIGMOD Int. Conf. Management of Data (SIGMOD'97), pages 159--170, Tucson, Arizona, May 1997.

Digital Library

Cited By

Silva RHirata Cde Castro Lima J(2020)Big high-dimension data cube designs for hybrid memory systemsKnowledge and Information Systems10.1007/s10115-020-01505-962:12(4717-4746)Online publication date: 26-Aug-2020
https://doi.org/10.1007/s10115-020-01505-9
Wu ZWang MQin H(2019)Filter based continuous median query algorithm in sensor networkJournal of Computational Methods in Sciences and Engineering10.3233/JCM-19001319:3(695-706)Online publication date: 17-Jul-2019
https://doi.org/10.3233/JCM-190013
Wang SLi CChen H(2012)A conversation with Professor Shan Wang et al.ACM SIGKDD Explorations Newsletter10.1145/2207243.220726513:2(92-95)Online publication date: 1-May-2012
https://dl.acm.org/doi/10.1145/2207243.2207265
Show More Cited By

Index Terms

Incremental maintenance of quotient cube for median
1. Information systems
  1. Information systems applications
    1. Data mining

Recommendations

Semi-closed cube: an effective approach to trading off data cube size and query response time

The results of data cube will occupy huge amount of disk space when the base table is of a large number of attributes. A new type of data cube, compact data cube like condensed cube and quotient cube, was proposed to solve the problem. It compresses ...
Incremental maintenance of quotient cube based on galois lattice
Abstract
Data cube computation is a well-known expensive operation and has been studied extensively. It is often not feasible to compute a complete data cube due to the huge storage requirement. Recently proposed quotient cube addressed this fundamental ...
Data Cube: A Relational Aggregation Operator Generalizing Group-By, Cross-Tab, and Sub-Totals

Data analysis applications typically aggregate data across many dimensions looking for anomalies or unusual patterns. The SQL aggregate functions and the GROUP BY operator produce zero-dimensional or one-dimensional aggregates. Applications need the N-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '04: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining

August 2004

874 pages

ISBN:1581138881

DOI:10.1145/1014052

General Chairs:
Won Kim
Cyber Database Solutions
,
Ronny Kohavi
Amazon.com
,
Program Chairs:
Johannes Gehrke
Cornell University
,
William DuMouchel
AT&T Labs Research

Copyright © 2004 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 August 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

KDD04

Sponsor:

KDD04: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 22 - 25, 2004

WA, Seattle, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
782
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 19 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Silva RHirata Cde Castro Lima J(2020)Big high-dimension data cube designs for hybrid memory systemsKnowledge and Information Systems10.1007/s10115-020-01505-962:12(4717-4746)Online publication date: 26-Aug-2020
https://doi.org/10.1007/s10115-020-01505-9
Wu ZWang MQin H(2019)Filter based continuous median query algorithm in sensor networkJournal of Computational Methods in Sciences and Engineering10.3233/JCM-19001319:3(695-706)Online publication date: 17-Jul-2019
https://doi.org/10.3233/JCM-190013
Wang SLi CChen H(2012)A conversation with Professor Shan Wang et al.ACM SIGKDD Explorations Newsletter10.1145/2207243.220726513:2(92-95)Online publication date: 1-May-2012
https://dl.acm.org/doi/10.1145/2207243.2207265
Feng MDong GLi JTan YWong L(2010)PATTERN SPACE MAINTENANCE FOR DATA UPDATES AND INTERACTIVE MINING*Computational Intelligence10.1111/j.1467-8640.2010.00360.x26:3(282-317)Online publication date: 27-Jul-2010
https://doi.org/10.1111/j.1467-8640.2010.00360.x
Li H(2010)A GPU-based closed frequent itemsets mining algorithm over stream2010 IEEE International Conference on Intelligent Computing and Intelligent Systems10.1109/ICICISYS.2010.5658432(6-10)Online publication date: Oct-2010
https://doi.org/10.1109/ICICISYS.2010.5658432
Morfonios KIoannidis Y(2010)Revisiting the cube lifecycle in the presence of hierarchiesThe VLDB Journal — The International Journal on Very Large Data Bases10.1007/s00778-009-0160-319:2(257-282)Online publication date: 1-Apr-2010
https://dl.acm.org/doi/10.1007/s00778-009-0160-3
Xiao YZhang YWang SChen H(2009)Incremental Computation for MEDIAN Cubes in What-If AnalysisAdvances in Data and Web Management10.1007/978-3-642-00672-2_23(248-259)Online publication date: 2009
https://doi.org/10.1007/978-3-642-00672-2_23
Morfonios KIoannidis Y(2008)Supporting the data cube lifecycleThe VLDB Journal — The International Journal on Very Large Data Bases10.1007/s00778-006-0036-817:4(729-764)Online publication date: 1-Jul-2008
https://dl.acm.org/doi/10.1007/s00778-006-0036-8
Li HChen H(2008)Moment+Proceedings of the 4th international conference on Advanced Data Mining and Applications10.1007/978-3-540-88192-6_63(612-619)Online publication date: 8-Oct-2008
https://dl.acm.org/doi/10.1007/978-3-540-88192-6_63
Xi JChen FZhang P(2008)A New Bitmap Index and a New Data Cube Compression TechnologyProceedings of the international conference on Computational Science and Its Applications, Part II10.1007/978-3-540-69848-7_97(1218-1228)Online publication date: 30-Jun-2008
https://dl.acm.org/doi/10.1007/978-3-540-69848-7_97
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten