research-article

Change detection in categorical evolving data streams

Authors:

Bernhard Pfahringer,

Pascal PonceletAuthors Info & Claims

SAC '14: Proceedings of the 29th Annual ACM Symposium on Applied Computing

Pages 792 - 797

https://doi.org/10.1145/2554850.2554864

Published: 24 March 2014 Publication History

Abstract

Detecting change in evolving data streams is a central issue for accurate adaptive learning. In real world applications, data streams have categorical features, and changes induced in the data distribution of these categorical features have not been considered extensively so far. Previous work on change detection focused on detecting changes in the accuracy of the learners, but without considering changes in the data distribution.

To cope with these issues, we propose a new unsupervised change detection method, called CDCStream (Change Detection in Categorical Data Streams), well suited for categorical data streams. The proposed method is able to detect changes in a batch incremental scenario. It is based on the two following characteristics: (i) a summarization strategy is proposed to compress the actual batch by extracting a descriptive summary and (ii) a new segmentation algorithm is proposed to highlight changes and issue warnings for a data stream. To evaluate our proposal we employ it in a learning task over real world data and we compare its results with state of the art methods. We also report qualitative evaluation in order to show the behavior of CDCStream.

References

[1]

C. C. Aggarwal, editor. Data Streams - Models and Algorithms. Advances in Database Systems. Springer, 2007.

Digital Library

[2]

A. Asuncion and D. Newman. UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences, 2007.

[3]

A. Bifet, G. Holmes, R. Kirkby, and B. Pfahringer. MOA: Massive Online Analysis. J. Mach. Learn. Res., 11(May): 1601--1604, 2010.

Digital Library

[4]

F. Cao and J. Z. Huang. A concept-drifting detection algorithm for categorical evolving data. In PAKDD (2), 2013.

[5]

F. Cao, J. Liang, L. Bai, X. Zhao, and C. Dang. A framework for clustering categorical time-evolving data. IEEE T. Fuzzy Syst., 18(5): 872--882, 2010.

Digital Library

[6]

H. Chen, M. Chen, and S. Lin. Catching the trend: A framework for clustering concept-drifting categorical data. IEEE TKDE, 21(5): 652--665, 2009.

Digital Library

[7]

K. Chen and L. Liu. He-tree: a framework for detecting changes in clustering structure for categorical data streams. VLDB J., 18(6): 1241--1260, 2009.

Digital Library

[8]

J. Gama, P. Medas, G. Castillo, and P. Rodrigues. Learning with drift detection. In SBIA, 2004.

[9]

J. Gao, W. Fan, J. Han, and P. S. Yu. A general framework for mining concept-drifting data streams with skewed distributions. In SDM, 2007.

[10]

M. Harries, C. Sammut, and K. Horn. Extracting hidden context. Mach. Learn., 32(2): 101--126, 1998.

Digital Library

[11]

D. Ienco, A. Bifet, I. Zliobaite, and B. Pfahringer. Clustering based active learning for evolving data streams. In Discovery Science, 2013.

[12]

D. Ienco, R. G. Pensa, and R. Meo. From context to distance: Learning dissimilarity for categorical data clustering. ACM TKDD, 6(1): 1:1--1:25, mar 2012.

Digital Library

[13]

E. Ikonomovska, J. Gama, and S. Dzeroski. Learning model trees from evolving data streams. Data Mining and Know. Disc., 23(1): 128--168, 2010.

Digital Library

[14]

I. H. Witten and E. Frank. Data Mining: Practical Machine Learning Tools and Techniques. Data Management Systems. Morgan Kaufmann, 2005.

Digital Library

[15]

L. Yu and H. Liu. Feature selection for high-dimensional data: A fast correlation-based filter solution. In ICML, 2003.

Cited By

Wang XBorsoi RRichard C(2023)Online Change Point Detection on Riemannian Manifolds with Karcher Mean Estimates2023 31st European Signal Processing Conference (EUSIPCO)10.23919/EUSIPCO58844.2023.10289744(2033-2037)Online publication date: 4-Sep-2023
https://doi.org/10.23919/EUSIPCO58844.2023.10289744
Plasse JAdams N(2019)Multiple changepoint detection in categorical data streamsStatistics and Computing10.1007/s11222-019-09858-0Online publication date: 15-Feb-2019
https://doi.org/10.1007/s11222-019-09858-0
Maaradji ADumas MRosa MOstovar A(2017)Detecting Sudden and Gradual Drifts in Business Processes from Execution TracesIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2017.272060129:10(2140-2154)Online publication date: 1-Oct-2017
https://doi.org/10.1109/TKDE.2017.2720601
Show More Cited By

Index Terms

Change detection in categorical evolving data streams
1. Information systems
  1. Information systems applications

Recommendations

On clustering massive text and categorical data streams

In this paper, we will study the data stream clustering problem in the context of text and categorical data domains. While the clustering problem has been studied recently for numeric data streams, the problems of text and categorical data present ...
Clustering categorical data streams

In this paper, we propose an efficient clustering algorithm for analyzing categorical data streams. It has been proved that the proposed algorithm uses small memory footprints. We provide empirical analysis on the performance of the algorithm in ...
Automated anomaly detection for categorical data by repurposing a form filling recommender system
Data quality is crucial in modern software systems, like data-driven decision support systems. However, data quality is affected by data anomalies, which represent instances that deviate from most of the data. These anomalies affect the reliability and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SAC '14: Proceedings of the 29th Annual ACM Symposium on Applied Computing

March 2014

1890 pages

ISBN:9781450324694

DOI:10.1145/2554850

Conference Chairs:
Yookun Cho
Seoul National University, Korea
,
Sung Y. Shin
South Dakota State University
,
Program Chairs:
Sangwook Kim
Kyungpook National University, Korea
,
Chih-Cheng Hung
Southern Polytechnic State University
,
Jiman Hong
Soongsil University, South Korea

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAPP: ACM Special Interest Group on Applied Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 March 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SAC 2014

Sponsor:

SIGAPP

SAC 2014: Symposium on Applied Computing

March 24 - 28, 2014

Gyeongju, Republic of Korea

Acceptance Rates

SAC '14 Paper Acceptance Rate 218 of 939 submissions, 23%;

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
199
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)1

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang XBorsoi RRichard C(2023)Online Change Point Detection on Riemannian Manifolds with Karcher Mean Estimates2023 31st European Signal Processing Conference (EUSIPCO)10.23919/EUSIPCO58844.2023.10289744(2033-2037)Online publication date: 4-Sep-2023
https://doi.org/10.23919/EUSIPCO58844.2023.10289744
Plasse JAdams N(2019)Multiple changepoint detection in categorical data streamsStatistics and Computing10.1007/s11222-019-09858-0Online publication date: 15-Feb-2019
https://doi.org/10.1007/s11222-019-09858-0
Maaradji ADumas MRosa MOstovar A(2017)Detecting Sudden and Gradual Drifts in Business Processes from Execution TracesIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2017.272060129:10(2140-2154)Online publication date: 1-Oct-2017
https://doi.org/10.1109/TKDE.2017.2720601
D’Ettorre SViktor HPaquet E(2017)Context-Based Abrupt Change Detection and Adaptation for Categorical Data StreamsDiscovery Science10.1007/978-3-319-67786-6_1(3-17)Online publication date: 16-Sep-2017
https://doi.org/10.1007/978-3-319-67786-6_1
Chen KKoh YRiddle P(2015)Tracking Drift Severity in Data StreamsAI 2015: Advances in Artificial Intelligence10.1007/978-3-319-26350-2_9(96-108)Online publication date: 22-Nov-2015
https://doi.org/10.1007/978-3-319-26350-2_9

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents