article

Automatic outlier detection for time series: an application to sensor data

Authors:

Sabyasachi Basu,

Martin MeckesheimerAuthors Info & Claims

Knowledge and Information Systems, Volume 11, Issue 2

Pages 137 - 154

Published: 01 February 2007 Publication History

Abstract

In this article we consider the problem of detecting unusual values or outliers from time series data where the process by which the data are created is difficult to model. The main consideration is the fact that data closer in time are more correlated to each other than those farther apart. We propose two variations of a method that uses the median from a neighborhood of a data point and a threshold value to compare the difference between the median and the observed data value. Both variations of the method are fast and can be used for data streams that occur in quick succession such as sensor data on an airplane.

References

[1]

Box GEP, Jenkins GM, Reinsel GC (1994) Time series analysis: forecasting and control, 3rd edn. Prentice-Hall, Englewood Cliffs, NJ

Crossref

Google Scholar

[2]

Chang I, Tiao GC, Chen C (1988) Estimation of time series parameters in the presence of outliers. Technometrics 30:193---204

Digital Library

Google Scholar

[3]

Pearson RK (2002) Data mining in the face of contaminated and incomplete records. In: Second SIAM conference on data mining, Arlington, VA

Google Scholar

[4]

Peña D (2001) Outliers, influential observations, and missing data. In: Peña D, Tiao GC, Tsay RS (eds) A course in time series analysis. Wiley, New York, pp 136---170

Google Scholar

[5]

Zhang S, Zhang C, Yang Q (2003) Data preparation for data mining. Appl Artif Intell 17:375---382

Crossref

Google Scholar

[6]

Zhang S, Zhang C, Yang Q (2004) Information enhancement for data mining. IEEE Intell Syst March/April 12---13

Crossref

Google Scholar

Cited By

View all

Tu DHe YCui WGe SZhang HHan SZhang DChaudhuri SSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Auto-Validate by-History: Auto-Program Data Quality Constraints to Validate Recurring Data PipelinesProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599776(4991-5003)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599776
Alanqary AAlomar AShah DRanzato MBeygelzimer ADauphin YLiang PVaughan J(2021)Change point detection via multivariate singular spectrum analysisProceedings of the 35th International Conference on Neural Information Processing Systems10.5555/3540261.3542039(23218-23230)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.5555/3540261.3542039
Blázquez-García AConde AMori ULozano J(2021)A Review on Outlier/Anomaly Detection in Time Series DataACM Computing Surveys10.1145/344469054:3(1-33)Online publication date: 17-Apr-2021
https://dl.acm.org/doi/10.1145/3444690
Show More Cited By

Recommendations

A Review on Outlier/Anomaly Detection in Time Series Data

Recent advances in technology have brought major breakthroughs in data collection, enabling a large amount of data to be gathered over time and thus generating time series. Mining this data has become an important task for researchers and practitioners ...
Automatic outlier detection for time series: an application to sensor data
Special Issue on Mining Low-Quality Data

In this article we consider the problem of detecting unusual values or outliers from time series data where the process by which the data are created is difficult to model. The main consideration is the fact that data closer in time are more correlated ...
Distance outlier detection with a mobile beacon in wireless sensor networks localisation

Range-based localisation algorithms rely on distance measurements between nodes. If outlier distances exist, they make the estimated positions deviate from the true positions. Thus, it is important to detect and remove outlier distance measurements to ...

Comments

Information & Contributors

Information

Published In

Knowledge and Information Systems Volume 11, Issue 2

February 2007

127 pages

ISSN:0219-1377

Issue’s Table of Contents

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 February 2007

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Tu DHe YCui WGe SZhang HHan SZhang DChaudhuri SSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Auto-Validate by-History: Auto-Program Data Quality Constraints to Validate Recurring Data PipelinesProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599776(4991-5003)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599776
Alanqary AAlomar AShah DRanzato MBeygelzimer ADauphin YLiang PVaughan J(2021)Change point detection via multivariate singular spectrum analysisProceedings of the 35th International Conference on Neural Information Processing Systems10.5555/3540261.3542039(23218-23230)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.5555/3540261.3542039
Blázquez-García AConde AMori ULozano J(2021)A Review on Outlier/Anomaly Detection in Time Series DataACM Computing Surveys10.1145/344469054:3(1-33)Online publication date: 17-Apr-2021
https://dl.acm.org/doi/10.1145/3444690
Huang YZhong LChen Y(2020)Filtering Infrequent Behavior in Business Process Discovery by Using the Minimum ExpectationInternational Journal of Cognitive Informatics and Natural Intelligence10.4018/IJCINI.202004010114:2(1-15)Online publication date: 1-Apr-2020
https://dl.acm.org/doi/10.4018/IJCINI.2020040101
Luo PQiu CWang Y(2020)DAS: Deep Autoencoder with Scoring Neural Network for Anomaly DetectionProceedings of the 2020 3rd International Conference on Algorithms, Computing and Artificial Intelligence10.1145/3446132.3446181(1-6)Online publication date: 24-Dec-2020
https://dl.acm.org/doi/10.1145/3446132.3446181
Marathe A(2020)LRZ Convolution: An Algorithm for Automatic Anomaly Detection in Time-series DataProceedings of the 32nd International Conference on Scientific and Statistical Database Management10.1145/3400903.3400904(1-12)Online publication date: 7-Jul-2020
https://dl.acm.org/doi/10.1145/3400903.3400904
Belhadi ADjenouri YLin JCano A(2020)Trajectory Outlier DetectionACM Transactions on Management Information Systems10.1145/339963111:3(1-29)Online publication date: 21-Jun-2020
https://dl.acm.org/doi/10.1145/3399631
Wu ZLiu PHu ZLi BWang J(2020)CRATOSProceedings of the 2020 European Symposium on Software Engineering10.1145/3393822.3432319(194-203)Online publication date: 6-Nov-2020
https://dl.acm.org/doi/10.1145/3393822.3432319
Alquier PBertin KDoukhan PGarnier R(2020)High-dimensional VAR with low-rank transitionStatistics and Computing10.1007/s11222-020-09929-730:4(1139-1153)Online publication date: 1-Jul-2020
https://dl.acm.org/doi/10.1007/s11222-020-09929-7
(2018)Time series classification using MACD-histogram-based recurrence plotInternational Journal of Computational Intelligence Studies10.5555/3302649.33026517:3-4(192-213)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.5555/3302649.3302651
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Recommendations

A Review on Outlier/Anomaly Detection in Time Series Data