Article

Enhancing Effectiveness of Outlier Detections for Low Density Patterns

Authors:

Ada Wai-Chee Fu,

David Wai-Lok CheungAuthors Info & Claims

PAKDD '02: Proceedings of the 6th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining

Pages 535 - 548

Published: 06 May 2002 Publication History

Abstract

Outlier detection is concerned with discovering exceptional behaviors of objects in data sets.It is becoming a growingly useful tool in applications such as credit card fraud detection, discovering criminal behaviors in e-commerce, identifying computer intrusion, detecting health problems, etc. In this paper, we introduce a connectivity-based outlier factor (COF) scheme that improves the effectiveness of an existing local outlier factor (LOF) scheme when a pattern itself has similar neighbourhood density as an outlier. We give theoretical and empirical analysis to demonstrate the improvement in effectiveness and the capability of the COF scheme in comparison with the LOF scheme.

References

[1]

A. Arning, R. Agrawal, P. Raghavan: "A Linear Method for Deviation detection in Large Databases", Proc. of 2nd Intl. Conf. On Knowledge Discovery and Data Mining, 1996, pp 164-169.

[2]

V. Barnett, T. Lewis: "Outliers in Statistical Data", John Wiley, 1994.

[3]

M. Breuning, Hans-Peter Kriegel, R. Ng, J. Sander: "LOF: Identifying density based Local Outliers", Proc. of the ACM SIGMOD Conf. On Management of Data, 2000.

[4]

W. DuMouchel, M. Schonlau: "A Fast Computer Intrusion Detection Algorithm based on Hypothesis Testing of Command Transition Probabilities", Proc. of 4th Intl. Conf. On Knowledge Discovery and Data Mining, 1998, pp. 189-193.

[5]

M. Ester, H. Kriegel, J. Sander, X. Xu: "A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise", Proc. of 2nd Intl. Conf. On Knowledge Discovery and Data Mining, 1996, pp 226-231.

[6]

T. Fawcett, F. Provost: "Adaptive Fraud Detection", Data Mining and Knowledge Discovery Journal, Kluwer Academic Publishers, Vol. 1, No. 3, 1997, pp 291-316.

[7]

D. Hawkins: "Identification of Outliers", Chapman and Hall, London, 1980.

[8]

E. Knorr, R. Ng: "Algorithms for Mining Distance based Outliers in Large Datasets", Proc. of 24th Intl. Conf. On Very Large Data Bases, 1998, pp 392-403.

[9]

E. Knorr, R. Ng : "Finding Intensional Knowledge of Distance-based Outliers", Proc. of 25th Intl. Conf. On Very Large Data Bases, 1999, pp 211-222.

[10]

R. Ng, J. Han: "Efficient and Effective Clustering Methods for Spatial Data Mining", Proc. of 20th Intl. Conf. On Very Large Data Bases, 1994, pp 144-155.

[11]

S. Ramaswamy, R. Rastogi, S. Kyuseok: "Efficient Algorithms for Mining Outliers from Large Data Sets", Proc. of ACM SIGMOD Intl. Conf. On Management of Data, 2000, pp 427-438.

[12]

N. Roussopoulos, S. Kelley, F. Vincent, "Nearest Neighbor Queries", Proc. of ACM SIGMOD Intl. Conf. On Management of Data, 1995, pp 71-79.

[13]

G. Sheikholeslami, S. Chatterjee, A. Zhang: "WaveCluster: A multi-Resolution Clustering Approach for Very Large Spatial Databases", Proc. of 24th Intl. Conf. On Very Large Data Bases, 1998, pp 428-439.

[14]

S. Guha, R. Rastogi, K. Shim: "Cure: An Efficient Clustering Algorithm for Large Databases", In Proc. of the ACM SIGMOD Conf. On Management of Data, 1998, pp 73-84.

[15]

J. Tang, Z. Chen, A. Fu and D. Cheung: "A General Framework for Outlier Formulations: Density versus Connectivity", Manuscript.

[16]

T. Zhang, R. Ramakrishnan, M. Linvy: "BIRCH: An Efficient Data Clustering Method for Very Large Databases", Proc. of ACM SIGMOD Intl. Conf. On Management of Data, 1996, pp 103-114.

Cited By

Han SHu XHuang HJiang MZhao YKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)ADBenchProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3602599(32142-32159)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3602599
Baset ABecker CDerr KSarkar SKasera S(2022)AviSense: A Real-time System for Detection, Classification, and Analysis of Aviation SignalsACM Transactions on Sensor Networks10.1145/352608919:1(1-35)Online publication date: 8-Dec-2022
https://dl.acm.org/doi/10.1145/3526089
Zoppi TGharib MAtif MBondavalli A(2021)Meta-Learning to Improve Unsupervised Intrusion Detection in Cyber-Physical SystemsACM Transactions on Cyber-Physical Systems10.1145/34674705:4(1-27)Online publication date: 22-Sep-2021
https://dl.acm.org/doi/10.1145/3467470
Show More Cited By

Index Terms

Enhancing Effectiveness of Outlier Detections for Low Density Patterns
1. Information systems
  1. Information systems applications
    1. Data mining

Index terms have been assigned to the content through auto-classification.

Recommendations

Enhancing Outlier Detection by an Outlier Indicator
Machine Learning and Data Mining in Pattern Recognition
Abstract
Outlier detection is an important task in data mining and has high practical value in numerous applications such as astronomical observation, text detection, fraud detection and so on. At present, a large number of popular outlier detection ...
Enhancing Effectiveness of Density-Based Outlier Mining
ISIP '08: Proceedings of the 2008 International Symposiums on Information Processing

Outlier mining is an important work of data mining and a density-similarity-neighbor based outlier factor (DSNOF) algorithm is proposed to indicate the degree of outlier-ness of an object. The proposed algorithm calculates the densities of an object and ...
Enhancing effectiveness of density-based outlier mining scheme with density-similarity-neighbor-based outlier factor

This paper proposes a density-similarity-neighbor-based outlier mining algorithm for the data preprocess of data mining technique. First, the concept of k-density of an object is presented and the similar density series (SDS) of the object is ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

PAKDD '02: Proceedings of the 6th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining

May 2002

566 pages

ISBN:3540437045

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 06 May 2002

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

49
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Han SHu XHuang HJiang MZhao YKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)ADBenchProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3602599(32142-32159)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3602599
Baset ABecker CDerr KSarkar SKasera S(2022)AviSense: A Real-time System for Detection, Classification, and Analysis of Aviation SignalsACM Transactions on Sensor Networks10.1145/352608919:1(1-35)Online publication date: 8-Dec-2022
https://dl.acm.org/doi/10.1145/3526089
Zoppi TGharib MAtif MBondavalli A(2021)Meta-Learning to Improve Unsupervised Intrusion Detection in Cyber-Physical SystemsACM Transactions on Cyber-Physical Systems10.1145/34674705:4(1-27)Online publication date: 22-Sep-2021
https://dl.acm.org/doi/10.1145/3467470
Song HLi PLiu HZhu FChin Ooi BMiao CWang HSkrypnyk IHsu WChawla S(2021)Deep Clustering based Fair Outlier DetectionProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining10.1145/3447548.3467225(1481-1489)Online publication date: 14-Aug-2021
https://dl.acm.org/doi/10.1145/3447548.3467225
Zoppi TCeccarelli ACapecchi TBondavalli A(2021)Unsupervised Anomaly Detectors to Detect Intrusions in the Current Threat LandscapeACM/IMS Transactions on Data Science10.1145/34411402:2(1-26)Online publication date: 8-Apr-2021
https://dl.acm.org/doi/10.1145/3441140
Yang XZhou WShu NZhang H(2019)A Fast and Efficient Local Outlier Detection in Data StreamsProceedings of the 2019 International Conference on Image, Video and Signal Processing10.1145/3317640.3317653(111-116)Online publication date: 25-Feb-2019
https://dl.acm.org/doi/10.1145/3317640.3317653
Falcão FZoppi TSilva CSantos AFonseca BCeccarelli ABondavalli AHung CPapadopoulos G(2019)Quantitative comparison of unsupervised anomaly detection algorithms for intrusion detectionProceedings of the 34th ACM/SIGAPP Symposium on Applied Computing10.1145/3297280.3297314(318-327)Online publication date: 8-Apr-2019
https://dl.acm.org/doi/10.1145/3297280.3297314
Djenouri YZimek A(2018)Outlier Detection in Urban Traffic DataProceedings of the 8th International Conference on Web Intelligence, Mining and Semantics10.1145/3227609.3227692(1-12)Online publication date: 25-Jun-2018
https://dl.acm.org/doi/10.1145/3227609.3227692
Luo CShrivastava AChampin PGandon FMédini LLalmas MIpeirotis P(2018)Arrays of (locality-sensitive) Count Estimators (ACE)Proceedings of the 2018 World Wide Web Conference10.1145/3178876.3186056(1439-1448)Online publication date: 10-Apr-2018
https://dl.acm.org/doi/10.1145/3178876.3186056
He JXiong N(2018)An effective information detection method for social big dataMultimedia Tools and Applications10.1007/s11042-017-5523-y77:9(11277-11305)Online publication date: 1-May-2018
https://dl.acm.org/doi/10.1007/s11042-017-5523-y
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents