Abstract
Outlier detection is to find objects that do not comply with the general behavior of the data. Partition is a kind of method of dividing data space into a set of non-overlapping rectangular cells. There exists very large data skew in real-life datasets so that partition will produce many empty cells. The cell-based algorithms for outlier detection don’t get enough attention to the existence of many empty cells, which affects the efficiency of algorithms. In this paper, we propose the concept of Skew of Data (SOD) to measure the degree of data skew, and which approximates the percentage of empty cells under a partition of a dataset. An efficient index structure called CD-Tree and the related algorithms are designed. This paper applies the CD-Tree to detect outliers. Compared with cell-based algorithms on real-life datasets, the speed of CD-Tree-based algorithm increases 4 times at least and that the number of dimensions processed also increases obviously.
Supported by the National Natural Science Foundation of China under Grant No.60173051, the Teaching and Research Award Program for Outstanding Young Teachers in Higher Education Institutions of the Ministry of Education, China.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Hawkins, D.: Identification of Outliers. Chapman and Hall, London (1980)
Knorr, E., Ng, R.: Algorithms for Mining Distance-based Outliers in Large Data Sets. In: VLDB 1998, pp. 392–430 (1998)
Barnett, V., Lewis, T.: Outliers in Statistical Data. John Wiley and Sons, New York (1994)
Knorr, E., Ng, R.: Finding Intensional Knowledge of Distance-based Outliers. In: VLDB 1999, pp. 211–222 (1999)
Ramaswamy, S., Rastogi, R., Shim, K.: Efficient Algorithms for Mining Outliers from Large Data Sets. In: SIGMOD 2000, pp. 427–438 (2000)
Breunig, M.M., Kriegel, H.-P., Ng, R., Sander, J.: LOF: Identifying Density-Based Local Outliers. In: ACM SIGMOD 2000, pp. 93–104 (2000)
Agarwal, C.C., Yu, P.: Outlier Detection for High Dimensional Data. In: SIGMOD 2001 (2001)
Blake, C.L., Merz, C.J.: UCI Repository of machine learning databases, University of California, Department of Information and Computer Science, Irvine (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
Berchtold, S., Keim, D.A., Kriegel, H.: The X-tree: An Index Structure for Highdimensional Data. In: VLDB 1996, pp. 28–39 (1996)
Agrawal, R., Gehrke, J., Gunopulos, D., Raghavan, P.: Automatic subspace clustering of high dimensional data for data mining applications. In: ACM SIGMOD 1998, pp. 94–105 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sun, H., Bao, Y., Zhao, F., Yu, G., Wang, D. (2004). CD-Trees: An Efficient Index Structure for Outlier Detection. In: Li, Q., Wang, G., Feng, L. (eds) Advances in Web-Age Information Management. WAIM 2004. Lecture Notes in Computer Science, vol 3129. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-27772-9_60
Download citation
DOI: https://doi.org/10.1007/978-3-540-27772-9_60
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22418-1
Online ISBN: 978-3-540-27772-9
eBook Packages: Springer Book Archive