article

Kernel Fisher Discriminants for Outlier Detection

Author:

Volker RothAuthors Info & Claims

Neural Computation, Volume 18, Issue 4

Pages 942 - 960

https://doi.org/10.1162/089976606775774679

Published: 01 April 2006 Publication History

Abstract

The problem of detecting atypical objects or outliers is one of the classical topics in (robust) statistics. Recently, it has been proposed to address this problem by means of one-class SVM classifiers. The method presented in this letter bridges the gap between kernelized one-class classification and gaussian density estimation in the induced feature space. Having established the exact relation between the two concepts, it is now possible to identify atypical objects by quantifying their deviations from the gaussian model. This model-based formalization of outliers overcomes the main conceptual shortcoming of most one-class approaches, which, in a strict sense, are unable to detect outliers, since the expected fraction of outliers has to be specified in advance. In order to overcome the inherent model selection problem of unsupervised kernel methods, a cross-validated likelihood criterion for selecting all free model parameters is applied. Experiments for detecting atypical objects in image databases effectively demonstrate the applicability of the proposed method in real-world scenarios.

References

[1]

Duda, R., Hart, P., & Stork, D. (2001). Pattern classification. Hoboken, NJ: Wiley.

Crossref

Google Scholar

[2]

Fox, J. (1997). Applied regression, linear models, and related methods. Thousand Oaks, CA: Sage.

Google Scholar

[3]

Hastie, T., Buja, A., & Tibshirani, R. (1995). Penalized discriminant analysis. Annals of Statistics, 23, 73-102.

Crossref

Google Scholar

[4]

Huber, P. (1981). Robust statistics. Hoboken, NJ: Wiley.

Google Scholar

[5]

Kendall, M., & Stuart, A. (1977). The advanced theory of statistics (Vol. 1). New York: Macmillan.

Google Scholar

[6]

Lepage, G. (1980). Vegas: An adaptive multidimensional integration program (Tech. Rep. CLNS-80/447). Ithaca, NY: Cornell University.

Google Scholar

[7]

Mika, S., Rätsch, G., Weston, J., Schölkopf, B., & Müller, K.-R. (1999). Fisher discriminant analysis with kernels. In Y.-H. Hu, J. Larsen, E. Wilson, & S. Douglas (Eds.), Neural networks for signal processing IX (pp. 41-48). Piscataway, NJ: IEEE.

Google Scholar

[8]

Moody, J. (1992). The effective number of parameters: An analysis of generalization and regularization in nonlinear learning systems. In J. Moody, S. Hanson, & R. Lippmann (Eds.), Advances in neural information processing systems, 4 (pp. 847- 854). Cambridge, MA: MIT Press.

Google Scholar

[9]

Roth, V., & Steinhage, V. (2000). Nonlinear discriminant analysis using kernel functions. In S. Solla, T. Leen, & K.-R. Müller (Eds.), Advances in neural information processing systems, 12 (pp. 568-574). Cambridge, MA: MIT Press.

Google Scholar

[10]

Schölkopf, B., Mika, S., Burges, C., Knirsch, P., Müller, K.-R., Rätsch, G., & Smola, A. (1999). Input space vs. feature space in kernel-based methods. IEEE Trans. Neural Networks, 10(5), 1000-1017.

Digital Library

Google Scholar

[11]

Schölkopf, B., & Smola, A. (2002). Learning with kernels. Cambridge, MA: MIT Press.

Google Scholar

[12]

Schölkopf, B., Williamson, R., Smola, A., & Shawe-Taylor, J. (2000). SV estimation of a distribution's support. In S. Solla, T. Leen, & K.-R. Müller (Eds.), Advances in neural information processing systems, 12 (pp. 582-588). Cambridge, MA: MIT Press.

Google Scholar

[13]

Tax, D., & Duin, R. (1999). Support vector data description. Pattern Recognition Letters, 20(11-13), 1191-1199.

Digital Library

Google Scholar

[14]

van der Laan, M., Dudoit, S., & Keles, S. (2004). Asymptotic optimality of likelihood-based cross-validation. Statistical Applications in Genetics and Molecular Biology, 3(1), art. 4.

Google Scholar

Cited By

View all

Perez-Careta EHernández-Farías DGuzman-Sepulveda JCisneros MCordoba-Fraga TMartinez Espinoza JGuzman-Cabrera R(2022)One-class Classification for Identifying COVID-19 in X-Ray ImagesProgramming and Computing Software10.1134/S036176882204004148:4(235-242)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1134/S0361768822040041
Demertzis KIliadis LBougoudis I(2020)Gryphon: a semi-supervised anomaly detection system based on one-class evolving spiking neural networkNeural Computing and Applications10.1007/s00521-019-04363-x32:9(4303-4314)Online publication date: 1-May-2020
https://dl.acm.org/doi/10.1007/s00521-019-04363-x
Zhang XLei SZhao LBoedihardjo ALu C(2019)Robust Regression via Heuristic Corruption Thresholding and Its Adaptive Estimation VariationACM Transactions on Knowledge Discovery from Data10.1145/331410513:3(1-22)Online publication date: 7-Jun-2019
https://dl.acm.org/doi/10.1145/3314105
Show More Cited By

Index Terms

Kernel Fisher Discriminants for Outlier Detection
1. Computing methodologies
  1. Machine learning
  2. Modeling and simulation
    1. Model development and analysis
      1. Model verification and validation

Recommendations

Outlier detection with one-class kernel fisher discriminants
NIPS'04: Proceedings of the 17th International Conference on Neural Information Processing Systems

The problem of detecting "atypical objects" or "outliers" is one of the classical topics in (robust) statistics. Recently, it has been proposed to address this problem by means of one-class SVM classifiers. The main conceptual shortcoming of most one-...
Outlier Detection with Kernel Density Functions
MLDM '07: Proceedings of the 5th international conference on Machine Learning and Data Mining in Pattern Recognition

Outlier detection has recently become an important problem in many industrial and financial applications. In this paper, a novel unsupervised algorithm for outlier detection with a solid statistical foundation is proposed. First we modify a ...
Enhancing Outlier Detection by an Outlier Indicator
Machine Learning and Data Mining in Pattern Recognition
Abstract
Outlier detection is an important task in data mining and has high practical value in numerous applications such as astronomical observation, text detection, fraud detection and so on. At present, a large number of popular outlier detection ...

Comments

Information & Contributors

Information

Published In

Neural Computation Volume 18, Issue 4

April 2006

258 pages

ISSN:0899-7667

Issue’s Table of Contents

Publisher

MIT Press

Cambridge, MA, United States

Publication History

Published: 01 April 2006

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
38
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Perez-Careta EHernández-Farías DGuzman-Sepulveda JCisneros MCordoba-Fraga TMartinez Espinoza JGuzman-Cabrera R(2022)One-class Classification for Identifying COVID-19 in X-Ray ImagesProgramming and Computing Software10.1134/S036176882204004148:4(235-242)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1134/S0361768822040041
Demertzis KIliadis LBougoudis I(2020)Gryphon: a semi-supervised anomaly detection system based on one-class evolving spiking neural networkNeural Computing and Applications10.1007/s00521-019-04363-x32:9(4303-4314)Online publication date: 1-May-2020
https://dl.acm.org/doi/10.1007/s00521-019-04363-x
Zhang XLei SZhao LBoedihardjo ALu C(2019)Robust Regression via Heuristic Corruption Thresholding and Its Adaptive Estimation VariationACM Transactions on Knowledge Discovery from Data10.1145/331410513:3(1-22)Online publication date: 7-Jun-2019
https://dl.acm.org/doi/10.1145/3314105
Wen ZShi JLi QHe BChen J(2018)ThunderSVMThe Journal of Machine Learning Research10.5555/3291125.329114619:1(797-801)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.5555/3291125.3291146
Hamidzadeh JMoradi M(2018)Improved one-class classification using filled functionApplied Intelligence10.1007/s10489-018-1145-y48:10(3263-3279)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1007/s10489-018-1145-y
Alam MCalhoun VWang Y(2016)Influence Function of Multiple Kernel Canonical Analysis to Identify Outliers in Imaging Genetics DataProceedings of the 7th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics10.1145/2975167.2975189(210-219)Online publication date: 2-Oct-2016
https://dl.acm.org/doi/10.1145/2975167.2975189
Dufrenois FNoyer J(2016)One class proximal support vector machinesPattern Recognition10.1016/j.patcog.2015.09.03652:C(96-112)Online publication date: 1-Apr-2016
https://dl.acm.org/doi/10.1016/j.patcog.2015.09.036
Nguyen TNguyen ANguyen TVu LNguyen QHai LThang HPhuong LDe Raedt LDeville YBui MLinh TOanh NSang DNgoc N(2015)Unsupervised Anomaly Detection in Online GameProceedings of the 6th International Symposium on Information and Communication Technology10.1145/2833258.2833305(4-10)Online publication date: 3-Dec-2015
https://dl.acm.org/doi/10.1145/2833258.2833305
Ramirez-Padron RForegger DManuel JGeorgiopoulos MMederos B(2010)Similarity kernels for nearest neighbor-based outlier detectionProceedings of the 9th international conference on Advances in Intelligent Data Analysis10.1007/978-3-642-13062-5_16(159-170)Online publication date: 19-May-2010
https://dl.acm.org/doi/10.1007/978-3-642-13062-5_16
Chandola VBanerjee AKumar V(2009)Anomaly detectionACM Computing Surveys10.1145/1541880.154188241:3(1-58)Online publication date: 30-Jul-2009
https://dl.acm.org/doi/10.1145/1541880.1541882
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

Outlier detection with one-class kernel fisher discriminants

Outlier Detection with Kernel Density Functions

Enhancing Outlier Detection by an Outlier Indicator

Comments

Information

Published In

Publisher

Publication History

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations