research-article

Enhancing one-class support vector machines for unsupervised anomaly detection

Authors:

Mennatallah Amer,

Markus Goldstein,

Slim AbdennadherAuthors Info & Claims

ODD '13: Proceedings of the ACM SIGKDD Workshop on Outlier Detection and Description

Pages 8 - 15

https://doi.org/10.1145/2500853.2500857

Published: 11 August 2013 Publication History

Abstract

Support Vector Machines (SVMs) have been one of the most successful machine learning techniques for the past decade. For anomaly detection, also a semi-supervised variant, the one-class SVM, exists. Here, only normal data is required for training before anomalies can be detected. In theory, the one-class SVM could also be used in an unsupervised anomaly detection setup, where no prior training is conducted. Unfortunately, it turns out that a one-class SVM is sensitive to outliers in the data. In this work, we apply two modifications in order to make one-class SVMs more suitable for unsupervised anomaly detection: Robust one-class SVMs and eta one-class SVMs. The key idea of both modifications is, that outliers should contribute less to the decision boundary as normal instances. Experiments performed on datasets from UCI machine learning repository show that our modifications are very promising: Comparing with other standard unsupervised anomaly detection algorithms, the enhanced one-class SVMs are superior on two out of four datasets. In particular, the proposed eta one-class SVM has shown the most promising results.

References

[1]

Mennatallah Amer and Markus Goldstein. Nearest-neighbor and clustering based anomaly detection algorithms for rapidminer. Proc. of the 3rd RapidMiner Community Meeting and Conference (RCOMM 2012), pages 1--12, 2012.

[2]

Fabrizio Angiulli and Clara Pizzuti. Fast outlier detection in high dimensional spaces. In Tapio Elomaa, Heikki Mannila, and Hannu Toivonen, editors, Principles of Data Mining and Knowledge Discovery, volume 2431 of Lecture Notes in Computer Science, pages 43--78. Springer, 2002.

Digital Library

[3]

Arslan Basharat, Alexei Gritai, and Mubarak Shah. Learning object motion patterns for anomaly detection and improved object detection. CVPR, pages 1--8, 01 2008.

[4]

Christopher M. Bishop. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer, 1st ed. 2006. corr. 2nd printing 2011 edition, October 2007.

Digital Library

[5]

Markus M. Breunig, Hans-Peter Kriegel, Raymond T. Ng, and Jörg Sander. LOF: Identifying density-based local outliers. In Proc. of the 2000 ACM SIGMOD Int. Conf. on Management of Data, pages 93--104, Dallas, Texas, USA, 05 2000. ACM.

Digital Library

[6]

Christopher J. C. Burges. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2:121--167, 1998.

Digital Library

[7]

Varun Chandola, Arindam Banerjee, and Vipin Kumar. Anomaly detection: A survey. ACM Comput. Surv., 41(3):15:1--15:58, July 2009.

Digital Library

[8]

Paul F. Evangelista, Mark J. Embrechts, and Boleslaw K. Szymanski. Some properties of the gaussian kernel for one class learning. In Proc. of the 17th Int. Conf. on Artificial neural networks, ICANN'07, pages 269--278. Springer, 2007.

Digital Library

[9]

A. Frank and A. Asuncion. UCI machine learning repository, 2010.

[10]

Markus Goldstein and Andreas Dengel. Histogram-based outlier score (hbos): A fast unsupervised anomaly detection algorithm. In Stefan Wölfl, editor, KI-2012: Poster and Demo Track, pages 59--63. Online, 9 2012.

[11]

Simon Hawkins, Hongxing He, Graham Williams, and Rohan Baxter. Outlier detection using replicator neural networks. In In Proc. of the Fifth Int. Conf. and Data Warehousing and Knowledge Discovery (DaWaK02, pages 170--180, 2002.

Digital Library

[12]

Zengyou He, Xiaofei Xu, and Shengchun Deng. Discovering cluster-based local outliers. Pattern Recognition Letters, 24(9-10):1641--1650, 2003.

Digital Library

[13]

W. J. Hu and Q. Song. An accelerated decomposition algorithm for robust support vector machines. Circuits and Systems II: Express Briefs, IEEE Transactions on, 51(5):234--240, 2004.

[14]

Wen Jin, Anthony Tung, Jiawei Han, and Wei Wang. Ranking outliers using symmetric neighborhood relationship. In Wee-Keong Ng and et al., editors, Advances in Knowledge Discovery and Data Mining, volume 3918 of Lecture Notes in Computer Science, pages 577--593. Springer, 2006.

Digital Library

[15]

Hans-Peter Kriegel, Peer Kröger, Erich Schubert, and Arthur Zimek. Loop: local outlier probabilities. In Proceeding of the 18th ACM Conf. on Information and knowledge management, CIKM '09, pages 1649--1652, New York, NY, USA, 2009. ACM.

Digital Library

[16]

Pavel Laskov, Christin Schäfer, Igor V. Kotenko, and Klaus-Robert Müller. Intrusion detection in unlabeled data with quarter-sphere support vector machines. Praxis der Informationsverarbeitung und Kommunikation, 27(4):228--236, 2007.

[17]

Jessica Lin, Eamonn Keogh, Ada Fu, and Helga Van Herle. Approximations to magic: Finding unusual medical time series. In In 18th IEEE Symp. on Computer-Based Medical Systems (CBMS, pages 23--24, 2005.

Digital Library

[18]

Yi Liu and Yuan F. Zheng. Minimum enclosing and maximum excluding machine for pattern description and discrimination. Pattern Recognition, International Conference on, 3:129--132, 2006.

Digital Library

[19]

I. Mierswa, M. Wurst, R. Klinkenberg, M. Scholz, and T. Euler. Yale (now: Rapidminer): Rapid prototyping for complex data mining tasks. In Proc. of the ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining (KDD 2006), 2006.

Digital Library

[20]

S. Mukkamala, G. Janoski, and A. Sung. Intrusion detection using neural networks and support vector machines. Proc. of the 2002 Int. Joint Conf. on Neural Networks. IJCNN'02 (Cat. No.02CH37290), pages 1702--1707, 2002.

[21]

Spiros Papadimitriou, Hiroyuki Kitagawa, Phillip B. Gibbons, and Christos Faloutsos. Loci: Fast outlier detection using the local correlation integral. Data Engineering, Int. Conf. on, 0:315, 2003.

[22]

Leonid Portnoy, Eleazar Eskin, and Sal Stolfo. Intrusion detection with unlabeled data using clustering. In In Proc. of ACM CSS Workshop on Data Mining Applied to Security (DMSA-2001), pages 5--8, 2001.

[23]

Sridhar Ramaswamy, Rajeev Rastogi, and Kyuseok Shim. Efficient algorithms for mining outliers from large data sets. In Proc. of the 2000 ACM SIGMOD Int. Conf. on Management of Data, SIGMOD '00, pages 427--438, New York, NY, USA, 2000. ACM.

Digital Library

[24]

B Schölkopf, J C Platt, J Shawe-Taylor, a J Smola, and R C Williamson. Estimating the support of a high-dimensional distribution. Neural computation, 13(7):1443--71, July 2001.

Digital Library

[25]

Bernhard Schölkopf, Robert C. Williamson, Alex J. Smola, John Shawe-Taylor, and John C. Platt. Support vector method for novelty detection. In Advances in Neural Information Processing Systems 12, (NIPS) Conf., pages 582--588. The MIT Press, 11 1999.

[26]

Qing Song, Wenjie Hu, and Wenfang Xie. Robust support vector machine with bullet hole image classification. Systems, Man and Cybernetics, Part C, IEEE Transactions on, 32(4):440--448, 2002.

Digital Library

[27]

Jian Tang, Zhixiang Chen, Ada Fu, and David Cheung. Enhancing effectiveness of outlier detections for low density patterns. In Ming-Syan Chen, Philip Yu, and Bing Liu, editors, Advances in Knowledge Discovery and Data Mining, volume 2336 of Lecture Notes in Computer Science, pages 535--548. Springer, 2002.

Digital Library

[28]

David M. J. Tax and Robert P. W. Duin. Support vector domain description. Pattern Recognition Letters, 20:1191--1199, 1999.

Digital Library

[29]

Joost van Beusekom and Faisal Shafait. Distortion measurement for automatic document verification. In Proc. of the 11th Int. Conf. on Document Analysis and Recognition. IEEE, 9 2011.

Digital Library

[30]

V. Vapnik and A. Lerner. Pattern recognition using generalized portrait method. Automation and Remote Control, 24:774--780, 1963.

[31]

Vladimir N. Vapnik. Statistical learning theory. Wiley, 1 edition, September 1998.

[32]

Wenjian Wang, Zongben Xu, Weizhen Lu, and Xiaoyun Zhang. Determination of the spread parameter in the gaussian kernel for classification and regression. Neurocomputing, 55(3-4):643--663, October 2003.

[33]

Linli Xu, K Crammer, and Dale Schuurmans. Robust support vector machine training via convex outlier ablation. Proc. of the National Conf. On Artificial Intelligence, pages 536--542, 2006.

Digital Library

[34]

Alan Yuille and Anand Rangarajan. The concave-convex procedure (cccp. In Advances in Neural Information Processing Systems 14. MIT Press, 2002.

[35]

Tong Zhang. Analysis of multi-stage convex relaxation for sparse regularization. J. Mach. Learn. Res., 11:1081--1107, March 2010.

Digital Library

[36]

Xi-chuan Zhou, Hai-bin Shen, and Jie-ping Ye. Integrating outlier filtering in large margin training. Journal of Zhejiang University-Science C, 12(5):362--370, May 2011.

Cited By

Cariello NEslinger RGallagher RKurtzer IGasti PBalagani K(2025)Posture and Body Movement Effects on Behavioral Biometrics for Continuous Smartphone AuthenticationIEEE Transactions on Biometrics, Behavior, and Identity Science10.1109/TBIOM.2024.34093497:1(3-15)Online publication date: Jan-2025
https://doi.org/10.1109/TBIOM.2024.3409349
Rasmussen TAlgán FBaum A(2025)Anomaly detection in broadband networks: Using normalizing flows for multivariate time seriesSignal Processing10.1016/j.sigpro.2024.109874230(109874)Online publication date: May-2025
https://doi.org/10.1016/j.sigpro.2024.109874
Guimarães VSousa ICunha RMagalhães RMachado ÁFernandes VReis SCorreia M(2025)One-class classification with confound control for cognitive screening in older adults using gait, fingertapping, cognitive, and dual tasksComputer Methods and Programs in Biomedicine10.1016/j.cmpb.2024.108508259(108508)Online publication date: Mar-2025
https://doi.org/10.1016/j.cmpb.2024.108508
Show More Cited By

Recommendations

Unsupervised Parameter Estimation for One-Class Support Vector Machines
PAKDD 2016: Proceedings, Part II, of the 20th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining - Volume 9652

Although the hyper-plane based One-Class Support Vector Machine OCSVM and the hyper-spherical based Support Vector Data Description SVDD algorithms have been shown to be very effective in detecting outliers, their performance on noisy and unlabeled ...
Centered hyperspherical and hyperellipsoidal one-class support vector machines for anomaly detection in sensor networks

Anomaly detection in wireless sensor networks is an important challenge for tasks such as intrusion detection and monitoring applications. This paper proposes two approaches to detecting anomalies from measurements from sensor networks. The first ...
ONE-CLASS SUPPORT VECTOR MACHINES APPROACH TO ANOMALY DETECTION

This article presents two-class and one-class support vector machines SVM for detection of fraudulent credit card transactions. One-class SVM classification with different kernels is considered for a dataset of fraudulent credit card transactions ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ODD '13: Proceedings of the ACM SIGKDD Workshop on Outlier Detection and Description

August 2013

53 pages

ISBN:9781450323352

DOI:10.1145/2500853

General Chairs:
Leman Akoglu
Stony Brook University
,
Emmanuel Müller
Karlsruhe Institute of Technology
,
Jilles Vreeken
Universiteit Antwerpen

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 August 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Stiftung Rheinland-Pfalz für Innovation

Conference

KDD' 13

Sponsor:

KDD' 13: The 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 11, 2013

Illinois, Chicago

Acceptance Rates

ODD '13 Paper Acceptance Rate 6 of 9 submissions, 67%;

Overall Acceptance Rate 6 of 9 submissions, 67%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

290
Total Citations
View Citations
2,547
Total Downloads

Downloads (Last 12 months)296
Downloads (Last 6 weeks)27

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cariello NEslinger RGallagher RKurtzer IGasti PBalagani K(2025)Posture and Body Movement Effects on Behavioral Biometrics for Continuous Smartphone AuthenticationIEEE Transactions on Biometrics, Behavior, and Identity Science10.1109/TBIOM.2024.34093497:1(3-15)Online publication date: Jan-2025
https://doi.org/10.1109/TBIOM.2024.3409349
Rasmussen TAlgán FBaum A(2025)Anomaly detection in broadband networks: Using normalizing flows for multivariate time seriesSignal Processing10.1016/j.sigpro.2024.109874230(109874)Online publication date: May-2025
https://doi.org/10.1016/j.sigpro.2024.109874
Guimarães VSousa ICunha RMagalhães RMachado ÁFernandes VReis SCorreia M(2025)One-class classification with confound control for cognitive screening in older adults using gait, fingertapping, cognitive, and dual tasksComputer Methods and Programs in Biomedicine10.1016/j.cmpb.2024.108508259(108508)Online publication date: Mar-2025
https://doi.org/10.1016/j.cmpb.2024.108508
Moroke NMakatjane K(2024)Detecting Aberrations in Renewable Energy With the One-Class Support Vector Machine ModelMachine Learning and Computer Vision for Renewable Energy10.4018/979-8-3693-2355-7.ch011(186-205)Online publication date: 17-May-2024
https://doi.org/10.4018/979-8-3693-2355-7.ch011
Bai KFossaceca J(2024)EM-AUC: A Novel Algorithm for Evaluating Anomaly Based Network Intrusion Detection SystemsSensors10.3390/s2501007825:1(78)Online publication date: 26-Dec-2024
https://doi.org/10.3390/s25010078
Choudhary AVuković MMutlu BHaslgrübler MKern R(2024)Interpretability of Causal Discovery in Tracking Deterioration in a Highly Dynamic ProcessSensors10.3390/s2412372824:12(3728)Online publication date: 8-Jun-2024
https://doi.org/10.3390/s24123728
Hrast TAhlström DHitz M(2024)Observations and Considerations for Implementing Vibration Signals as an Input Technique for Mobile DevicesMultimodal Technologies and Interaction10.3390/mti80900768:9(76)Online publication date: 2-Sep-2024
https://doi.org/10.3390/mti8090076
Kim SPark JAhn HLee Y(2024)Detection of Korean Phishing Messages Using Biased Discriminant Analysis under Extreme Class Imbalance ProblemInformation10.3390/info1505026515:5(265)Online publication date: 7-May-2024
https://doi.org/10.3390/info15050265
Bondok AAbdelsalam OBadr MMahmoud MAlsabaan MAlsaqhan MIbrahem M(2024)Accurate Power Consumption Predictor and One-Class Electricity Theft Detector for Smart Grid “Change-and-Transmit” Advanced Metering InfrastructureApplied Sciences10.3390/app1420930814:20(9308)Online publication date: 12-Oct-2024
https://doi.org/10.3390/app14209308
Chen ZJia DSun YYang LJin WLiu R(2024)Univariate Time Series Anomaly Detection Based on Hierarchical Attention NetworkTsinghua Science and Technology10.26599/TST.2023.901007329:4(1181-1193)Online publication date: Aug-2024
https://doi.org/10.26599/TST.2023.9010073
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten