Article

Weighting versus pruning in rule validation for detecting network and host anomalies

Authors:

Philip K. ChanAuthors Info & Claims

KDD '07: Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining

Pages 697 - 706

https://doi.org/10.1145/1281192.1281267

Published: 12 August 2007 Publication History

Abstract

For intrusion detection, the LERAD algorithm learns a succinct set of comprehensible rules for detecting anomalies, which could be novel attacks. LERAD validates the learned rules on a separate held-out validation set and removes rules that cause false alarms. However, removing rules with possible high coverage can lead to missed detections. We propose to retain these rules and associate weights to them. We present three weighting schemes and our empirical results indicate that, for LERAD, rule weighting can detect more attacks than pruning with minimal computational overhead.

References

[1]

S. Forrest and S. Hofmeyr and A.Somayaji and T. Longstaff. A Sense of Self for UNIX Processes. IEEE Security and Privacy. 1996.

Digital Library

[2]

W. Cohen. Fast Effective Rule Induction. ICML. 1995. 115--123.

[3]

H. Feng and O. Kolesnikov and P. Fogla and W. Lee and W. Gong. Anomaly Detection Using Call Stack Information. IEEE Security and Privacy 2003.

Digital Library

[4]

A. Ghosh and A. Schwartzbard. A Study in Using Neural Networks for Anomaly and Misuse Detection. USENIX Security Symposium. 1999

Digital Library

[5]

K. Kendell. A Database of Computer Attacks for the Evaluation of Intrusion Detection Systems. MIT. Cambridge, MA. 1999

[6]

R. Lippmann and J. Haines and D. Fried and J. Korba and K. Das. The 1999 DARPA Off-Line Intrusion Detection Evaluation, Computer Networks, 2000.

Digital Library

[7]

M. Mahoney and P. Chan. Learning Rules for Anomaly Detection of Hostile Network Traffic. ICDM. 2003.

Digital Library

[8]

R. Sekar and M. Bendre and D. Dhurjati and P. Bollineni. A Fast Automaton-based Method for Detecting Anomalous Program Behaviors. IEEE Security and Privacy, 2001.

Digital Library

[9]

C. Warrender and S. Forrest and B. Pearlmutter. Detecting Intrusions Using System Calls: Alternative Data Models. IEEE Security and Privacy, 1999.

[10]

A. Wespi and M. Dacier and H. Debar. Intrusion detection using variable--length audit trail patterns. Recent Advances in Intrusion Detection. 2000.

Digital Library

[11]

I. Witten and T. Bell. The zero-frequency problem: estimating the probabilities of novel events in adaptive text compression. IEEE Trans. Information Theory. 1991.

Digital Library

[12]

R. Agrawal and R. Srikant, Fast Algorithms for Mining Association Rules, VLDB, 1994.

Digital Library

[13]

A. Blum. Empirical Support for Winnow and Weighted--Majority Algorithms: Results on a Calendar Scheduling Domain. Machine Learning. 1997, 26, 5. 5--23.

Digital Library

[14]

Littlestone, N., Learning quickly when irrelevant attributes abound: A new linear threshold algorithm. Machine Learning. 2. 1988, 285--318.

Digital Library

[15]

Littlestone, N. and Warmuth, M.K. The weighted majority algorithm. Information and Computation. 108, 1994. 212--261. 2.

Digital Library

[16]

Robertson, W. and Vigna, G. and Kruegel, C. and Kemmerer, R. A., Using Generalization and Characterization Techniques in the Anomaly-based Detection of Web Attacks. NDSS, 2006.

[17]

Bhatkar, S. and Chaturvedi, A. and Sekar, R., Dataflow Anomaly Detection. IEEE Security and Privacy, 2006.

Digital Library

[18]

Gao, D. and Reiter, M. and Song, D., On gray-box program tracking for anomaly detection, USENIX Security Symposium, 2004.

Digital Library

[19]

Barbara, D. and Couto, J. and Jajodia, S. and Popyack, L. and Wu, N., ADAM: Detecting Intrusions by Data Mining, IEEE Workshop on Information Assurance and Security, 2001.

[20]

Roesch, M., Snort - Lightweight intrusion detection for networks, USENIX LISA, 1999.

Digital Library

[21]

Paxson, V., Bro: A system for detecting network intruders in real time, USENIX Security Symposium, 1998.

Digital Library

[22]

Anderson, D. and Lunt, T. F. and Javitz, H. and Tamaru, A. and Valdes, A., Detecting unusual program behavior using the statistical component of the Next generation Intrusion Detection Expert System (NIDES), Computer Science Laboratory SRI, "SRI-CSL-95-06", 1995.

[23]

Staniford, S. and Hoagland, J. A. and McAlerney, J. M., Practical automated detection of stealthy portscans, Journal of Computer Security, 10, 105--136, 2002.

Digital Library

[24]

Mutz, D. and Valeur, F. and Kruegel, C. and Vigna, G., Anomalous System Call Detection, ACM Trans. Information and System Security, 2006.

Digital Library

[25]

Krugel, C. and Toth, T. and Kirda, E., Service specific anomaly detection for network intrusion detection, ACM SAC, 2002.

Digital Library

[26]

Tandon, G. and Chan, P. K., On the learning of system call attributes for host--based anomaly detection, Intl. Journal on AI Tools, 15, 6, 875--892, 2006.

[27]

Paxson, V. and Floyd, S., The failure of Poisson modeling, IEEE/ACM Trans. Networking, 3, 226--244, 1995.

Digital Library

[28]

Flach, P .A., The many faces of ROC analysis in Machine Learning, ICML Tutorial, 2004.

Cited By

Wang CTang HZhu HZheng JJiang C(2024)Behavioral authentication for security and safetySecurity and Safety10.1051/sands/20240033(2024003)Online publication date: 30-Apr-2024
https://doi.org/10.1051/sands/2024003
Wei XWang JSun CTowey DZhang SZuo WYu YRuan RSong G(2024)Log‐based anomaly detection for distributed systems: State of the art, industry experience, and open issuesJournal of Software: Evolution and Process10.1002/smr.2650Online publication date: 7-Feb-2024
https://doi.org/10.1002/smr.2650
Deng XShi YYao D(2023)Outlier detection toward high-dimensional industrial data using extreme tensor-train learning machine with compressionJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2023.10157635:6(101576)Online publication date: Jun-2023
https://doi.org/10.1016/j.jksuci.2023.101576
Show More Cited By

Index Terms

Weighting versus pruning in rule validation for detecting network and host anomalies

Recommendations

Increasing coverage to improve detection of network and host anomalies

For intrusion detection, the LERAD algorithm learns a succinct set of comprehensible rules for detecting anomalies, which could be novel attacks. LERAD validates the learned rules on a separate held-out validation set and removes rules that cause false ...
Constructing Accurate Fuzzy Rule-Based Classification Systems Using Apriori Principles and Rule-Weighting
Intelligent Data Engineering and Automated Learning - IDEAL 2007
Abstract
A fuzzy rule-based classification system (FRBCS) is one of the most popular approaches used in pattern classification problems. One advantage of a fuzzy rule-based system is its interpretability. However, we’re faced with some challenges when ...
Using artificial anomalies to detect unknown and known network intrusions

Intrusion detection systems (IDSs) must be capable of detecting new and unknown attacks, or anomalies. We study the problem of building detection models for both pure anomaly detection and combined misuse and anomaly detection (i.e., detection of both ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '07: Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining

August 2007

1080 pages

ISBN:9781595936097

DOI:10.1145/1281192

General Chair:
Pavel Berkhin
Yahoo!, USA
,
Program Chairs:
Rich Caruana
Cornell University, USA
,
Xindong Wu
University of Vermont, USA

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 August 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

KDD07

Sponsor:

KDD07: The 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 12 - 15, 2007

California, San Jose, USA

Acceptance Rates

KDD '07 Paper Acceptance Rate 111 of 573 submissions, 19%;

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
751
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang CTang HZhu HZheng JJiang C(2024)Behavioral authentication for security and safetySecurity and Safety10.1051/sands/20240033(2024003)Online publication date: 30-Apr-2024
https://doi.org/10.1051/sands/2024003
Wei XWang JSun CTowey DZhang SZuo WYu YRuan RSong G(2024)Log‐based anomaly detection for distributed systems: State of the art, industry experience, and open issuesJournal of Software: Evolution and Process10.1002/smr.2650Online publication date: 7-Feb-2024
https://doi.org/10.1002/smr.2650
Deng XShi YYao D(2023)Outlier detection toward high-dimensional industrial data using extreme tensor-train learning machine with compressionJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2023.10157635:6(101576)Online publication date: Jun-2023
https://doi.org/10.1016/j.jksuci.2023.101576
Kim DChoi J(2022)Density-Based Geometric One-Class Classifier Combined with Genetic AlgorithmMathematical Problems in Engineering10.1155/2022/78524562022(1-18)Online publication date: 16-Apr-2022
https://doi.org/10.1155/2022/7852456
Dou SYang KPoor H(2019) PC 2 A: Predicting Collective Contextual Anomalies via LSTM With Deep Generative Model IEEE Internet of Things Journal10.1109/JIOT.2019.29302026:6(9645-9655)Online publication date: Dec-2019
https://doi.org/10.1109/JIOT.2019.2930202
Kong XBi YGlass D(2019)Detecting anomalies in sequential data augmented with new featuresArtificial Intelligence Review10.1007/s10462-018-9671-xOnline publication date: 3-Jan-2019
https://doi.org/10.1007/s10462-018-9671-x
Ku JZheng BYun D(2017)Intrusion Detection Based on Self-Adaptive Differential Evolutionary Extreme Learning Machine2017 International Conference on Computer Network, Electronic and Automation (ICCNEA)10.1109/ICCNEA.2017.57(94-100)Online publication date: Sep-2017
https://doi.org/10.1109/ICCNEA.2017.57
Ku JXing K(2017)Self-Adaptive Differential Evolutionary Extreme Learning Machine and Its Application in Facial Age Estimation2017 International Conference on Computer Network, Electronic and Automation (ICCNEA)10.1109/ICCNEA.2017.31(112-117)Online publication date: Sep-2017
https://doi.org/10.1109/ICCNEA.2017.31
Bigdeli EMohammadi MRaahemi BMatwin S(2017)A fast and noise resilient cluster-based anomaly detectionPattern Analysis & Applications10.1007/s10044-015-0484-020:1(183-199)Online publication date: 1-Feb-2017
https://dl.acm.org/doi/10.1007/s10044-015-0484-0
Ku JZheng B(2017)Intrusion Detection Based on Self-adaptive Differential Evolution Extreme Learning Machine with Gaussian KernelParallel Architecture, Algorithm and Programming10.1007/978-981-10-6442-5_2(13-24)Online publication date: 6-Oct-2017
https://doi.org/10.1007/978-981-10-6442-5_2
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents