article

Applying data mining to intrusion detection: the quest for automation, efficiency, and credibility

Author:

Wenke LeeAuthors Info & Claims

ACM SIGKDD Explorations Newsletter, Volume 4, Issue 2

Pages 35 - 42

https://doi.org/10.1145/772862.772868

Published: 01 December 2002 Publication History

Abstract

Intrusion detection is an essential component of the layered computer security mechanisms. It requires accurate and efficient models for analyzing a large amount of system and network audit data. This paper is an overview of our research in applying data mining techniques to build intrusion detection models. We describe a framework for mining patterns from system and network audit data, and constructing features according to analysis of intrusion patterns. We discuss approaches for improving the run-time efficiency as well as the credibility of detection models. We report the ideas, algorithms, and prototype systems we have developed, and discuss open research problems.

References

[1]

R. Agrawal, T. Imielinski, and A. Swami. Mining association rules between sets of items in large databases. In Proceedings of the ACM SIGMOD Conference on Management of Data, pages 207--216, 1993.]]

Digital Library

[2]

D. Anderson, T. Frivold, and A. Valdes. Next-generation intrusion detection expert system (NIDES): A summary. Technical Report SRI-CSL-95-07, Computer Science Laboratory, SRI International, Menlo Park, California, May 1995.]]

[3]

S. Axelsson. The base-rate fallacy and the difficulty of intrusion detection. ACM Transactions on Information and System Security, 3(3), 2000.]]

Digital Library

[4]

W. W. Cohen. Fast effective rule induction. In Machine Learning: the 12th International Conference, Lake Taho, CA, 1995. Morgan Kaufmann.]]

[5]

Wei Fan, Wenke Lee, Sal Stolfo, and Matt Miller. A multiple model cost-sensitive approach for intrusion detection. In Proceedings of The Eleventh European Conference on Machine Learning (ECML 2000), Lecture Notes in Artificial Intelligence No. 1810, Barcelona, Spain, May 2000.]]

Digital Library

[6]

A. K. Ghosh and A. Schwartzbard. A study in using neural networks for anomaly and misuse detection. In Proceedings of the 8th USENIX Security symposium, August 1999.]]

Digital Library

[7]

K. Ilgun, R. A. Kemmerer, and P. A. Porras. State transition analysis: A rule-based intrusion detection approach. IEEE Transactions on Software Engineering, 21(3):181--199, March 1995.]]

Digital Library

[8]

S. Kumar and E. H. Spafford. A software architecture to support misuse intrusion detection. In Proceedings of the 18th National Information security Conference, pages 194--204, 1995.]]

[9]

T. Lane and C. E. Brodley. Temporal sequence learning and data reduction for anomaly detection. ACM Transactions on Information and System Security, 2(3):295--331, August 1999.]]

Digital Library

[10]

W. Lee. A Data Mining Framework for Constructing Features and Models for Intrusion Detection Systems. PhD thesis, Columbia University, June 1999.]]

Digital Library

[11]

W. Lee and S. J. Stolfo. A framework for constructing features and models for intrusion detection systems. ACM Transactions on Information and System Security, 3(4), November 2000.]]

Digital Library

[12]

W. Lee, S. J. Stolfo, and K. W. Mok. Mining audit data to build intrusion detection models. In Proceedings of the 4th International Conference on Knowledge Discovery and Data Mining, New York, NY, August 1998. AAAI Press.]]

[13]

W. Lee, S. J. Stolfo, and K. W. Mok. Mining in a data-flow environment: Experience in network intrusion detection. In Proceedings of the 5th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD-99), August 1999.]]

Digital Library

[14]

R. Lippmann, D. Fried, I. Graf, J. Haines, K. Kendall, D. McClung, D. Weber, S. Webster, D. Wyschogrod, R. Cunninghan, and M. Zissman. Evaluating intrusion detection systems: The 1998 DARPA off-line intrusion detection evaluation. In Proceedings of the 2000 DARPA Information Survivability Conference and Exposition, January 2000.]]

Digital Library

[15]

H. Mannila, H. Toivonen, and A. I. Verkamo. Discovering frequent episodes in sequences. In Proceedings of the 1st International Conference on Knowledge Discovery in Databases and Data Mining, Montreal, Canada, August 1995.]]

[16]

S. McCanne, C. Leres, and V. Jacobson. libpcap. available via anonymous ftp to ftp.ee.lbl.gov, 1994.]]

[17]

F. Provost, D. Jensen, and T. Oates. Efficient progressive sampling. In Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. AAAI Press, August 1999.]]

Digital Library

[18]

B. Schneier. Secrets & Lies: Digital Security in a Networked World. John Wiley & Sons, Inc., 2000.]]

Digital Library

[19]

S.J. Stolfo, W. Lee, P.K. Chan, W. Fan, and E. Eskin. Data mining-based intrusion detectors: An overview of the Columbia IDS project. ACM SIGMOD Record, 30(4), December 2001.]]

Digital Library

[20]

SunSoft. SunSHIELD Basic Security Module Guide. SunSoft, Mountain View, CA, 1995.]]

[21]

P. Viola and M. Jones. Robust real-time object detection. In Proceedings of the Second International Workshop on Statistical and Computational Theories of Vison - Modeling, Learning, Computing, and Sampling, May 2002.]]

[22]

C. Warrender, S. Forrest, and B. Pearlmutter. Detecting intrusions using system calls: Alternative data models. In Proceedings of the 1999 IEEE Symposium on Security and Privacy, May 1999.]]

Cited By

Mironeanu CArchip AAtomei G(2022)Application of Association Rule Mining in Preventing CyberattacksBulletin of the Polytechnic Institute of Iași. Electrical Engineering, Power Engineering, Electronics Section10.2478/bipie-2021-002067:4(25-41)Online publication date: 22-Sep-2022
https://doi.org/10.2478/bipie-2021-0020
Mironeanu CArchip AAmarandei CCraus M(2021)Experimental Cyber Attack Detection FrameworkElectronics10.3390/electronics1014168210:14(1682)Online publication date: 14-Jul-2021
https://doi.org/10.3390/electronics10141682
Kahles JTorronen JHuuhtanen TJung A(2019)Automating Root Cause Analysis via Machine Learning in Agile Software Testing Environments2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST)10.1109/ICST.2019.00047(379-390)Online publication date: Apr-2019
https://doi.org/10.1109/ICST.2019.00047
Show More Cited By

Recommendations

A framework for constructing features and models for intrusion detection systems

Intrusion detection (ID) is an important component of infrastructure protection mechanisms. Intrusion detection systems (IDSs) need to be accurate, adaptive, and extensible. Given these requirements and the complexities of today's network environments, ...
Rule generalisation in intrusion detection systems using SNORT

Intrusion Detection Systems (IDSs) provide an important layer of security for computer systems and networks. An IDS's responsibility is to detect suspicious or unacceptable system and network activity and to alert a systems administrator to this ...
Network Intrusion Detection: Automated and Manual Methods Prone to Attack and Evasion

In this article, the authors describe common intrusion detection techniques, NIDS evasion methods, and how NIDSs detect intrusions. Additionally, we introduce new evasion methods, present test results for confirming attack outcomes based on server ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGKDD Explorations Newsletter

ACM SIGKDD Explorations Newsletter Volume 4, Issue 2

December 2002

127 pages

ISSN:1931-0145

EISSN:1931-0153

DOI:10.1145/772862

Issue’s Table of Contents

Copyright © 2002 Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 December 2002

Published in SIGKDD Volume 4, Issue 2

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
826
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mironeanu CArchip AAtomei G(2022)Application of Association Rule Mining in Preventing CyberattacksBulletin of the Polytechnic Institute of Iași. Electrical Engineering, Power Engineering, Electronics Section10.2478/bipie-2021-002067:4(25-41)Online publication date: 22-Sep-2022
https://doi.org/10.2478/bipie-2021-0020
Mironeanu CArchip AAmarandei CCraus M(2021)Experimental Cyber Attack Detection FrameworkElectronics10.3390/electronics1014168210:14(1682)Online publication date: 14-Jul-2021
https://doi.org/10.3390/electronics10141682
Kahles JTorronen JHuuhtanen TJung A(2019)Automating Root Cause Analysis via Machine Learning in Agile Software Testing Environments2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST)10.1109/ICST.2019.00047(379-390)Online publication date: Apr-2019
https://doi.org/10.1109/ICST.2019.00047
Tan WFang XZhao LTang A(2018)Anomaly Detection Algorithm Based on Cluster of EntropyComputer Supported Cooperative Work and Social Computing10.1007/978-981-13-3044-5_26(359-370)Online publication date: 11-Dec-2018
https://doi.org/10.1007/978-981-13-3044-5_26
Yin CFeng LMa L(2016)An improved Hoeffding-ID data-stream classification algorithmThe Journal of Supercomputing10.1007/s11227-015-1573-y72:7(2670-2681)Online publication date: 1-Jul-2016
https://dl.acm.org/doi/10.1007/s11227-015-1573-y
He Z(2015)Research on Network Intrusion Detection Based on Data Mining TechnologyApplied Mechanics and Materials10.4028/www.scientific.net/AMM.713-715.2081713-715(2081-2084)Online publication date: Jan-2015
https://doi.org/10.4028/www.scientific.net/AMM.713-715.2081
Boukhtouta AMokhov SLakhdari NDebbabi MPaquet J(2015)Network malware classification comparison using DPI and flow packet headersJournal of Computer Virology and Hacking Techniques10.1007/s11416-015-0247-x12:2(69-100)Online publication date: 29-Jul-2015
https://doi.org/10.1007/s11416-015-0247-x
Corona IGiacinto GRoli F(2013)Adversarial attacks against intrusion detection systemsInformation Sciences: an International Journal10.1016/j.ins.2013.03.022239(201-225)Online publication date: 1-Aug-2013
https://dl.acm.org/doi/10.1016/j.ins.2013.03.022
Cui XBeaver JPotok TYang L(2011)Visual Mining Intrusion Behaviors by Using Swarm TechnologyProceedings of the 2011 44th Hawaii International Conference on System Sciences10.1109/HICSS.2011.486(1-7)Online publication date: 4-Jan-2011
https://dl.acm.org/doi/10.1109/HICSS.2011.486
Stearley JCorwell SLord KBronevetsky GMohror KZheng A(2010)Bridging the gapsProceedings of the 2010 workshop on Managing systems via log analysis and machine learning techniques10.5555/1928991.1929003(8-8)Online publication date: 3-Oct-2010
https://dl.acm.org/doi/10.5555/1928991.1929003
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents