research-article

Active learning for network intrusion detection

Authors:

Ulf BrefeldAuthors Info & Claims

AISec '09: Proceedings of the 2nd ACM workshop on Security and artificial intelligence

Pages 47 - 54

https://doi.org/10.1145/1654988.1655002

Published: 09 November 2009 Publication History

Abstract

Anomaly detection for network intrusion detection is usually considered an unsupervised task. Prominent techniques, such as one-class support vector machines, learn a hypersphere enclosing network data, mapped to a vector space, such that points outside of the ball are considered anomalous. However, this setup ignores relevant information such as expert and background knowledge. In this paper, we rephrase anomaly detection as an active learning task. We propose an effective active learning strategy to query low-confidence observations and to expand the data basis with minimal labeling effort. Our empirical evaluation on network intrusion detection shows that our approach consistently outperforms existing methods in relevant scenarios.

References

[1]

M. Almgren and E. Jonsson. Using active learning in intrusion detection. Proc. IEEE Computer Security Foundation Workshop, 2004.

Digital Library

[2]

O. Chapelle and A. Zien. Semi-supervised classification by low density separation. In Proceedings of the International Workshop on AI and Statistics, 2005.

[3]

M. Damashek. Gauging similarity with n-grams: Language-independent categorization of text. Science, 267(5199):843--848, 1995.

[4]

P. Fogla, M. Sharif, R. Perdisci, O. Kolesnikov, and W. Lee. Polymorphic blending attacks. In Proceedings of USENIX Security Symposium, 2006.

Digital Library

[5]

C.-H. Hoi, C.-H. Chan, K. Huang, M. Lyu, and I. King. Support vector machines for class representation and discrimination. In Proceedings of the International Joint Conference on Neural Networks, 2003.

[6]

K. L. Ingham, A. Somayaji, J. Burge, and S. Forrest. Learning DFA representations of HTTP for protecting web applications. Computer Networks, 51(5):1239--1255, 2007.

Digital Library

[7]

M. Kloft and P. Laskov. A poisoning attack against online anomaly detection. In NIPS Workshop on Machine Learning in Adversarial Environments for Computer Security, 2007.

[8]

C. Kruegel, G. Vigna, and W. Robertson. A multi-model approach to the detection of web-based attacks. Computer Networks, 48(5), 2005.

Digital Library

[9]

Y. Liu and Y. F. Zheng. Minimum enclosing and maximum excluding machine for pattern description and discrimination. In ICPR '06: Proceedings of the 18th International Conference on Pattern Recognition, pages 129--132, Washington, DC, USA, 2006. IEEE Computer Society.

Digital Library

[10]

M. Mahoney. Network traffic anomaly detection based on packet bytes. In Proc. of ACM Symposium on Applied Computing, pages 346--350, 2003.

Digital Library

[11]

M. Mahoney and P. Chan. PHAD: Packet header anomaly detection for identifying hostile network traffic. Technical Report CS-2001-2, Florida Institute of Technology, 2001.

[12]

K. Maynor, K. Mookhey, J. F. R. Cervini, and K. Beaver. Metasploit toolkit. In Syngress, 2007.

[13]

Microsoft. Microsoft security intelligence report: January to June 2008. Microsoft Corporation, 2008.

[14]

D. Moore, C. Shannon, and J. Brown. Code-Red: a case study on the spread and victims of an internet worm. In Proc. of Internet Measurement Workshop (IMW), pages 273--284, 2002.

Digital Library

[15]

D. Pelleg and A. Moore. Active learning for anomaly and rare-category detection. Proc. Advances in Neural Information Processing Systems, 2004.

[16]

R. Perdisci, D. Ariu, P. Fogla, G. Giacinto, and W. Lee. McPAD: A multiple classifier system for accurate payload-based anomaly detection. Computer Networks, 2009. in press.

Digital Library

[17]

R. Perdisci, D. Dagon, W. Lee, P. Fogla, and M. Sharif. Misleading worm signature generators using deliberate noise injection. In Proc. of IEEE Symposium on Security and Privacy, pages 17--31, 2006.

Digital Library

[18]

J. C. Platt. Fast training of support vector machines using sequential minimal optimization. In Advances in kernel methods: support vector learning, 1999.

Digital Library

[19]

K. Rieck and P. Laskov. Detecting unknown network attacks using language models. In Detection of Intrusions and Malware, and Vulnerability Assessment, Proc. of 3rd DIMVA Conference, LNCS, pages 74--90, July 2006.

Digital Library

[20]

K. Rieck and P. Laskov. Language models for detection of unknown attacks in network traffic. Journal in Computer Virology, 2(4):243--256, 2007.

[21]

K. Rieck, S. Wahl, P. Laskov, P. Domschitz, and K.-R. Müller. A self-learning system for detection of anomalous sip messages. In Principles, Systems and Applications of IP Telecommunications (IPTCOMM), Second International Conference, LNCS, pages 90--106, 2008.

Digital Library

[22]

G. Salton, A. Wong, and C. Yang. A vector space model for automatic indexing. Communications of the ACM, 18(11):613--620, 1975.

Digital Library

[23]

B. Schölkopf, J. Platt, J. Shawe-Taylor, A. Smola, and R. Williamson. Estimating the support of a high-dimensional distribution. Neural Computation, 13(7):1443--1471, 2001.

Digital Library

[24]

C. Shannon and D. Moore. The spread of the Witty worm. IEEE Security and Privacy, 2(4):46--50, 2004.

Digital Library

[25]

Y. Song, M. Locasto, A. Stavrou, A. Keromytis, and S. Stolfo. On the infeasibility of modeling polymorphic shellcode. In Conference on Computer and Communications Security (CCS), pages 541--551, 2007.

Digital Library

[26]

J. W. Stokes and J. C. Platt. Aladin: Active learning of anomalies to detect intrusion. Technical report, Microsoft Research, 2008.

[27]

Symantec. Symantex report on the underground economy: July 07 to June 08. Symantec Corporation, 2008.

[28]

D. M. Tax and R. P. Duin. Support vector data description. Machine Learning, 54:45--66, 2004.

Digital Library

[29]

D. M. Tax. One-class classification. PhD thesis, Technical University Delft, 2001.

[30]

J. Wang, P. Neskovic, and L. N. Cooper. Pattern classification via single spheres. Computer Science: Discovery Science (DS), 2005.

Digital Library

[31]

K. Wang, J. Parekh, and S. Stolfo. Anagram: A content anomaly detector resistant to mimicry attack. In Recent Adances in Intrusion Detection (RAID), pages 226--248, 2006.

Digital Library

[32]

K. Wang and S. Stolfo. Anomalous payload-based network intrusion detection. In Recent Adances in Intrusion Detection (RAID), pages 203--222, 2004.

[33]

M. K. Warmuth, J. Liao, G. Rätsch, M. Mathieson, S. Putta, and C. Lemmen. Active learning with support vector machines in the drug discovery process. Journal of Chemical Information and Computer Sciences, 43(2):667--673, 2003.

[34]

A. Zien, U. Brefeld, and T. Scheffer. Transductive support vector machines for structured variables. In Proceedings of the International Conference on Machine Learning, 2007.

Digital Library

Cited By

Wu LXie YLi JFeng DLiang JWu Y(2025)Angus: efficient active learning strategies for provenance based intrusion detectionCybersecurity10.1186/s42400-024-00311-y8:1Online publication date: 27-Jan-2025
https://doi.org/10.1186/s42400-024-00311-y
Makris IKarampasi ARadoglou-Grammatikis PEpiskopos NIturbe ERios EPiperigkos NLalos AXenakis CLagkas TArgyriou VSarigiannidis P(2025)A comprehensive survey of Federated Intrusion Detection Systems: Techniques, challenges and solutionsComputer Science Review10.1016/j.cosrev.2024.10071756(100717)Online publication date: May-2025
https://doi.org/10.1016/j.cosrev.2024.100717
Zhang YDong HNottingham ABuchanan MBrown DSun Y(2023)Global Analysis with Aggregation-based Beaconing Detection across Large Campus NetworksProceedings of the 39th Annual Computer Security Applications Conference10.1145/3627106.3627126(565-579)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1145/3627106.3627126
Show More Cited By

Index Terms

Active learning for network intrusion detection
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Classification and regression trees
      2. Markov decision processes
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Markov decision processes

Recommendations

An active learning based TCM-KNN algorithm for supervised network intrusion detection

As network attacks have increased in number and severity over the past few years, intrusion detection is increasingly becoming a critical component of secure information systems and supervised network intrusion detection has been an active and difficult ...
A Hybrid Network Intrusion Detection Technique Using Random Forests
ARES '06: Proceedings of the First International Conference on Availability, Reliability and Security

Intrusion detection is important in network security. Most current network intrusion detection systems (NIDSs) employ either misuse detection or anomaly detection. However, misuse detection cannot detect unknown intrusions, and anomaly detection usually ...
A Novel Outlier Detection Scheme for Network Intrusion Detection Systems
ISA '08: Proceedings of the 2008 International Conference on Information Security and Assurance (isa 2008)

Network intrusion detection system serves as a second line of defense to intrusion prevention. Anomaly detection approach is important in order to detect new attacks. This paper adopted connectivity-based outlier detection scheme from statistical field ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

AISec '09: Proceedings of the 2nd ACM workshop on Security and artificial intelligence

November 2009

72 pages

ISBN:9781605587813

DOI:10.1145/1654988

Program Chairs:
Dirk Balfanz
Google, USA
,
Jessica Staddon
PARC, USA

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 November 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CCS '09

Sponsor:

SIGSAC

CCS '09: 16th ACM Conference on Computer and Communications Security 2009

November 9, 2009

Illinois, Chicago, USA

Acceptance Rates

Overall Acceptance Rate 94 of 231 submissions, 41%

Upcoming Conference

CCS '25

Sponsor:
sigsac

ACM SIGSAC Conference on Computer and Communications Security

October 13 - 17, 2025

Taipei , Taiwan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

57
Total Citations
View Citations
895
Total Downloads

Downloads (Last 12 months)36
Downloads (Last 6 weeks)7

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wu LXie YLi JFeng DLiang JWu Y(2025)Angus: efficient active learning strategies for provenance based intrusion detectionCybersecurity10.1186/s42400-024-00311-y8:1Online publication date: 27-Jan-2025
https://doi.org/10.1186/s42400-024-00311-y
Makris IKarampasi ARadoglou-Grammatikis PEpiskopos NIturbe ERios EPiperigkos NLalos AXenakis CLagkas TArgyriou VSarigiannidis P(2025)A comprehensive survey of Federated Intrusion Detection Systems: Techniques, challenges and solutionsComputer Science Review10.1016/j.cosrev.2024.10071756(100717)Online publication date: May-2025
https://doi.org/10.1016/j.cosrev.2024.100717
Zhang YDong HNottingham ABuchanan MBrown DSun Y(2023)Global Analysis with Aggregation-based Beaconing Detection across Large Campus NetworksProceedings of the 39th Annual Computer Security Applications Conference10.1145/3627106.3627126(565-579)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1145/3627106.3627126
Leite Cden Hartog JKoster PRanise SCarbone RTakabi D(2023)A Framework for Privacy-Preserving White-Box Anomaly Detection using a Lattice-Based Access ControlProceedings of the 28th ACM Symposium on Access Control Models and Technologies10.1145/3589608.3593831(7-18)Online publication date: 24-May-2023
https://dl.acm.org/doi/10.1145/3589608.3593831
Cao VNguyen MLe Dinh T(2023)Few-Shot Learning with Discriminative Representation for Cyberattack Detection2023 15th International Conference on Knowledge and Systems Engineering (KSE)10.1109/KSE59128.2023.10299444(1-6)Online publication date: 18-Oct-2023
https://doi.org/10.1109/KSE59128.2023.10299444
Zeng QNait-Abdesselam FChen Z(2023)A HITL-Integrated Machine Learning Approach to Secure Drone Networks for IIoT Applications2023 IEEE Globecom Workshops (GC Wkshps)10.1109/GCWkshps58843.2023.10465098(614-619)Online publication date: 4-Dec-2023
https://doi.org/10.1109/GCWkshps58843.2023.10465098
Kumar AKhimani VChatzopoulos DHui P(2022)FedClean: A Defense Mechanism against Parameter Poisoning Attacks in Federated LearningICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP43922.2022.9747497(4333-4337)Online publication date: 23-May-2022
https://doi.org/10.1109/ICASSP43922.2022.9747497
Apruzzese GLaskov PTastemirova A(2022)SoK: The Impact of Unlabelled Data in Cyberthreat Detection2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P)10.1109/EuroSP53844.2022.00010(20-42)Online publication date: Jun-2022
https://doi.org/10.1109/EuroSP53844.2022.00010
Faber KCorizzo RSniezynski BJapkowicz N(2022)Active Lifelong Anomaly Detection with Experience Replay2022 IEEE 9th International Conference on Data Science and Advanced Analytics (DSAA)10.1109/DSAA54385.2022.10032405(1-10)Online publication date: 13-Oct-2022
https://doi.org/10.1109/DSAA54385.2022.10032405
Trofti PPatrason AHiji A(2022)Unsupervised Abnormal Traffic Detection through Topological Flow Analysis2022 14th International Conference on Communications (COMM)10.1109/COMM54429.2022.9817285(1-6)Online publication date: 16-Jun-2022
https://doi.org/10.1109/COMM54429.2022.9817285
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten