Article

Analysis and Results of the 1999 DARPA Off-Line Intrusion Detection Evaluation

Authors:

Richard Lippmann,

Joshua W. Haines,

David J. Fried,

Jonathan Korba,

Kumar DasAuthors Info & Claims

RAID '00: Proceedings of the Third International Workshop on Recent Advances in Intrusion Detection

Pages 162 - 182

Published: 02 October 2000 Publication History

Abstract

Eight sites participated in the second DARPA off-line intrusion detection evaluation in 1999. Three weeks of training and two weeks of test data were generated on a test bed that emulates a small government site. More than 200 instances of 58 attack types were launched against victim UNIX and Windows NT hosts. False alarm rates were low (less than 10 per day). Best detection was provided by network-based systems for old probe and old denial-of-service (DoS) attacks and by host-based systems for Solaris user-to-root (U2R) attacks. Best overall performance would have been provided by a combined system that used both host- and network-based intrusion detection. Detection accuracy was poor for previously unseen new, stealthy, and Windows NT attacks. Ten of the 58 attack types were completely missed by all systems. Systems missed attacks because protocols and TCP services were not analyzed at all or to the depth required, because signatures for old attacks did not generalize to new attacks, and because auditing was not available on all hosts.

References

[1]

E. G. Amoroso, Intrusion Detection: An Introduction to Internet Surveillance, Correlation, Trace Back, Traps, and Response, Intrusion. Net Books, 1999.

[2]

K. Das, The Development of Stealthy Attacks to Evaluate Intrusion Detection Systems, S. M. Thesis, MIT Department of Electrical Engineering and Computer Science, June 2000.

[3]

R. Durst, Terrence Champion, Brian Witten, Eric Miller and Luigi Spagnuolo, Testing and evaluating computer intrusion detection systems, Communications of the ACM, 42, 1999, 53-61.

[4]

A. K. Ghosh and A. Schwartzbard, A Study in Using Neural Networks for Anomaly and Misuse Detection, in Proceedings of the USENIX Security Symposium, August 23-26, 1999, Washington, D.C, http://www.rstcorp.com/~anup.

[5]

T. Heberlein, T., Network Security Monitor (NSM) - Final Report, U. C. Davis: February 1995, http://seclab.cs.ucdavis.edu/papers/NSM-final.pdf

[6]

K. Jackson, Intrusion Detection System (IDS) Product Survey, Los Alamos National Laboratory, Report LA-UR-99-3883, 1999.

[7]

S. Jajodia, D. Barbara, B. Speegle, and N. Wu, Audit Data Analysis and Mining (ADAM), project described in http://www.isse.gmu.edu/~dbarbara/adam.html, April, 2000.

[8]

K. Kendall, A Database of Computer Attacks for the Evaluation of Intrusion Detection Systems, S. M. Thesis, MIT Department of Electrical Engineering and Computer Science, June 1999.

[9]

J. Korba, Windows NT Attacks for the Evaluation of Intrusion Detection Systems, S. M. Thesis, MIT Department of Electrical Engineering and Computer Science, June 2000.

[10]

Lawrence Berkeley National Laboratory Network Research Group provides tcp-dump at http://www-nrg.ee.lbl.gov.

[11]

R. P. Lippmann, Joshua W. Haines, David J. Fried, Jonathan Korba, and Kumar Das, The 1999 DARPA Off-Line Intrusion Detection Evaluation, Computer Networks, In Press, 2000.

[12]

R. P. Lippmann, David J. Fried, Isaac Graf, Joshua W. Haines, Kristopher R. Kendall, David McClung, Dan Weber, Seth E. Webster, Dan Wyschogrod, Robert K. Cunningham, and Marc A. Zissman, Evaluating Intrusion Detection Systems: the 1998 DARPA Off-Line Intrusion Detection Evaluation, in Proceedings of the 2000 DARPA Information Survivability Conference and Exposition (DISCEX), Vol. 2, IEEE Press, January 2000.

[13]

R. P. Lippmann and R. K. Cunningham, Guide to Creating Stealthy Attacks for the 1999 DARPA Off-Line Intrusion Detection Evaluation, MIT Lincoln Laboratory Project Report IDDE-1, June 1999.

[14]

MIT Lincoln Laboratory, A public web site http://www.ll.mit.edu/IST/ideval/index.html, contains limited information on the 1998 and 1999 evaluations. Follow instructions on this web site or send email to the authors (rpl or [email protected]) to obtain access to a password-protected site with more complete information on these evaluations and results. Software scripts to execute attacks are not provided on these or other web sites.

[15]

P. Neumann and P. Porras, Experience with EMERALD to DATE, in Proceedings 1st USENIX Workshop on Intrusion Detection and Network Monitoring, Santa Clara, California, April 1999, 73-80, http://www.sdl.sri.com/emerald/index.html.

[16]

S. Northcutt, Network Intrusion Detection; An Analysis Handbook, New Riders Publishing, Indianapolis, 1999.

[17]

V. Paxson, "Empirically-Derived Analytic Models of Wide-Area TCP Connections", IEEE/ACM Transactions on Networking, Vol. 2, No. 4, August, 1994, ftp://ftp.ee.lbl.gov/papers/WAN-TCP-models.ps.Z.

[18]

T. H. Ptacek and T. N. Newsham, Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection, Secure Networks, Inc. Report, January 1998.

[19]

N. Puketza, M. Chung, R. A. Olsson, and B. Mukherjee, A Software Platform for Testing Intrusion Detection Systems, IEEE Software, September/October, 1997, 43-51.

[20]

A. Schwartzbard and A. K. Ghosh, A Study in the Feasibility of Performing Host-based Anomaly Detection on Windows NT, in Proceedings of the 2nd Recent Advances in Intrusion Detection (RAID 1999) Workshop, West Lafayette, IN, September 7-9, 1999.

[21]

R. Sekar and P. Uppuluri, Synthesizing Fast Intrusion Prevention/Detection Systems from High-Level Specifications, in Proceedings 8th Usenix Security Symposium, Washington DC, Aug. 1999, http://rcs-sgi.cs.iastate.edu/sekar/abs/usenixsec99.htm.

[22]

M. Tyson, P. Berry, N. Williams, D. Moran, D. Blei, DERBI: Diagnosis, Explanation and Recovery from computer Break-Ins, project described in http://www.ai.sri.com/~derbi/, April. 2000.

[23]

G. Vigna, S. T. Eckmann, and R. A. Kemmerer, The STAT Tool Suite, in Proceedings of the 2000 DARPA Information Survivability Conference and Exposition (DISCEX), IEEE Press, January 2000.

Cited By

Ekle OEberle W(2024)Anomaly Detection in Dynamic Graphs: A Comprehensive SurveyACM Transactions on Knowledge Discovery from Data10.1145/366990618:8(1-44)Online publication date: 29-May-2024
https://dl.acm.org/doi/10.1145/3669906
Tan PTan ZJiang YZhou ZElkind E(2023)Handling learnwares developed from heterogeneous feature spaces without auxiliary dataProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/471(4235-4243)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/471
Sun YJi WWeng JZhao BLi YWu X(2020)Selection of Optimal Strategy for Moving Target Defense Based on Signal GameProceedings of the 2020 International Conference on Cyberspace Innovation of Advanced Technologies10.1145/3444370.3444543(28-32)Online publication date: 4-Dec-2020
https://dl.acm.org/doi/10.1145/3444370.3444543
Show More Cited By

Analysis and Results of the 1999 DARPA Off-Line Intrusion Detection Evaluation
1. Networks
2. Security and privacy
  1. Systems security

Recommendations

The 1999 DARPA off-line intrusion detection evaluation
Special issue on recent advances in intrusion detection systems
Network intrusion detection

Intrusion detection is a new, retrofit approach for providing a sense of security in existing computers and data networks, while allowing them to operate in their current "open" mode. The goal of intrusion detection is to identify unauthorized use, ...
A Survey on Intrusion Detection and Prevention Systems
Abstract
In the digital world, malicious activities that violate the confidentiality, integrity, or availability of data and devices are known as intrusions. An intrusion detection system (IDS) analyses the activities of a single system or a network to ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

RAID '00: Proceedings of the Third International Workshop on Recent Advances in Intrusion Detection

October 2000

226 pages

ISBN:3540410856

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 02 October 2000

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

57
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ekle OEberle W(2024)Anomaly Detection in Dynamic Graphs: A Comprehensive SurveyACM Transactions on Knowledge Discovery from Data10.1145/366990618:8(1-44)Online publication date: 29-May-2024
https://dl.acm.org/doi/10.1145/3669906
Tan PTan ZJiang YZhou ZElkind E(2023)Handling learnwares developed from heterogeneous feature spaces without auxiliary dataProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/471(4235-4243)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/471
Sun YJi WWeng JZhao BLi YWu X(2020)Selection of Optimal Strategy for Moving Target Defense Based on Signal GameProceedings of the 2020 International Conference on Cyberspace Innovation of Advanced Technologies10.1145/3444370.3444543(28-32)Online publication date: 4-Dec-2020
https://dl.acm.org/doi/10.1145/3444370.3444543
AIT TCHAKOUCHT TEZZIYYANI M(2018)Building A Fast Intrusion Detection System For High-Speed-NetworksProcedia Computer Science10.1016/j.procs.2018.01.151127:C(521-530)Online publication date: 1-May-2018
https://dl.acm.org/doi/10.1016/j.procs.2018.01.151
Chun LXiaoxian GJing ZWei WHanji SPeng G(2017)Intrusion Detection Using End-to-End Memory NetworkProceedings of the 2017 2nd International Conference on Communication and Information Systems10.1145/3158233.3159367(244-249)Online publication date: 7-Nov-2017
https://dl.acm.org/doi/10.1145/3158233.3159367
Torrano-Gimenez CNguyen HAlvarez GFranke K(2015)Combining expert knowledge with automatic feature extraction for reliable web attack detectionSecurity and Communication Networks10.1002/sec.6038:16(2750-2767)Online publication date: 10-Nov-2015
https://dl.acm.org/doi/10.1002/sec.603
Nanda RKrishna PElçi AGaur MOrgun MMakarevich O(2013)D0M-WLANProceedings of the 6th International Conference on Security of Information and Networks10.1145/2523514.2523575(421-424)Online publication date: 26-Nov-2013
https://dl.acm.org/doi/10.1145/2523514.2523575
Elfeshawy NFaragallah O(2013)Divided two-part adaptive intrusion detection systemWireless Networks10.1007/s11276-012-0467-719:3(301-321)Online publication date: 1-Apr-2013
https://dl.acm.org/doi/10.1007/s11276-012-0467-7
Valgenti VKim MRiley GQuaglia FHimmelspach J(2012)An application-level content generative model for network applicationsProceedings of the 5th International ICST Conference on Simulation Tools and Techniques10.5555/2263019.2263026(47-56)Online publication date: 19-Mar-2012
https://dl.acm.org/doi/10.5555/2263019.2263026
Valgenti VKim MLiu JQuaglia FEidenbenz SGilmore S(2011)Simulating content in traffic for benchmarking intrusion detection systemsProceedings of the 4th International ICST Conference on Simulation Tools and Techniques10.5555/2151054.2151062(44-50)Online publication date: 21-Mar-2011
https://dl.acm.org/doi/10.5555/2151054.2151062
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents