research-article

Open access

RAPID: Real-Time Alert Investigation with Context-aware Prioritization for Efficient Threat Discovery

Authors:

Prateek MittalAuthors Info & Claims

ACSAC '22: Proceedings of the 38th Annual Computer Security Applications Conference

Pages 827 - 840

https://doi.org/10.1145/3564625.3567997

Published: 05 December 2022 Publication History

All formats PDF

Abstract

Alerts reported by intrusion detection systems (IDSes) are often the starting points for attack campaign discovery and response procedures. However, the sheer number of alerts compared to the number of real attacks, along with the complexity of alert investigations, poses a challenge to achieving effective alert triage with limited computational resources. Automated procedures and human analysts could suffer from the burden of analyzing floods of alerts, and fail to respond to critical alerts promptly.

To scale out the alert processing capability in enterprises, we present RAPID, a real-time alert investigation system to aid analysts perform provenance analysis tasks around alerts in an efficient and collaborative manner. RAPID is built based on two key insights: 1) space and time efficiency of alert investigations can be improved by avoiding the significant overlap between alert triage tasks; 2) prioritization of alert triage tasks should be dynamic to adapt to the newly discovered context. In doing so, RAPID maximizes the utilization of limited computation resources and time, and reacts to the most critical reasoning steps in a timely manner. More specifically, RAPID employs an interruptible tracking algorithm that efficiently uncovers the causal connections between alerts and propagates priorities based on the connections. Unlike prior work, RAPID does not rely on knowledge of existing threat ontologies and focuses on providing a general concurrent alert investigation platform with provenance analysis capabilities. We evaluate RAPID on a 1TB dataset from DARPA Transparent Computing (TC) program with 411 million events, including three attack campaigns. The results show that RAPID is able to improve space efficiency by up to three orders of magnitude and reduce the time of alert provenance analysis to discover all the major attack traces by up to 99%.

References

[1]

2003. Artificial Intelligence: A Modern Approach. (2003).

[2]

2018. System Auditing RedHat Enterprise Linux 6. https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/security_guide/chap-system_auditing.

[3]

2019. Public exploits. https://www.hackingloops.com/exploit/.

[4]

2019. Zynga data breach. https://us.norton.com/internetsecurity-emerging-threats-new-report-says-zynga-breach-in-september-affected-172-million-a.html.

[5]

2020. APT Notes. https://github.com/aptnotes/data.

[6]

2020. DARPA TC engagement data release. https://github.com/darpa-i2o/Transparent-Computing/.

[7]

2020. Equifax data breach. https://www.ftc.gov/equifax-data-breach.

[8]

2020. Intel Threading Building Blocks. https://software.intel.com/en-us/tbb-documentation.

[9]

2020. Libcuckoo: high-performance, compact concurrent hash table. https://github.com/efficient/libcuckoo.

[10]

2021. Falcon Insight. https://www.crowdstrike.com/endpoint-security-products/falcon-insight-endpoint-detection-response/.

[11]

2021. McAfee MVISION EDR. https://www.mcafee.com/enterprise/en-us/products/mvision-edr.html.

[12]

2021. MITRE ATT&CK. https://attack.mitre.org/.

[13]

2021. VMware Carbon Black. https://www.carbonblack.com/.

[14]

Abdulellah Alsaheel, Yuhong Nan, Shiqing Ma, Le Yu, Gregory Walkup, Z Berkay Celik, Xiangyu Zhang, and Dongyan Xu. 2021. {ATLAS}: A sequence-based learning approach for attack investigation. In 30th USENIX Security Symposium (USENIX Security 21). 3005–3022.

[15]

Md. Monowar Anjum, Shahrear Iqbal, and Benoit Hamelin. 2022. ANUBIS: A Provenance Graph-Based Framework for Advanced Persistent Threat Detection. In Association for Computing Machinery (SAC).

[16]

Adam Bates, Dave Tian, Kevin R. B. Butler, and Thomas Moyer. 2015. Trustworthy Whole-System Provenance for the Linux Kernel. In Proceedings of the 24th USENIX Conference on Security Symposium (Washington, D.C.) (SEC’15). USENIX Association, USA, 319–334.

Digital Library

[17]

D Elliott Bell and Leonard J La Padula. 1976. Secure computer system: Unified exposition and multics interpretation. Technical Report. MITRE CORP BEDFORD MA.

[18]

Kenneth J Biba. 1977. Integrity considerations for secure computer systems. Technical Report. MITRE CORP BEDFORD MA.

[19]

Leyla Bilge and Tudor Dumitraş. 2012. Before we knew it: an empirical study of zero-day attacks in the real world. In Proceedings of the 2012 ACM conference on Computer and communications security. 833–844.

Digital Library

[20]

BRICATA. 2021. Alert volume report. https://bricata.com/blog/how-many-daily-cybersecurity-alerts/.

[21]

Jim Chow, Ben Pfaff, Tal Garfinkel, Kevin Christopher, and Mendel Rosenblum. 2004. Understanding data lifetime via whole system simulation. In USENIX Security Symposium. 321–336.

Digital Library

[22]

McAfee Corp.2020. How Collaboration Can Optimize Security Operations. https://cdw-prod.adobecqms.net/content/dam/cdw/on-domain-cdw/brands/intel/soc-collaboration-advanced-threats.pdf.

[23]

Peng Gao, Xusheng Xiao, Ding Li, Zhichun Li, Kangkook Jee, Zhenyu Wu, Chung Hwan Kim, Sanjeev R. Kulkarni, and Prateek Mittal. 2018. SAQL: A Stream-based Query System for Real-Time Abnormal System Behavior Detection. In USENIX Security.

[24]

Peng Gao, Xusheng Xiao, Zhichun Li, Fengyuan Xu, Sanjeev R. Kulkarni, and Prateek Mittal. 2018. AIQL: Enabling Efficient Attack Investigation from System Monitoring Data. In USENIX ATC.

[25]

Ashvin Goel, Kenneth Po, Kamran Farhadi, Zheng Li, and Eyal De Lara. 2005. The taser intrusion recovery system. In ACM SIGOPS Operating Systems Review, Vol. 39. ACM, 163–176.

[26]

Guofei Gu, Alvaro A Cárdenas, and Wenke Lee. 2008. Principled reasoning and practical applications of alert fusion in intrusion detection systems. In Proceedings of the 2008 ACM symposium on Information, computer and communications security. ACM, 136–147.

Digital Library

[27]

Xueyuan Han, Thomas Pasquier, Adam Bates, James Mickens, and Margo Seltzer. 2022. Unicorn: Runtime provenance-based detector for advanced persistent threats. In The Network and Distributed System Security Symposium (NDSS).

[28]

Wajih Ul Hassan, Adam Bates, and Daniel Marino. 2020. Tactical provenance analysis for endpoint detection and response systems. In 2020 IEEE Symposium on Security and Privacy (SP). IEEE, 1172–1189.

[29]

Wajih Ul Hassan, Shengjian Guo, Ding Li, Zhengzhang Chen, Kangkook Jee, Zhichun Li, and Adam Bates. 2019. NODOZE: Combatting Threat Alert Fatigue with Automated Provenance Triage. In Network and Distributed Systems Security Symposium.

[30]

Md Nahid Hossain, Sadegh M Milajerdi, Junao Wang, Birhanu Eshete, Rigel Gjomemo, R Sekar, Scott D Stoller, and VN Venkatakrishnan. 2017. SLEUTH: Real-time attack scenario reconstruction from COTS audit data. In Proc. USENIX Secur.487–504.

[31]

Md Nahid Hossain, Sanaz Sheikhi, and R Sekar. 2020. Combating Dependence Explosion in Forensic Analysis Using Alternative Tag Propagation Semantics. In IEEE Symposium on Security and Privacy.

[32]

Yang Ji, Sangho Lee, Evan Downing, Weiren Wang, Mattia Fazzini, Taesoo Kim, Alessandro Orso, and Wenke Lee. 2017. RAIN: Refinable Attack Investigation with On-Demand Inter-Process Information Flow Tracking. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (Dallas, Texas, USA) (CCS ’17). Association for Computing Machinery, New York, NY, USA, 377–390. https://doi.org/10.1145/3133956.3134045

Digital Library

[33]

Xuxian Jiang, AAron Walters, Dongyan Xu, Eugene H Spafford, Florian Buchholz, and Yi-Min Wang. 2006. Provenance-aware tracing ofworm break-in and contaminations: A process coloring approach. In Distributed Computing Systems, 2006. ICDCS 2006. 26th IEEE International Conference on. IEEE, 38–38.

[34]

Samuel T King and Peter M Chen. 2005. Backtracking intrusions. ACM Transactions on Computer Systems (TOCS) 23, 1 (2005), 51–76.

Digital Library

[35]

Samuel T King, Zhuoqing Morley Mao, Dominic G Lucchetti, and Peter M Chen. 2005. Enriching Intrusion Alerts Through Multi-Host Causality. In NDSS.

[36]

Aron Laszka, Jian Lou, and Yevgeniy Vorobeychik. 2016. Multi-Defender Strategic Filtering against Spear-Phishing Attacks. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (Phoenix, Arizona) (AAAI’16). AAAI Press, 537–543.

Digital Library

[37]

Yushan Liu, Mu Zhang, Ding Li, Kangkook Jee, Zhichun Li, Zhenyu Wu, Junghwan Rhee, and Prateek Mittal. 2018. Towards a timely causality analysis for enterprise security. In Proceedings of the 25th Network and Distributed System Security Symposium (NDSS). The Internet Society, San Diego, California, USA.

[38]

Shiqing Ma, Kyu Hyung Lee, Chung Hwan Kim, Junghwan Rhee, Xiangyu Zhang, and Dongyan Xu. 2015. Accurate, low cost and instrumentation-free security audit logging for windows. In Proceedings of the 31st Annual Computer Security Applications Conference. ACM, 401–410.

Digital Library

[39]

Shiqing Ma, Juan Zhai, Fei Wang, Kyu Hyung Lee, Xiangyu Zhang, and Dongyan Xu. 2017. MPI: Multiple Perspective Attack Investigation with Semantic Aware Execution Partitioning. In 26th USENIX Security Symposium (USENIX Security 17). USENIX Association, Vancouver, BC, 1111–1128.

[40]

Shiqing Ma, Xiangyu Zhang, and Dongyan Xu. 2016. ProTracer: Towards Practical Provenance Tracing by Alternating Between Logging and Tainting. In NDSS.

[41]

Emaad Manzoor, Sadegh M Milajerdi, and Leman Akoglu. 2016. Fast memory-efficient anomaly detection in streaming heterogeneous graphs. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1035–1044.

Digital Library

[42]

Noor Michael, Jaron Mink, Jason Liu, Sneha Gaur, Wajih Ul Hassan, and Adam Bates. 2020. On the Forensic Validity of Approximated Audit Logs. In Annual Computer Security Applications Conference. 189–202.

[43]

Sadegh M. Milajerdi, Birhanu Eshete, Rigel Gjomemo, and V.N. Venkatakrishnan. 2019. POIROT: Aligning Attack Behavior with Kernel Audit Records for Cyber Threat Hunting. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security(London, United Kingdom) (CCS ’19). Association for Computing Machinery, New York, NY, USA, 1795–1812.

Digital Library

[44]

Sadegh M Milajerdi, Rigel Gjomemo, Birhanu Eshete, R Sekar, and VN Venkatakrishnan. 2019. HOLMES: real-time APT detection through correlation of suspicious information flows. In IEEE Symposium on Security and Privacy.

[45]

Peng Ning, Yun Cui, and Douglas S Reeves. 2002. Constructing attack scenarios through correlation of intrusion alerts. In Proceedings of the 9th ACM Conference on Computer and Communications Security. ACM, 245–254.

Digital Library

[46]

Peng Ning and Dingbang Xu. 2004. Hypothesizing and reasoning about attacks missed by intrusion detection systems. ACM Transactions on Information and System Security (TISSEC) 7, 4(2004), 591–627.

Digital Library

[47]

Peng Ning, Dingbang Xu, Christopher G Healey, and Robert St Amant. 2004. Building Attack Scenarios through Integration of Complementary Alert Correlation Method. In NDSS, Vol. 4. 97–111.

[48]

Xinzhou Qin and Wenke Lee. 2003. Statistical causality analysis of infosec alert data. In International Workshop on Recent Advances in Intrusion Detection. Springer, 73–93.

[49]

Xiaokui Shu, Frederico Araujo, Douglas L. Schales, Marc Ph. Stoecklin, Jiyong Jang, Heqing Huang, and Josyula R. Rao. 2018. Threat Intelligence Computing. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (CCS) (Toronto, Canada). ACM, New York, NY, USA, 1883–1898.

Digital Library

[50]

C. Zhong, J. Yen, P. Liu, and R. F. Erbacher. 2016. Automate Cybersecurity Data Triage by Leveraging Human Analysts’ Cognitive Process. In 2016 IEEE 2nd International Conference on Big Data Security on Cloud (BigDataSecurity), IEEE International Conference on High Performance and Smart Computing (HPSC), and IEEE International Conference on Intelligent Data and Security (IDS). 357–363.

Cited By

Sang AWang YYang LJia JZhou L(2024)Obfuscating Provenance-Based Forensic Investigations with Mapping System Meta-BehaviorProceedings of the 27th International Symposium on Research in Attacks, Intrusions and Defenses10.1145/3678890.3678916(248-262)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3678890.3678916
Sarda KNamrud ZLitoiu MShwartz LWatts Id'Amorim M(2024)Leveraging Large Language Models for the Auto-remediation of Microservice Applications: An Experimental StudyCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3663855(358-369)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3663855
Empl PBöhm FPernul GVilela JSchulmann HLi N(2024)Process-Aware Intrusion Detection in MQTT NetworksProceedings of the Fourteenth ACM Conference on Data and Application Security and Privacy10.1145/3626232.3653271(91-102)Online publication date: 19-Jun-2024
https://dl.acm.org/doi/10.1145/3626232.3653271
Show More Cited By

Index Terms

RAPID: Real-Time Alert Investigation with Context-aware Prioritization for Efficient Threat Discovery

Index terms have been assigned to the content through auto-classification.

Recommendations

FuzMet: a fuzzy-logic based alert prioritization engine for intrusion detection systems

Intrusion detection systems (IDSs) are designed to monitor a networked environment and generate alerts whenever abnormal activities are detected. The number of these alerts can be very large, making their evaluation by security analysts a difficult ...
Alert verification through alert correlation: An empirical test of SnIPS

A significant problem with today’s intrusion detection systems is the high number of alerts they produce for events that are regarded as benign or noncritical by system administrators. A large number of solutions has been proposed to deal with this ...
Alert verification evasion through server response forging
RAID'07: Proceedings of the 10th international conference on Recent advances in intrusion detection

Intrusion Detection Systems (IDSs) are necessary components in the defense of any computer network. Network administrators rely on IDSs to detect attacks, but ultimately it is their responsibility to investigate IDS alerts and determine the damage done. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ACSAC '22: Proceedings of the 38th Annual Computer Security Applications Conference

December 2022

1021 pages

ISBN:9781450397599

DOI:10.1145/3564625

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 December 2022

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

DARPA

Conference

ACSAC

ACSAC: Annual Computer Security Applications Conference

December 5 - 9, 2022

TX, Austin, USA

Acceptance Rates

Overall Acceptance Rate 104 of 497 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
1,302
Total Downloads

Downloads (Last 12 months)668
Downloads (Last 6 weeks)87

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sang AWang YYang LJia JZhou L(2024)Obfuscating Provenance-Based Forensic Investigations with Mapping System Meta-BehaviorProceedings of the 27th International Symposium on Research in Attacks, Intrusions and Defenses10.1145/3678890.3678916(248-262)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3678890.3678916
Sarda KNamrud ZLitoiu MShwartz LWatts Id'Amorim M(2024)Leveraging Large Language Models for the Auto-remediation of Microservice Applications: An Experimental StudyCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3663855(358-369)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3663855
Empl PBöhm FPernul GVilela JSchulmann HLi N(2024)Process-Aware Intrusion Detection in MQTT NetworksProceedings of the Fourteenth ACM Conference on Data and Application Security and Privacy10.1145/3626232.3653271(91-102)Online publication date: 19-Jun-2024
https://dl.acm.org/doi/10.1145/3626232.3653271
Mundt MBaier H(2024)Enhancing Incident Management by an Improved Understanding of Data Exfiltration: Definition, Evaluation, ReviewDigital Forensics and Cyber Crime10.1007/978-3-031-56580-9_3(33-57)Online publication date: 3-Apr-2024
https://doi.org/10.1007/978-3-031-56580-9_3
Dong FLi SJiang PLi DWang HHuang LXiao XChen JLuo XGuo YChen XMeng WJensen CCremers CKirda E(2023)Are we there yet? An Industrial Viewpoint on Provenance-based Endpoint Detection and Response ToolsProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security10.1145/3576915.3616580(2396-2410)Online publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1145/3576915.3616580
Tang YHan YSun RZhou SFeng YZhang Q(2023)Threat Tracing of Power Distribution Automation System Based on Spatiotemporal and Business Logic Correlation Technology of Security Events2023 International Conference on Networks, Communications and Intelligent Computing (NCIC)10.1109/NCIC61838.2023.00052(273-278)Online publication date: 17-Nov-2023
https://doi.org/10.1109/NCIC61838.2023.00052

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents