research-article

Process out-grafting: an efficient "out-of-VM" approach for fine-grained process execution monitoring

Authors:

Deepa Srinivasan,

Dongyan XuAuthors Info & Claims

CCS '11: Proceedings of the 18th ACM conference on Computer and communications security

Pages 363 - 374

https://doi.org/10.1145/2046707.2046751

Published: 17 October 2011 Publication History

Abstract

Recent rapid malware growth has exposed the limitations of traditional in-host malware-defense systems and motivated the development of secure virtualization-based out-of-VM solutions. By running vulnerable systems as virtual machines (VMs) and moving security software from inside the VMs to outside, the out-of-VM solutions securely isolate the anti-malware software from the vulnerable system. However, the presence of semantic gap also leads to the compatibility problem in not supporting existing defense software. In this paper, we present process out-grafting, an architectural approach to address both isolation and compatibility challenges in out-of-VM approaches for fine-grained process-level execution monitoring. Specifically, by relocating a suspect process from inside a VM to run side-by-side with the out-of-VM security tool, our technique effectively removes the semantic gap and supports existing user-mode process monitoring tools without any modification. Moreover, by forwarding the system calls back to the VM, we can smoothly continue the execution of the out-grafted process without weakening the isolation of the monitoring tool. We have developed a KVM-based prototype and used it to natively support a number of existing tools without any modification. The evaluation results including measurement with benchmark programs show it is effective and practical with a small performance overhead.

References

[1]

Kaiten. http://packetstormsecurity.org/irc/kaiten.c. {last accessed: May 2011}.

[2]

Kernel Virtual Machine. http://www.linux-kvm.org. {last accessed: May 2011}.

[3]

McAfee Threats Report: Fourth Quarter 2010. http://www.mcafee.com/us/resources/reports/rp-quarterly-threat-q4--2010.pdf. {last accessed: May 2011}.

[4]

QEMU. http://www.qemu.org. {last accessed: May 2011}.

[5]

UPX: The Ultimate Packer for eXecutables. http://upx.sourceforge.net. {last accessed: May 2011}.

[6]

Adams, K., and Agesen, O. A Comparison of Software and Hardware Techniques for x86 Virtualization. In Proceedings of the 12th International Conference on Architectural Support for Programming Languages and Operating Systems (2006).

Digital Library

[7]

AMD. AMD-V Nested Paging. AMD White Paper (2008).

[8]

Azab, A. M., Ning, P., Sezer, E. C., and Zhang, X. HIMA: A Hypervisor-Based Integrity Measurement Agent. In Proceedings of the 25th Annual Computer Security Applications Conference (2009).

Digital Library

[9]

Bayer, U., Kruegel, C., and Kirda, E. TTAnalyze: A Tool for Analyzing Malware. In Proceedings of the 15th Annual Conference of the European Institute for Computer Antivirus Research (2006).

[10]

cker Chiueh, T., Conover, M., Lu, M., and Montague, B. Stealthy Deployment and Execution of In-Guest Kernel Agents. In BlackHat 2009.

[11]

Dinaburg, A., Royal, P., Sharif, M., and Lee, W. Ether: Malware Analysis via Hardware Virtualization Extensions. In Proceedings of the 15th ACM Conference on Computer and Communications Security (2008).

Digital Library

[12]

Dolan-Gavitt, B., Leek, T., Zhivich, M., Giffin, J., and Lee, W. Virtuoso: Narrowing the Semantic Gap in Virtual Machine Introspection. In Proceedings of the 32nd IEEE Symposium on Security and Privacy (2011).

Digital Library

[13]

Forrest, S., Hofmeyr, S., and Somayaji, A. The Evolution of System-Call Monitoring. In Proceedings of the 24th Annual Computer Security Applications Conference (2008).

Digital Library

[14]

Garfinkel, T., Pfaff, B., and Rosenblum, M. Ostia: A Delegating Architecture for Secure System Call Interposition. In Proceedings of the 11th Annual Network and Distributed System Security Symposium (2004).

[15]

Garfinkel, T., and Rosenblum, M. A Virtual Machine Introspection Based Architecture for Intrusion Detection. In Proceedings of the 10th Annual Network and Distributed Systems Security Symposium (2003).

[16]

Goldberg, I., Wagner, D., Thomas, R., and Brewer, E. A. A Secure Environment for Untrusted Helper Applications: Confining the Wily Hacker. In Proceedings of the 6th USENIX Security Symposium (1996).

Digital Library

[17]

Guo, F., Ferrie, P., and Chiueh, T.-c. A Study of the Packer Problem and Its Solutions. In Proceedings of the 11th International Symposium on Recent Advances in Intrusion Detection. (2008).

Digital Library

[18]

Intel. Intel Virtualization Technology: Hardware Support for Efficient Processor Virtualization. Intel(R) Technology Journal 10, 3 (2006).

[19]

Intel. Intel 64 and IA-32 Architectures Software Developer's Manual Volume 3: System Programming Guide, Part 1 and Part 2, (2010).

[20]

Jiang, X., and Wang, X. "Out-of-the-Box" Monitoring of VM-Based High-Interaction Honeypots. In Proceedings of the 10th International Symposium on Recent Advances in Intrusion Detection (2007).

Digital Library

[21]

Jiang, X., Wang, X., and Xu, D. Stealthy Malware Detection through VMM-based "Out-of-the-Box" Semantic View Reconstruction. In Proceedings of the 14th ACM Conference on Computer and Communications Security (2007).

Digital Library

[22]

Gu, Z., Deng, Z., Xu, D., and Jiang, X. Process Implanting: A New Active Introspection Framework for Virtualization. In Proceedings of the 30th IEEE Symposium on Reliable Distributed Systems(2011).

Digital Library

[23]

Joshi, A., King, S. T., Dunlap, G. W., and Chen, P. M. Detecting Past and Present Intrusions through Vulnerability-specific Predicates. In Proceedings of the 20th ACM Symposium on Operating Systems Principles (2005).

Digital Library

[24]

King, S. T., and Chen, P. M. Backtracking Intrusions. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (2003).

Digital Library

[25]

Klein, G., Elphinstone, K., Heiser, G., Andronick, J., Cock, D., Derrin, P., Elkaduwe, D., Engelhardt, K., Kolanski, R., Norrish, M., Sewell, T., Tuch, H., and Winwood, S. seL4: Formal Verification of an OS Kernel. In Proceedings of the 22nd Symposium on Operating Systems Principles (2009).

Digital Library

[26]

Martignoni, L., Christodorescu, M., and Jha, S. OmniUnpack: Fast, Generic, and Safe Unpacking of Malware. In Proceedings of the 23rd Annual Computer Security Applications Conference (2007).

[27]

Martignoni, L., Paleari, R., and Bruschi, D. A Framework for Behavior-Based Malware Analysis in the Cloud. In Proceedings of the 5th International Conference on Information Systems Security (2009).

Digital Library

[28]

Nuttall, M. A Brief Survey of Systems Providing Process or Object Migration Facilities. ACM SIGOPS Operating Systems Review 28 (1994).

Digital Library

[29]

Osman, S., Subhraveti, D., Su, G., and Nieh, J. The Design and Implementation of Zap: a System for Migrating Computing Environments. ACM SIGOPS Operating Systems Review 36 (2002).

Digital Library

[30]

Payne, B., de Carbone, M., and Lee, W. Secure and Flexible Monitoring of Virtual Machines. In Proceedings of the 23rd Annual Computer Security Applications Conference (2007).

[31]

Payne, B. D., Carbone, M., Sharif, M., and Lee, W. Lares: An Architecture for Secure Active Monitoring Using Virtualization. In Proceedings of the 29th IEEE Symposium on Security and Privacy (2008).

Digital Library

[32]

Provos, N. Improving Host Security with System Call Policies. In Proceedings of the 12th USENIX Security Symposium (2003).

Digital Library

[33]

Royal, P., Halpin, M., Dagon, D., Edmonds, R., and Lee, W. PolyUnpack: Automating the Hidden-Code Extraction of Unpack-Executing Malware. In Proceedings of the 22nd Annual Computer Security Applications Conference (2006).

Digital Library

[34]

Sharif, M. I., Lee, W., Cui, W., and Lanzi, A. Secure In-VM Monitoring Using Hardware Virtualization. In Proceedings of the 16th ACM Conference on Computer and Communications Security (2009).

Digital Library

[35]

Smith, J. M. A survey of process migration mechanisms. ACM SIGOPS Operating Systems Review 22 (1988).

Digital Library

[36]

Smith, J. M. The Design and Implementation of Berkeley Lab's Linux Checkpoint/Restart. Berkeley Lab Technical Report (2002).

[37]

Srivastava, A., and Giffin, J. Tamper-Resistant, Application-Aware Blocking of Malicious Network Connections. In Proceedings of the 11th International Symposium on Recent Advances in Intrusion Detection (2008).

Digital Library

[38]

Srivastava, A., and Giffin, J. Efficient Monitoring of Untrusted Kernel-mode Execution. In Proceedings of the 18th Annual Network and Distributed Systems Security Symposium (2011).

[39]

Ta-Min, R., Litty, L., and Lie, D. Splitting Interfaces: Making Trust between Applications and Operating Systems Configurable. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (2006).

Digital Library

[40]

Wang, Z., and Jiang, X. HyperSafe: A Lightweight Approach to Provide Lifetime Hypervisor Control-Flow Integrity. Proceedings of the 31st IEEE Symposium on Security and Privacy (2010).

Digital Library

Cited By

Zheng LZhang JWang XLin FMeng Z(2024)Multimodal-based abnormal behavior detection method in virtualization environmentComputers & Security10.1016/j.cose.2024.103908143(103908)Online publication date: Aug-2024
https://doi.org/10.1016/j.cose.2024.103908
Chen ZQiu HDing X(2024)DScope: To Reliably and Securely Acquire Live Data from Kernel-Compromised ARM DevicesComputer Security – ESORICS 202310.1007/978-3-031-51482-1_14(271-289)Online publication date: 11-Jan-2024
https://doi.org/10.1007/978-3-031-51482-1_14
Zheng LZhang JLin FWang X(2023)Feature-Fusion-Based Abnormal-Behavior-Detection Method in Virtualization EnvironmentElectronics10.3390/electronics1216338612:16(3386)Online publication date: 9-Aug-2023
https://doi.org/10.3390/electronics12163386
Show More Cited By

Index Terms

Process out-grafting: an efficient "out-of-VM" approach for fine-grained process execution monitoring
1. Security and privacy
  1. Intrusion/anomaly detection and malware mitigation
  2. Systems security
    1. Operating systems security

Recommendations

Secure and robust monitoring of virtual machines through guest-assisted introspection
RAID'12: Proceedings of the 15th international conference on Research in Attacks, Intrusions, and Defenses

Current monitoring solutions for virtual machines do not incorporate both security and robustness. Out-of-guest applications achieve security by using virtual machine introspection and not relying on in-guest components, but do not achieve robustness ...
RapidVMI: Fast and multi-core aware active virtual machine introspection
ARES '21: Proceedings of the 16th International Conference on Availability, Reliability and Security

Virtual machine introspection (VMI) is a technique for the external monitoring of virtual machines. Through previous work, it became apparent that VMI can contribute to the security of distributed systems and cloud architectures by facilitating ...
Libvmi: A Library for Bridging the Semantic Gap between Guest OS and VMM
CIT '12: Proceedings of the 2012 IEEE 12th International Conference on Computer and Information Technology

Semantic gap is one of the most important problems in the virtualized computer systems. Solving this problem not only helps to develop security and virtual machine monitoring applications, but also benefits for VMM resource management and VMM-based ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CCS '11: Proceedings of the 18th ACM conference on Computer and communications security

October 2011

742 pages

ISBN:9781450309486

DOI:10.1145/2046707

General Chair:
Yan Chen
Northwestern University, USA
,
Program Chairs:
George Danezis
Microsoft Research Cambridge, UK
,
Vitaly Shmatikov
University of Texas at Austin, USA

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CCS'11

Sponsor:

SIGSAC

CCS'11: the ACM Conference on Computer and Communications Security

October 17 - 21, 2011

Illinois, Chicago, USA

Acceptance Rates

CCS '11 Paper Acceptance Rate 60 of 429 submissions, 14%;

Overall Acceptance Rate 1,261 of 6,999 submissions, 18%

Upcoming Conference

CCS '24

Sponsor:
sigsac

ACM SIGSAC Conference on Computer and Communications Security

October 14 - 18, 2024

Salt Lake City , UT , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

66
Total Citations
View Citations
879
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)4

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zheng LZhang JWang XLin FMeng Z(2024)Multimodal-based abnormal behavior detection method in virtualization environmentComputers & Security10.1016/j.cose.2024.103908143(103908)Online publication date: Aug-2024
https://doi.org/10.1016/j.cose.2024.103908
Chen ZQiu HDing X(2024)DScope: To Reliably and Securely Acquire Live Data from Kernel-Compromised ARM DevicesComputer Security – ESORICS 202310.1007/978-3-031-51482-1_14(271-289)Online publication date: 11-Jan-2024
https://doi.org/10.1007/978-3-031-51482-1_14
Zheng LZhang JLin FWang X(2023)Feature-Fusion-Based Abnormal-Behavior-Detection Method in Virtualization EnvironmentElectronics10.3390/electronics1216338612:16(3386)Online publication date: 9-Aug-2023
https://doi.org/10.3390/electronics12163386
AghamirMohammadAli SMomeni BSalimi SKharrazi M(2023)Blue-Pill Oxpecker: A VMI Platform for Transactional ModificationIEEE Transactions on Cloud Computing10.1109/TCC.2021.306782911:1(1-12)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TCC.2021.3067829
Ding X(2023)How to Resuscitate a Sick VM in the Cloud2023 53rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks - Supplemental Volume (DSN-S)10.1109/DSN-S58398.2023.00030(89-93)Online publication date: Jun-2023
https://doi.org/10.1109/DSN-S58398.2023.00030
Zhang YZhong NYou WZou YJian KXu JSun JLiu BHuo W(2022)NDFuzz: a non-intrusive coverage-guided fuzzing framework for virtualized network devicesCybersecurity10.1186/s42400-022-00120-15:1Online publication date: 1-Nov-2022
https://doi.org/10.1186/s42400-022-00120-1
Ge XKuo HCui WYin HStavrou ACremers CShi E(2022)HecateProceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security10.1145/3548606.3560592(1231-1242)Online publication date: 7-Nov-2022
https://dl.acm.org/doi/10.1145/3548606.3560592
Tian DZhao RMa RJia XShen QHu CLiu W(2022)MDCD: A malware detection approach in cloud using deep learningTransactions on Emerging Telecommunications Technologies10.1002/ett.458433:11Online publication date: 18-Jun-2022
https://doi.org/10.1002/ett.4584
Hong JDing X(2021)A Novel Dynamic Analysis Infrastructure to Instrument Untrusted Execution Flow Across User-Kernel Spaces2021 IEEE Symposium on Security and Privacy (SP)10.1109/SP40001.2021.00024(1902-1918)Online publication date: May-2021
https://doi.org/10.1109/SP40001.2021.00024
Lutas ASebestyen GTosa RColesa A(2021)VE-VMI: High-Performance Virtual Machine Introspection Based on Virtualization Exception2021 20th International Symposium on Parallel and Distributed Computing (ISPDC)10.1109/ISPDC52870.2021.9521609(73-80)Online publication date: 28-Jul-2021
https://doi.org/10.1109/ISPDC52870.2021.9521609
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents