Article

Hybrid analysis and control of malware

Authors:

Kevin A. Roundy,

Barton P. MillerAuthors Info & Claims

RAID'10: Proceedings of the 13th international conference on Recent advances in intrusion detection

Pages 317 - 338

Published: 15 September 2010 Publication History

Abstract

Malware attacks necessitate extensive forensic analysis efforts that are manual-labor intensive because of the analysis-resistance techniques that malware authors employ. The most prevalent of these techniques are code unpacking, code overwriting, and control transfer obfuscations. We simplify the analyst's task by analyzing the code prior to its execution and by providing the ability to selectively monitor its execution. We achieve pre-execution analysis by combining static and dynamic techniques to construct control- and data-flow analyses. These analyses form the interface by which the analyst instruments the code. This interface simplifies the instrumentation task, allowing us to reduce the number of instrumented program locations by a hundred-fold relative to existing instrumentation-based methods of identifying unpacked code. We implement our techniques in SD-Dyninst and apply them to a large corpus of malware, performing analysis tasks such as code coverage tests and call-stack traversals that are greatly simplified by hybrid analysis.

References

[1]

Computer economics 2007 malware report: The economic impact of viruses, spyware, adware, botnets, and other malicious code (2007).

[2]

Darkparanoid virus (1998).

[3]

Offensive computing, http://www.offensivecomputing.net

[4]

Anckaert, B., Madou, M., Bosschere, K.D.: A model for self-modifying code. In: Information Hiding, Alexandria, VA, pp. 232-248 (2007).

Digital Library

[5]

Balakrishnan, G., Reps, T.: Analyzing memory accesses in x86 executables. In: International Conference on Compiler Construction, New York, NY, pp. 5-23 (2004).

[6]

Bayer, U., Moser, A., Kruegel, C., Kirda, E.: Dynamic analysis of malicious code. Journal in Computer Virology 2(1), 66-77 (2006).

[7]

Bellard, F.: QEMU, a fast and portable dynamic translator. In: USENIX Annual Technical Conference, Anaheim, CA, pp. 41-46 (2005).

Digital Library

[8]

BitDefender: BitDefender anti-virus technology. White Paper (2007).

[9]

Bustamante, P.: Malware prevalence. Panda Research web article (2008).

[10]

Bustamante, P.: Packer (r)evolution. Panda Research web article (2008).

[11]

Bustamante, P.: Personal correspondence (2009).

[12]

Chiang, K., Lloyd, L.: A case study of the rustock rootkit and spam bot. In: First Conference on Hot Topics in Understanding Botnets, Cambridge, MA (2007).

Digital Library

[13]

Cifuentes, C., Emmerik, M.V.: UQBT: adaptable binary translation at low cost. Computer 33(3), 60-66 (2000).

Digital Library

[14]

Collberg, C., Thomborson, C., Low, D.: Manufacturing cheap, resilient, and stealthy opaque constructs. In: Symposium on Principles of Programming Languages, San Diego, CA, pp. 184-196 (1998).

Digital Library

[15]

Coogan, K., Debray, S., Kaochar, T., Townsend, G.: Automatic static unpacking of malware binaries. In: Working Conference on Reverse Engineering, Antwerp, Belgium (2009).

Digital Library

[16]

Danehkar, A.: Inject your code into a portable executable file (2005), http://www.codeproject.com/KB/system/inject2exe.aspx

[17]

Dinaburg, A., Royal, P., Sharif, M., Lee, W.: Ether: Malware analysis via hardware virtualization extensions. In: Conference on Computer and Communications Security, Alexandria, VA (2008).

Digital Library

[18]

Ferrie, P.: Anti-unpacker tricks. In: International CARO Workshop. Amsterdam, Netherlands (2008).

[19]

Garfinkel, T., Rosenblum, M.: A virtual machine introspection based architecture for intrusion detection. In: Network and Distributed System Security Symposium, San Diego, CA (2003).

[20]

Guo, F., Ferrie, P., Chiueh, T.: A study of the packer problem and its solutions. In: Lippmann, R., Kirda, E., Trachtenberg, A. (eds.) RAID 2008. LNCS, vol. 5230, pp. 98-115. Springer, Heidelberg (2008).

Digital Library

[21]

Hind, M., Pioli, A.: Which pointer analysis should I use? In: International Symposium on Software Testing and Analysis, Portland, OR, pp. 113-123 (2000).

Digital Library

[22]

Hollingsworth, J.K., Miller, B.P., Cargille, J.: Dynamic program instrumentation for scalable performance tools. In: Scalable High Performance Computing Conference, Knoxville, TN (1994).

[23]

Kang, M.G., Poosankam, P., Yin, H.: Renovo: A hidden code extractor for packed executables. In: Workshop on Recurring Malcode, Alexandria, VA (2007).

Digital Library

[24]

Kruegel, C., Robertson, W., Valeur, F., Vigna, G.: Static disassembly of obfuscated binaries. In: USENIX Security Symposium, San Diego, CA (2004).

Digital Library

[25]

Linn, C., Debray, S.: Obfuscation of executable code to improve resistance to static disassembly. In: Conference on Computer and Communications Security, Washington, DC, pp. 290-299 (2003).

Digital Library

[26]

Linn, C., Debray, S., Andrews, G., Schwarz, B.: Stack analysis of x86 executables (2004) (manuscript).

[27]

Luk, C.K., Cohn, R., Muth, R., Patil, H., Klauser, A., Lowney, G., Wallace, S., Reddi, V.J., Hazelwood, K.: Pin: Building customized program analysis tools with dynamic instrumentation. In: Programming Language Design and Implementation, Chicago, IL, pp. 190-200 (2005).

Digital Library

[28]

Madou, M., Anckaert, B., de Sutter, B., Bosschere, K.D.: Hybrid static-dynamic attacks against software protection mechanisms. In: ACM Workshop on Digital Rights Management, Alexandria, VA, pp. 75-82 (2005).

Digital Library

[29]

Maebe, J., Bosschere, K.D.: Instrumenting self-modifying code. In: International Workshop on Automated and Algorithmic Debugging, Ghent, Belgium (2003).

[30]

Martignoni, L., Christodorescu, M., Jha, S.: Omniunpack: Fast, generic, and safe unpacking of malware. In: Annual Computer Security Applications Conference, Miami Beach, FL (2007).

[31]

Mirgorodskiy, A.V., Miller, B.P.: Autonomous analysis of interactive systems with self-propelled instrumentation. In: International Conference on Parallel Computing, San Jose, CA (2005).

[32]

Moser, A., Kruegel, C., Kirda, E.: Exploring multiple execution paths for malware analysis. In: Symposium on Security and Privacy, Oakland, CA, pp. 231-245 (2007).

Digital Library

[33]

Moser, A., Kruegel, C., Kirda, E.: Limits of static analysis for malware detection. In: Annual Computer Security Applications Conference, Miami Beach, FL (2007).

[34]

Nanda, S., Li, W., Lam, L.C., Cker Chiueh, T.: Bird: Binary interpretation using runtime disassembly. In: International Symposium on Code Generation and Optimization (CGO 2006), New York, NY, pp. 358-370 (2006).

Digital Library

[35]

Neumann, R.: Exepacker blacklisting part 2. Virus Bulletin pp. 10-13 (2007).

[36]

Nguyen, A.M., Schear, N., Jung, H., Godiyal, A., King, S.T., Nguyen, H.: Mavmm: A lightweight and purpose-built vmm for malware analysis. In: Annual Computer Security Applications Conference, Honolulu, HI (2009).

Digital Library

[37]

Perriot, F., Ferrie, P.: Principles and practise of x-raying. In: Virus Bulletin Conference, Chicago, IL, pp. 51-66 (2004).

[38]

Popov, I., Debray, S., Andrews, G.: Binary obfuscation using signals. In: USENIX Security Symposium, Boston, MA, pp. 275-290 (2007).

Digital Library

[39]

Porras, P., Saidi, H., Yegneswaran, V.: A multi-perspective analysis of the storm (peacomm) worm. SRI International Technical Report (2007).

[40]

Porras, P., Saidi, H., Yegneswaran, V.: An analysis of conficker's logic and rendezvous points. SRI International Technical Report (2009).

[41]

Quist, D., Ames, C.: Temporal reverse engineering. In: Blackhat, USA, Las Vegas, NV (2008).

[42]

Quist, D.A., Liebrock, L.M.: Visualizing compiled executables for malware analysis. In: Workshop on Visualization for Cyber Security, Atlantic City, NJ (2009).

[43]

Rosenblum, N.E., Zhu, X., Miller, B.P., Hunt, K.: Learning to analyze binary computer code. In: Conference on Artificial Intelligence, Chicago, IL (2008).

Digital Library

[44]

Royal, P., Halpin, M., Dagon, D., Edmonds, R., Lee, W.: PolyUnpack: Automating the hidden-code extraction of unpack-executing malware. In: Annual Computer Security Applications Conference, Miami Beach, FL, pp. 289-300 (2006).

Digital Library

[45]

Security, P.: Annual report Pandalabs (2008).

[46]

Sharif, M., Lanzi, A., Giffin, J., Lee, W.: Impeding malware analysis using conditional code obfuscation. In: Network and Distributed System Security Symposium, San Diego, CA (2008).

[47]

Sites, R.L., Chernoff, A., Kirk, M.B., Marks, M.P., Robinson, S.G.: Binary translation. Communications of the ACM 36(2), 69-81 (1993).

Digital Library

[48]

Srivastava, A., Edwards, A., Vo, H.: Vulcan: Binary transformation in a distributed environment. Technical Report MSR-TR-2001-50 (2001).

[49]

Srivastava, A., Eustace, A.: ATOM: a system for building customized program analysis tools. In: Programming Language Design and Implementation, Orlando, FL (1994).

Digital Library

[50]

Trilling, S.: Project green bay-calling a blitz on packers. In: CIO Digest: Strategies and Analysis from Symantec, p. 4 (2008).

[51]

Vigna, G.: Static disassembly and code analysis. In: Malware Detection. Advances in Information Security, vol. 35, pp. 19-42. Springer, Heidelberg (2007).

[52]

Yegneswaran, V., Saidi, H., Porras, P.: Eureka: A framework for enabling static analysis on malware. Technical Report SRI-CSL-08-01 (2008).

Cited By

Alrabaee SDebbabi MWang L(2022)A Survey of Binary Code Fingerprinting Approaches: Taxonomy, Methodologies, and FeaturesACM Computing Surveys10.1145/348686055:1(1-41)Online publication date: 17-Jan-2022
https://dl.acm.org/doi/10.1145/3486860
Cheng BMing JFu JPeng GChen TZhang XMarion JLie DMannan MBackes MWang X(2018)Towards Paving the Way for Large-Scale Windows Malware AnalysisProceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security10.1145/3243734.3243771(395-411)Online publication date: 15-Oct-2018
https://dl.acm.org/doi/10.1145/3243734.3243771
Jacobson EBernat AWilliams WMiller B(2014)Detecting Code Reuse Attacks with a Model of Conformant Program ExecutionProceedings of the 6th International Symposium on Engineering Secure Software and Systems - Volume 836410.1007/978-3-319-04897-0_1(1-18)Online publication date: 26-Feb-2014
https://dl.acm.org/doi/10.1007/978-3-319-04897-0_1
Show More Cited By

Index Terms

Hybrid analysis and control of malware
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Index terms have been assigned to the content through auto-classification.

Recommendations

The Detection of 8 Type Malware botnet using Hybrid Malware Analysis in Executable File Windows Operating Systems
ICEC '15: Proceedings of the 17th International Conference on Electronic Commerce 2015

Nowadays a lot of botnet are being used for the purpose of cybercrime such as distributed denial of services (DDos) or information stealing. Botnet is a collection of computers connected through Internet that has been taken over by an attacker using ...
Malware Detection by Static Checking and Dynamic Analysis of Executables

The advanced malware continue to be a challenge in digital world that signature-based detection techniques fail to conquer. The malware use many anti-detection techniques to mutate. Thus no virus scanner can claim complete malware detection even for ...
Hybrid Analysis Technique to detect Advanced Persistent Threats

Advanced persistent threats APT are major threats in the field of system and network security. They are extremely stealthy and use advanced evasion techniques like packing and behaviour obfuscation to hide their malicious behaviour and evade the ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

RAID'10: Proceedings of the 13th international conference on Recent advances in intrusion detection

September 2010

517 pages

ISBN:3642155111

Editors:
Somesh Jha
University of Wisconsin, Computer Sciences Department, Madison, WI
,
Robin Sommer
International Computer Science Institute, Berkeley, CA
,
Christian Kreibich
International Computer Science Institute, Berkeley, CA

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 15 September 2010

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Alrabaee SDebbabi MWang L(2022)A Survey of Binary Code Fingerprinting Approaches: Taxonomy, Methodologies, and FeaturesACM Computing Surveys10.1145/348686055:1(1-41)Online publication date: 17-Jan-2022
https://dl.acm.org/doi/10.1145/3486860
Cheng BMing JFu JPeng GChen TZhang XMarion JLie DMannan MBackes MWang X(2018)Towards Paving the Way for Large-Scale Windows Malware AnalysisProceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security10.1145/3243734.3243771(395-411)Online publication date: 15-Oct-2018
https://dl.acm.org/doi/10.1145/3243734.3243771
Jacobson EBernat AWilliams WMiller B(2014)Detecting Code Reuse Attacks with a Model of Conformant Program ExecutionProceedings of the 6th International Symposium on Engineering Secure Software and Systems - Volume 836410.1007/978-3-319-04897-0_1(1-18)Online publication date: 26-Feb-2014
https://dl.acm.org/doi/10.1007/978-3-319-04897-0_1
Roundy KMiller B(2013)Binary-code obfuscations in prevalent packer toolsACM Computing Surveys10.1145/2522968.252297246:1(1-32)Online publication date: 11-Jul-2013
https://dl.acm.org/doi/10.1145/2522968.2522972
Bernat AMiller BFoster JPollock L(2011)Anywhere, any-time binary instrumentationProceedings of the 10th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools10.1145/2024569.2024572(9-16)Online publication date: 5-Sep-2011
https://dl.acm.org/doi/10.1145/2024569.2024572
Bernat ARoundy KMiller BDwyer MTip F(2011)Efficient, sensitivity resistant binary instrumentationProceedings of the 2011 International Symposium on Software Testing and Analysis10.1145/2001420.2001432(89-99)Online publication date: 17-Jul-2011
https://dl.acm.org/doi/10.1145/2001420.2001432

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents