Abstract
Code obfuscation is often utilized by authors of malware to protect it from detection or to hide its maliciousness from code analysis. Obfuscation stealth describes how difficult it is to determine which protection technique has been applied to a program and which parts of the code have been protected. In previous literature, most of the presented obfuscation identification methods analyze the program code itself (for example, the frequency and distribution of opcodes). However, simple countermeasures such as instruction substitution can have a negative impact on the identification rate. In this paper, we present a novel approach for an accurate obfuscation identification model based on a combination of multiple code complexity metrics. An evaluation with 4124 samples protected with 11 different obfuscations, combinations of obfuscations, and various compiler configurations demonstrates an overall classification accuracy of 86.5%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abran, A., Lopez, M., Habra, N.: An analysis of the mccabe cyclomatic complexity number. In: Proceedings of the 14th International Workshop on Software Measurement (IWSM) IWSM-Metrikon, pp. 391–405 (2004)
Bacci, A., Bartoli, A., Martinelli, F., Medvet, E., Mercaldo, F.: Detection of obfuscation techniques in android applications. In: Proceedings of the 13th International Conference on Availability, Reliability and Security, pp. 1–9 (2018)
Basili, V.R., Perricone, B.T.: Software errors and complexity: an empirical investigation0. Commun. ACM 27(1), 42–52 (1984)
Brosch, T., Morgenstern, M.: Runtime packers: the hidden problem. Black Hat USA (2006)
Canavese, D., Regano, L., Basile, C., Viticchié, A.: Estimating software obfuscation potency with artificial neural networks. In: Livraga, G., Mitchell, C. (eds.) STM 2017. LNCS, vol. 10547, pp. 193–202. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68063-7_13
Coleman, D., Oman, P., Ash, D., Lowther, B.: Using metrics to evaluate software system maintainability. Computer 27(08), 44–49 (1994)
Collberg, C., Thomborson, C., Low, D.: A taxonomy of obfuscating transformations. Technical report, Department of Computer Science, The University of Auckland, New Zealand (1997)
Collberg, C., Thomborson, C., Low, D.: Manufacturing cheap, resilient, and stealthy opaque constructs. In: Proceedings of the 25th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pp. 184–196 (1998)
Ebad, S.A., Darem, A.A., Abawajy, J.H.: Measuring software obfuscation quality - a systematic literature review. IEEE Access 9, 99024–99038 (2021)
Ebert, C., Cain, J., Antoniol, G., Counsell, S., Laplante, P.: Cyclomatic complexity. IEEE Softw. 33(6), 27–29 (2016)
Fitzpatrick, J.: Applying the ABC metric to C, C++, and Java. Technical report, C++ report (1997)
Gibert, D., Mateu, C., Planes, J., Vicens, R.: Classification of malware by using structural entropy on convolutional neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1, April 2018. https://doi.org/10.1609/aaai.v32i1.11409, https://ojs.aaai.org/index.php/AAAI/article/view/11409
Halstead, M.H.: Elements of Software Science. Operating and Programming Systems Series. Elsevier Science Inc., USA (1977)
Hatton, L.: Re-examining the defect-density versus component size distribution. IEEE Softw. 110 (1997)
Honglei, T., Wei, S., Yanan, Z.: The research on software metrics and software complexity metrics. In: 2009 International Forum on Computer Science-Technology and Applications, vol. 1, pp. 131–136. IEEE (2009)
Ikerionwu, C.: Cyclomatic complexity as a software metric. Int. J. Acad. Res. 2(3) (2010)
Junod, P., Rinaldini, J., Wehrli, J., Michielin, J.: Obfuscator-LLVM-software protection for the masses. In: 2015 IEEE/ACM 1st International Workshop on Software Protection, pp. 3–9. IEEE (2015)
Kanzaki, Y., Monden, A., Collberg, C.: Code artificiality: a metric for the code stealth based on an n-gram model. In: 2015 IEEE/ACM 1st International Workshop on Software Protection, pp. 31–37. IEEE (2015)
Khan, A.A., Mahmood, A., Amralla, S.M., Mirza, T.H.: Comparison of software complexity metrics. Int. J. Comput. Netw. Technol. 4(01) (2016)
Kim, J., Kang, S., Cho, E.-S., Paik, J.-Y.: LOM: lightweight classifier for obfuscation methods. In: Kim, H. (ed.) WISA 2021. LNCS, vol. 13009, pp. 3–15. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-89432-0_1
Kurtukova, A., Romanov, A., Shelupanov, A.: Source code authorship identification using deep neural networks. Symmetry 12(12) (2020)
Madi, A., Zein, O.K., Kadry, S.: On the improvement of cyclomatic complexity metric. Int. J. Softw. Eng. Appl. 7(2), 67–82 (2013)
Madou, M., Anckaert, B., De Bus, B., De Bosschere, K., Cappaert, J., Preneel, B.: On the effectiveness of source code transformations for binary obfuscation. In: Proceedings of the International Conference on Software Engineering Research and Practice (SERP06), pp. 527–533. CSREA Press (2006)
Mason, J., Small, S., Monrose, F., MacManus, G.: English shellcode. In: Proceedings of the 16th ACM Conference on Computer and Communications Security, pp. 524–533 (2009)
McCabe, T.J.: A complexity measure. IEEE Trans. Softw. Eng. SE-2(4), 308–320 (1976). https://doi.org/10.1109/TSE.1976.233837
Morgenstern, M., Pilz, H.: Useful and useless statistics about viruses and anti-virus programs. In: Proceedings of the CARO Workshop (2010)
Myers, G.J.: An extension to the cyclomatic measure of program complexity. SIGPLAN Not. 12(10), 61–64 (1977)
Nagra, J., Collberg, C.: Surreptitious Software: Obfuscation, Watermarking, and Tamperproofing for Software Protection: Obfuscation, Watermarking, and Tamperproofing for Software Protection. Pearson Education (2009)
Necula, G.C., McPeak, S., Weimer, W.: Cil: intermediate language and tools for analysis and transformation of C programs (2002)
Oman, P., Hagemeister, J.: Metrics for assessing a software system’s maintainability. In: Proceedings Conference on Software Maintenance 1992, pp. 337–344 (1992)
Oman, P., Hagemeister, J.: Construction and testing of polynomials predicting software maintainability. J. Syst. Softw. 24(3), 251–266 (1994). Oregon Workshop on Software Metrics
Rahbarinia, B., Balduzzi, M., Perdisci, R.: Exploring the long tail of (malicious) software downloads. In: 2017 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 391–402. IEEE (2017)
Sarwar, M.M.S., Shahzad, S., Ahmad, I.: Cyclomatic complexity: the nesting problem. In: Eighth International Conference on Digital Information Management (ICDIM 2013), pp. 274–279. IEEE (2013)
Sebastian, S.A., Malgaonkar, S., Shah, P., Kapoor, M., Parekhji, T.: A study & review on code obfuscation. In: 2016 World Conference on Futuristic Trends in Research and Innovation for Social Welfare, pp. 1–6. IEEE (2016)
Sellers, B.H.: Modularization and Mccabe’s Cyclomatic complexity. Commun. ACM 35(12), 17–20 (1992)
Shen, V.Y., Yu, T.J., Thebaut, S.M., Paulsen, L.R.: Identifying error-prone software-an empirical study. IEEE Trans. Softw. Eng. (4), 317–324 (1985)
Snoek, J., Larochelle, H., Adams, R.P.: Practical Bayesian optimization of machine learning algorithms. In: Advances in Neural Information Processing Systems, vol. 25 (2012)
Wang, Y., Rountev, A.: Who changed you? Obfuscator identification for android. In: 2017 IEEE/ACM 4th International Conference on Mobile Software Engineering and Systems (MOBILESoft), pp. 154–164. IEEE (2017)
Withrow, C.: Error density and size in ADA software. IEEE Softw. 7(1), 26–30 (1990)
Wu, Z., Gianvecchio, S., Xie, M., Wang, H.: Mimimorphism: a new approach to binary code obfuscation. In: Proceedings of the 17th ACM Conference on Computer and Communications Security, pp. 536–546 (2010)
Yu, S., Zhou, S.: A survey on metric of software complexity. In: 2010 2nd IEEE International Conference on Information Management and Engineering, pp. 352–356. IEEE (2010)
Acknowledgments
This research was funded in whole, or in part, by the Austrian Science Fund (FWF) I 3646-N31. For the purpose of open access, the author has applied a CC BY public copyright license to any Author Accepted Manuscript version arising from this submission.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
A Specifications
A Specifications
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Schrittwieser, S. et al. (2024). Modeling Obfuscation Stealth Through Code Complexity. In: Katsikas, S., et al. Computer Security. ESORICS 2023 International Workshops. ESORICS 2023. Lecture Notes in Computer Science, vol 14399. Springer, Cham. https://doi.org/10.1007/978-3-031-54129-2_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-54129-2_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-54128-5
Online ISBN: 978-3-031-54129-2
eBook Packages: Computer ScienceComputer Science (R0)