Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

Q-learning and LSTM based deep active learning strategy for malware defense in industrial IoT applications

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Edge devices are extensively used as intermediaries between the device and the service layer in an industrial Internet of things (IIoT) environment. These devices are quite vulnerable to malware attacks. Existing studies have worked on designing complex learning algorithms or deep architectures to accurately classify malware assuming that a sufficient number of labeled examples are provided. In the real world, getting labeled examples is one of the major issues for training any classification algorithm. Recent advances have allowed researchers to use active learning strategies that are trained on a handful of labeled examples to perform the classification task, but they are based on the selection of informative instances. This study integrates the Q-learning characteristics into an active learning framework, which allows the network to either request or predict a label during the training process. We proposed the use of phase space embedding, sparse autoencoder, and LSTM with the action-value function to classify malware applications while using a handful of labeled examples. The network relies on its uncertainty to either request or predict a label. The experimental results show that the proposed method can achieve better accuracy than the supervised learning strategy while using few labeled requests. The results also show that the trained network is resilient to the adversarial attacks, which proves the robustness of the proposed method. Additionally, this study explores the tradeoff between classification accuracy and number of label requests via the choice of rewards and the use of decision-level fusion strategies to boost the classification performance. Furthermore, we also provide a hypothetical framework as an implication of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Abadi M, Agarwal A, Barham P, et al (2016) TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

  2. Abraham A, Schlecht D, Magaofei, et al Mobile Security framework (MobSF). In: GitHub https://github.com/MobSF/Mobile-Security-Framework-MobSF. Accessed 19 Dec 2019

  3. Afonso VM, de Amorim MF, Grégio ARA, Junquera GB, de Geus PL (2015) Identifying android malware using dynamically obtained features. J Comput Virol Hacking Tech 11:9–17. https://doi.org/10.1007/s11416-014-0226-7

    Article  Google Scholar 

  4. Ahmed A, Krishnan VVG, Foroutan SA, Touhiduzzaman M, Rublein C, Srivastava A, Wu Y, Hahn A, Suresh S (2019) Cyber physical Security analytics for anomalies in transmission protection systems. IEEE Trans Ind Appl 55:6313–6323. https://doi.org/10.1109/TIA.2019.2928500

    Article  Google Scholar 

  5. Anderson HS, Kharkar A, Filar B, Roth P (2017) Evading machine learning malware detection. In: Black Hat. pp. 1–6

  6. Arp D, Spreitzenbarth M, Hübner M et al (2014) Drebin: effective and explainable detection of android malware in your pocket. In: Network and Distributed System Security Symposium. Internet Society, Reston, VA

  7. Arshad J, Azad MA, Abdeltaif MM, Salah K (2020) An intrusion detection framework for energy constrained IoT devices. Mech Syst Signal Process 136:106436. https://doi.org/10.1016/j.ymssp.2019.106436

    Article  Google Scholar 

  8. Bachman P, Sordoni A, Trischler A (2017) Learning algorithms for active learning. In: ICML’17: proceedings of the 34th international conference on machine learning. 301–310

  9. Boche H, Staczak S (2006) The Kullback–Leibler divergence and nonnegative matrices. IEEE Trans Inf Theory 52:5539–5545. https://doi.org/10.1109/TIT.2006.885488

    Article  MathSciNet  MATH  Google Scholar 

  10. Boyes H, Hallaq B, Cunningham J, Watson T (2018) The industrial internet of things (IIoT): an analysis framework. Comput Ind 101:1–12. https://doi.org/10.1016/j.compind.2018.04.015

    Article  Google Scholar 

  11. Cao L (1997) Practical method for determining the minimum embedding dimension of a scalar time series. Phys D Nonlinear Phenom 110:43–50. https://doi.org/10.1016/S0167-2789(97)00118-8

    Article  MATH  Google Scholar 

  12. Carter K, Raich R, Finn W, Hero A III (2011) Information-geometric dimensionality reduction. IEEE Signal Process Mag 28:89–99. https://doi.org/10.1109/MSP.2010.939536

    Article  Google Scholar 

  13. Chakrabarty S, Engels DW (2016) A secure IoT architecture for smart cities. In: 13th IEEE Annual Consumer Communications & Networking Conference (CCNC). IEEE, pp 812–813

  14. Cheng F, Wang J, Qu L, Qiao W (2018) Rotor-current-based fault diagnosis for DFIG wind turbine drivetrain gearboxes using frequency analysis and a deep classifier. IEEE Trans Ind Appl 54:1062–1071. https://doi.org/10.1109/TIA.2017.2773426

    Article  Google Scholar 

  15. Chu W, Zinkevich M, Li L, et al (2011) Unbiased online active learning in data streams. In: proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining - KDD ‘11. ACM Press, New York, p 195

  16. D’Angelo G, Ficco M, Palmieri F (2020) Malware detection in mobile environments based on autoencoders and API-images. J Parallel Distrib Comput 137:26–33. https://doi.org/10.1016/j.jpdc.2019.11.001

    Article  Google Scholar 

  17. Da Xu L, He W, Li S (2014) Internet of things in industries: a survey. IEEE Trans Ind Inform 10:2233–2243. https://doi.org/10.1109/TII.2014.2300753

    Article  Google Scholar 

  18. Dang H, Huang Y, Chang E-C (2017) Evading classifiers by morphing in the dark. In: Proceedings of the ACM SIGSAC conference on computer and communications Security - CCS ‘17. ACM Press, New York, pp. 119–133

  19. Desnos A, Gueguen G (2016) Androguard. In: GitHub. https://github.com/androguard/androguard. Accessed 16 Dec 2019

  20. Ducoffe M, Precioso F (2018) Adversarial active learning for deep networks: a margin based approach

  21. Garcia J, Hammad M, Malek S (2018) Lightweight, obfuscation-resilient detection and family identification of android malware. ACM Trans Softw Eng Methodol 26:1–29. https://doi.org/10.1145/3162625

    Article  Google Scholar 

  22. Gascon H, Yamaguchi F, Arp D, Rieck K (2013) Structural detection of android malware using embedded call graphs. In: Proceedings of the 2013 ACM workshop on artificial intelligence and security - AISec ‘13. ACM Press, New York, pp. 45–54

  23. Gong Z (2017) Craft image adversarial samples with Tensorflow. In: GitHub. https://github.com/gongzhitaao/tensorflow-adversarial/tree/v0.1.2. Accessed 21 Dec 2019

  24. Grosse K, Papernot N, Manoharan P, et al (2017) Adversarial examples for malware detection. In: European Symposium on Research in Computer Security. pp. 62–79

  25. Hadgu AT, Nigam A, Diaz-Aviles E (2015) Large-scale learning with AdaGrad on spark. In: IEEE international conference on big data (big data). IEEE, pp 2828–2830

  26. Hassen M, Carvalho MM, Chan PK (2017) Malware classification using static analysis based features. In: IEEE symposium series on computational intelligence (SSCI). IEEE, pp 1–7

  27. He M, He D (2017) Deep learning based approach for bearing fault diagnosis. IEEE Trans Ind Appl 53:3057–3065. https://doi.org/10.1109/TIA.2017.2661250

    Article  Google Scholar 

  28. Huang Y, Kou G, Peng Y (2017) Nonlinear manifold learning for early warnings in financial markets. Eur J Oper Res 258:692–702. https://doi.org/10.1016/j.ejor.2016.08.058

    Article  MathSciNet  MATH  Google Scholar 

  29. Jerald AV, Rabara SA, Premila Bai D (2016) Secure IoT architecture for integrated smart services environment. In: 3rd international conference on computing for sustainable global development (INDIACom). Pp 800–805

  30. Kapratwar A (2016) Static and dynamic analysis for android malware detection. San Jose State University

  31. Karnouskos S (2011) Stuxnet worm impact on industrial cyber-physical system security. In: IECON 2011 - 37th Annual Conference of the IEEE Industrial Electronics Society. IEEE, pp 4490–4494

  32. Kaspersky (2017) Kaspersky Lab detects 360,000 new malicious files daily – up 11.5% from 2016. In: Kaspersky. https://www.kaspersky.com/about/press-releases/2017_kaspersky-lab-detects-360000-new-malicious-files-daily. Accessed 3 Mar 2020

  33. Khoda ME, Imam T, Kamruzzaman J, Gondal I, Rahman A (2019) Robust malware defense in industrial IoT applications using machine learning with selective adversarial samples. IEEE Trans Ind Appl:1–1. https://doi.org/10.1109/TIA.2019.2958530

  34. Khowaja SA, Lee S-L (2019) Hybrid and hierarchical fusion networks: a deep cross-modal learning architecture for action recognition. Neural Comput Appl 32:10423–10434. https://doi.org/10.1007/s00521-019-04578-y

    Article  Google Scholar 

  35. Khowaja SA, Lee S-L (2020) Semantic image networks for human action recognition. Int J Comput Vis 128:393–419. https://doi.org/10.1007/s11263-019-01248-3

    Article  Google Scholar 

  36. Khowaja SA, Yahya BN, Lee S-L (2017) Hierarchical classification method based on selective learning of slacked hierarchy for activity recognition systems. Expert Syst Appl 88:165–177. https://doi.org/10.1016/j.eswa.2017.06.040

    Article  Google Scholar 

  37. Khowaja SA, Prabono AG, Setiawan F, Yahya BN, Lee SL (2018) Contextual activity based healthcare internet of things, services, and people (HIoTSP): an architectural framework for healthcare monitoring using wearable sensors. Comput Netw 145:190–206. https://doi.org/10.1016/j.comnet.2018.09.003

    Article  Google Scholar 

  38. Khowaja SA, Khuwaja P, Ismaili IA (2019) A framework for retinal vessel segmentation from fundus images using hybrid feature set and hierarchical classification. Signal, Image Video Process 13:379–387. https://doi.org/10.1007/s11760-018-1366-x

    Article  Google Scholar 

  39. Khuwaja P, Khowaja SA, Khoso I, Lashari IA (2020) Prediction of stock movement using phase space reconstruction and extreme learning machines. J Exp Theor Artif Intell 32:59–79. https://doi.org/10.1080/0952813X.2019.1620870

    Article  Google Scholar 

  40. Lantz P, Spreitzenbarth M, Terra F, et al (2014) DroidBox. In: GitHub

  41. Laskey M, Staszak S, Hsieh WY-S et al (2016) SHIV: reducing supervisor burden in DAgger using support vectors for efficient learning from demonstrations in high dimensional state spaces. In: 2016 IEEE international conference on robotics and automation (ICRA). IEEE, pp 462–469

  42. Liu Y (2004) Active learning with support vector machine applied to gene expression data for Cancer classification. J Chem Inf Comput Sci 44:1936–1941. https://doi.org/10.1021/ci049810a

    Article  Google Scholar 

  43. Lughofer E (2012) Single-pass active learning with conflict and ignorance. Evol Syst 3:251–271. https://doi.org/10.1007/s12530-012-9060-7

    Article  Google Scholar 

  44. Martín A, Lara-Cabrera R, Camacho D (2019) Android malware detection through hybrid features fusion and ensemble classifiers: the AndroPyTool framework and the OmniDroid dataset. Inf Fusion 52:128–142. https://doi.org/10.1016/j.inffus.2018.12.006

    Article  Google Scholar 

  45. McLaughlin N, Martinez del Rincon J, Kang B, et al (2017) Deep android malware detection. In: Proceedings of the seventh ACM on conference on data and application Security and privacy. ACM, New York 301–308

  46. McWilliams G, Sezer S, Yerima SY (2014) Analysis of Bayesian classification-based approaches for android malware detection. IET Inf Secur 8:25–36. https://doi.org/10.1049/iet-ifs.2013.0095

    Article  Google Scholar 

  47. Meng S, Huang W, Yin X et al (2020) Security-aware dynamic scheduling for real-time optimization in cloud-based industrial applications. IEEE trans Ind inform:1–1. https://doi.org/10.1109/TII.2020.2995348

  48. Mnih V, Kavukcuoglu K, Silver D, et al (2013) Playing atari with deep reinforcementlearning. In: Proc. of the conference on neural information processing systems (NIPS), workshop on deep learning. 1–9

  49. Mnih V, Badia AP, Mirza M, et al (2016) Asynchronous methods for deep reinforcement learning. In: proceedings of the 33 rd international conference on machine learning. Pp 1928–1937

  50. Moskovitch R, Nissim N, Englert R, Elovici Y (2008) Active learning to improve the detection of unknown computer worms activity. In: 11th international conference on information fusion. Pp 1–8

  51. Muttik I, Yerima SY, Sezer S (2015) High accuracy android malware detection using ensemble learning. IET Inf Secur 9:313–320. https://doi.org/10.1049/iet-ifs.2014.0099

    Article  Google Scholar 

  52. Naeem H, Ullah F, Naeem MR, Khalid S, Vasan D, Jabbar S, Saeed S (2020) Malware detection in industrial internet of things based on hybrid image visualization and deep learning model. Ad Hoc Netw 105:102154. https://doi.org/10.1016/j.adhoc.2020.102154

    Article  Google Scholar 

  53. Nissim N, Boland MR, Tatonetti NP, Elovici Y, Hripcsak G, Shahar Y, Moskovitch R (2016) Improving condition severity classification with an efficient active learning based framework. J Biomed Inform 61:44–54. https://doi.org/10.1016/j.jbi.2016.03.016

    Article  Google Scholar 

  54. Nix R, Zhang J (2017) Classification of android apps and malware using deep neural networks. In: international joint conference on neural networks (IJCNN). IEEE, pp 1871–1878

  55. Pang K, Dong M, Wu Y, Hospedales T (2018) Meta-learning transferable active learning policies by deep reinforcement learning

  56. Papernot N, McDaniel P, Wu X, et al (2016) Distillation as a defense to adversarial perturbations against deep neural networks. In: 2016 IEEE symposium on Security and privacy (SP). IEEE, pp 582–597

  57. Pi L, Lu Z, Sagduyu Y, Chen S (2016) Defending active learning against adversarial inputs in automated document classification. In: IEEE global conference on signal and information processing (GlobalSIP). IEEE, pp 257–261

  58. Playdrone goodware dataset,. https://archive.org/details/playdrone-apks. Accessed 5 Dec 2019

  59. Povinelli RJ, Johnson MT, Lindgren AC, Ye J (2004) Time series classification using Gaussian mixture models of reconstructed phase spaces. IEEE Trans Knowl Data Eng 16:779–783. https://doi.org/10.1109/TKDE.2004.17

    Article  Google Scholar 

  60. Rajesh S, Paul V, Menon V, Khosravi M (2019) A secure and efficient lightweight symmetric encryption scheme for transfer of text files between embedded IoT devices. Symmetry (Basel) 11:293. https://doi.org/10.3390/sym11020293

    Article  Google Scholar 

  61. Rashidi B, Fung C, Bertino E (2017) Android malicious application detection using support vector machine and active learning. In: 13th international conference on network and service management (CNSM). IEEE, pp 1–9

  62. Rawlinson K (2014) HP study reveals 70 percent of internet of things devices vulnerable to attack. In: HP. https://www8.hp.com/us/en/hp-news/press-release.html?id=1744676. Accessed 3 Mar 2020

  63. Revivo I, Caspi O (2015) CuckooDroid - automated android malware analysis with cuckoo sandbox. In: GitHub

  64. Sahin DO, Kural OE, Akleylek S, Kilic E (2018) New results on permission based static analysis for android malware. In: 6th international symposium on digital forensic and Security (ISDFS). IEEE, pp 1–4

  65. Samra AAA, Ghanem OA (2013) Analysis of clustering technique in android malware detection. In: Seventh International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing. IEEE, pp. 729–733

  66. Santoro A, Bartunov S, Botvinick M et al (2016) One-shot learning with memory-augmented neural networks. In: Proc. of the international conference on machine learning (ICML). Pp 1–13

  67. Security H (2020) Malware and ransomware attack volume down due to more targeted attacks. In: Help Net Secur. https://www.helpnetsecurity.com/2020/02/05/ransomware-attack-volume-down/. Accessed 3 Mar 2020

  68. Settles B (2009) Active learning literature survey. University of Wisconsin-Madison, Department of Computer Sciences

  69. Sharmeen S, Huda S, Abawajy JH, Ismail WN, Hassan MM (2018) Malware threats and detection for industrial Mobile-IoT networks. IEEE Access 6:15941–15957. https://doi.org/10.1109/ACCESS.2018.2815660

    Article  Google Scholar 

  70. Shi Y, Sagduyu YE, Davaslioglu K, Li JH (2018) Active deep learning attacks under strict rate limitations for online API calls. In: IEEE international symposium on Technologies for Homeland Security (HST). IEEE, pp 1–6

  71. Shrivastava G, Kumar P (2019) SensDroid: analysis for malicious activity risk of android application. Multimed Tools Appl 78:35713–35731. https://doi.org/10.1007/s11042-019-07899-1

    Article  Google Scholar 

  72. Su X, Zhang D, Li W, Zhao K (2016) A Deep Learning Approach to Android Malware Feature Learning and Detection. In: 2016 IEEE Trustcom/BigDataSE/ISPA. IEEE, pp 244–251

  73. Sun H, Wang X, Buyya R, Su J (2017) CloudEyes: cloud-based malware detection with reversible sketch for resource-constrained internet of things (IoT) devices. Softw Pract Exp 47:421–441. https://doi.org/10.1002/spe.2420

    Article  Google Scholar 

  74. Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9:293–300. https://doi.org/10.1023/A:1018628609742

    Article  Google Scholar 

  75. Taken F (1981) Detecting strange attractors in turbulence. In: Rand DA, young L-S (eds) dynamical systems and turbulence. Lecture notes in computer science, vol. 898, springer-Verlag, pp 366–381

  76. Tong F, Yan Z (2017) A hybrid approach of mobile malware detection in android. J Parallel Distrib Comput 103:22–31. https://doi.org/10.1016/j.jpdc.2016.10.012

    Article  Google Scholar 

  77. Torres JLG, Catania CA, Veas E (2019) Active learning approach to label network traffic datasets. J Inf Secur Appl 49:102388. https://doi.org/10.1016/j.jisa.2019.102388

    Article  Google Scholar 

  78. Vinyals O, Blundell C, Lillicrap TP, et al (2016) Matching networks for one shot learning. In: Advances in Neural Information Processing Systems. pp. 3630–3638

  79. Virusshare malware dataset. https://virusshare.com/. Accessed 5 Dec 2019

  80. Wang Z, Cai J, Cheng S, Li W (2016) DroidDeepLearner: identifying android malware using deep learning. In: IEEE 37th Sarnoff symposium. IEEE, pp 160–165

  81. Wang Z, Schaul T, Hessel M, et al (2016) Dueling network architectures for deep reinforcement learning. In: proceedings of the 33rd international conference on machine learning. Pp 1995–2003

  82. Woodward M, Finn C (2017) Active one-shot learning

  83. Wright J, Yang AY, Ganesh A, Sastry SS, Yi Ma (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31:210–227. https://doi.org/10.1109/TPAMI.2008.79

    Article  Google Scholar 

  84. Xiao X, Zhang S, Mercaldo F, Hu G, Sangaiah AK (2019) Android malware detection based on system call sequences and LSTM. Multimed Tools Appl 78:3979–3999. https://doi.org/10.1007/s11042-017-5104-0

    Article  Google Scholar 

  85. Xu K, Li Y, Deng RH (2016) ICCDetector: ICC-based malware detection on android. IEEE Trans Inf Forensics Secur 11:1252–1264. https://doi.org/10.1109/TIFS.2016.2523912

    Article  Google Scholar 

  86. Yan LK, Yin H (2012) DroidScope: seamlessly reconstructing the OS and Dalvik semantic views for dynamic android malware analysis. In: Security’12: proceedings of the 21st USENIX conference on Security symposium. Pp 1–16

  87. Yang W, Kong D, Xie T, Gunter CA (2017) Malware Detection in Adversarial Settings. In: Proceedings of the 33rd Annual Computer Security Applications Conference on - ACSAC 2017. ACM press, New York, New York, USA, pp. 288–302

  88. Yuan Z, Lu Y, Wang Z, Xue Y (2014) Droid-sec: deep learning in android malware detection. In: Proceedings of the ACM conference on SIGCOMM - SIGCOMM ‘14. ACM Press, New York, pp. 371–372

  89. Yuan Z, Lu Y, Xue Y (2016) Droiddetector: android malware characterization and detection using deep learning. Tsinghua Sci Technol 21:114–123. https://doi.org/10.1109/TST.2016.7399288

    Article  Google Scholar 

  90. Zhang J, Cho K (2016) Query-efficient imitation learning for end-to-end autonomous driving

  91. Zhao P, Hoi SCH (2013) Cost-sensitive online active learning with application to malicious URL detection. In: proceedings of the 19th ACM SIGKDD international conference on knowledge discovery and data mining - KDD ‘13. ACM Press, New York, p 919

  92. Zhou Y, Kantarcioglu M, Xi B (2019) Adversarial active learning in the presence of weak and malicious oracles. In: PAKDD 2019: trends and applications in knowledge discovery and data mining. Pp 77–89

  93. Zhou Q, Feng F, Shen Z, Zhou R, Hsieh MY, Li KC (2019) A novel approach for mobile malware classification and detection in android systems. Multimed Tools Appl 78:3529–3552. https://doi.org/10.1007/s11042-018-6498-z

    Article  Google Scholar 

  94. Zhu H (2017) Active learning framework for android unknown malware detection. In: Automotive, Mechanical and Electrical Engineering. CRC Press, pp. 345–348

Download references

Acknowledgments

The authors would like to thank Dr. Faraz Bughio for his constructive comments on the manuscript.

Author information

Authors and Affiliations

  1. Department of Telecommunication, Faculty of Engineering and Technology, University of Sindh, Jamshoro, Pakistan

    Sunder Ali Khowaja

  2. Institute of Business Administration, University of Sindh, Jamshoro, Pakistan

    Parus Khuwaja

Authors
  1. Sunder Ali Khowaja
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Parus Khuwaja
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sunder Ali Khowaja.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khowaja, S.A., Khuwaja, P. Q-learning and LSTM based deep active learning strategy for malware defense in industrial IoT applications. Multimed Tools Appl 80, 14637–14663 (2021). https://doi.org/10.1007/s11042-020-10371-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10371-0

Keywords

Search

Navigation