Abstract
The artificial immune system and network anomaly detection system are developed with common goals and principles considered. Moreover, artificial immune-based network anomaly detection can adaptively learn and dynamically detect threats. However, existing immune recognition algorithms suffer from the curse of dimensionality, hole problems, and detector inefficiency tolerance. In this paper, we proposed a novel immune detector training mechanism for network anomaly detection. First, a hybrid filter embedded feature selection algorithm is designed to comprehensively evaluate features and select the optimal subset. Then, candidate detectors are generated based on self antigens, and the nonself region is represented using complementary space to circumvent the hole problem. Finally, considering the training efficiency during the evolution of the candidate detectors, an antigen clustering feature tree is constructed to rapidly index the tolerance objects. Furthermore, the algorithm considers the effect of the collaboration of multiple mature detectors on candidate detectors, and a Monte Carlo-based coverage estimation algorithm is designed to achieve more accurate and fine-grained maturation tolerance of candidate detectors. The theoretical analysis shows that the time complexity of our algorithm is significantly reduced. The experimental results show that our algorithm not only improves the detection accuracy but also reduces the time cost of detector training.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
References
Javaheri D, Gorgin S, Lee JA et al (2023) Fuzzy logic-based ddos attacks and network traffic anomaly detection methods: Classification, overview, and future perspectives. Inf Sci 626:315–338. https://doi.org/10.1016/j.ins.2023.01.067
Ding C, Sun S, Zhao J (2023) Mst-gat: A multimodal spatial-temporal graph attention network for time series anomaly detection. Inf Fusion 89:527–536. https://doi.org/10.1016/j.inffus.2022.08.011
Ullah W, Hussain T, Khan ZA et al (2022) Intelligent dual stream cnn and echo state network for anomaly detection. Knowl-Based Syst 253(109):456. https://doi.org/10.1016/j.knosys.2022.109456
Jain M, Kaur G, Saxena V (2022) A k-means clustering and svm based hybrid concept drift detection technique for network anomaly detection. Expert Syst Appl 193(116):510. https://doi.org/10.1016/j.eswa.2022.116510
Chen J, Wang X, Su M et al (2021) A fast detector generation algorithm for negative selection. Appl Intell 51(7):4525–4547. https://doi.org/10.1007/s10489-020-02001-x
Yang G, Wang L, Yu R et al (2023) A modified gray wolf optimizer-based negative selection algorithm for network anomaly detection. Int J Intell Syst 2023. https://doi.org/10.1155/2023/8980876
Zhu Y, Li T, Lan X (2023) Feature selection optimized by the artificial immune algorithm based on genome shuffling and conditional lethal mutation. Appl Intell 53(11):13,972–13,992. https://doi.org/10.1007/s10489-022-03971-w
Li Z, Li T, He J et al (2021) A hybrid real-valued negative selection algorithm with variable-sized detectors and the k-nearest neighbors algorithm. Knowl-Based Syst 232(107):477. https://doi.org/10.1016/j.knosys.2021.107477
Zhang R, Xiao X (2018) A clone selection based real-valued negative selection algorithm. Complexity 2018. https://doi.org/10.1155/2018/2520940
He J, Chen W, Li T et al (2021) Hd-nsa: A real-valued negative selection algorithm based on hierarchy division. Appl Soft Comput 112(107):726. https://doi.org/10.1016/j.asoc.2021.107726
Sun X, Wang H, Liu S et al. (2022) Self-updating continual learning classification method based on artificial immune system. Appl Intell 52(11):12,817–12,843. https://doi.org/10.1007/s10489-021-03123-6
Gupta KD, Dasgupta D (2022) Negative selection algorithm research and applications in the last decade: A review. IEEE Trans Artif Intell 3(2):110–128. https://doi.org/10.1109/TAI.2021.3114661
Wen C, Changzhi W (2022) Combine labeled and unlabeled data for immune detector training with label propagation. Knowl-Based Syst 236(107):661. https://doi.org/10.1016/j.knosys.2021.107661
Xi L, Wang RD, Yao ZY et al (2021) Multisource neighborhood immune detector adaptive model for anomaly detection. IEEE Trans Evolutionary Comput 25(3):582–594. https://doi.org/10.1109/TEVC.2021.3058687
Aissa NB, Guerroumi M, Derhab A (2019) Nsnad: negative selection-based network anomaly detection approach with relevant feature subset. Neural Comput Appl 32:3475–3501. https://doi.org/10.1007/s00521-019-04396-2
Mafarja M, Mirjalili S (2018) Whale optimization approaches for wrapper feature selection. Appl Soft Comput 62:441–453. https://doi.org/10.1016/j.asoc.2017.11.006
Liu H, Zhou M, Liu Q (2019) An embedded feature selection method for imbalanced data classification. IEEE/CAA J Automatica Sinica 6(3):703–715. https://doi.org/10.1109/JAS.2019.1911447
Han Y, Zhao S, Deng H et al (2023) Principal graph embedding convolutional recurrent network for traffic flow prediction. Appl Intell 1–15. https://doi.org/10.1007/s10489-022-04211-x
Zhong Y, Chalise P, He J (2023) Nested cross-validation with ensemble feature selection and classification model for high-dimensional biological data. Commun Stat-simulation Comput 52(1):110–125. https://doi.org/10.1080/03610918.2020.1850790
Zhu Y, Li T, Lan X (2022) Feature selection optimized by the artificial immune algorithm based on genome shuffling and conditional lethal mutation. Appl Intell 1–21. https://doi.org/10.1007/s10489-022-03971-w
Li J, Cheng K, Wang S et al (2017) Feature selection: A data perspective. ACM Comput Surv (CSUR) 50(6):1–45. https://doi.org/10.1145/3136625
Hadri A, Chougdali K, Touahni R (2016) Intrusion detection system using pca and fuzzy pca techniques. In: 2016 International conference on advanced communication systems and information security (ACOSIS), pp 1–7, https://doi.org/10.1109/ACOSIS.2016.7843930
Ambusaidi MA, He X, Nanda P et al (2016) Building an intrusion detection system using a filter-based feature selection algorithm. IEEE Trans Comput 65(10):2986–2998. https://doi.org/10.1109/TC.2016.2519914
Benaddi H, Ibrahimi K, Benslimane A (2018) Improving the intrusion detection system for nsl-kdd dataset based on pca-fuzzy clustering-knn. In: 2018 6th International conference on wireless networks and mobile communications (WINCOM), pp 1–6, https://doi.org/10.1109/WINCOM.2018.8629718
Nazir A, Khan RA (2021) A novel combinatorial optimization based feature selection method for network intrusion detection. Comput Sec 102:102,164. https://www.sciencedirect.com/science/article/pii/S0167404820304375
Popoola E, Adewumi AO (2017) Efficient feature selection technique for network intrusion detection system using discrete differential evolution and decision. Int J Netw Secur 19(5):660–669. https://doi.org/10.6633/IJNS.201709.19(5).02
Al-Yaseen WL, Idrees AK, Almasoudy FH (2022) Wrapper feature selection method based differential evolution and extreme learning machine for intrusion detection system. Pattern Recognit 132(108):912. https://doi.org/10.1016/j.patcog.2022.108912
Zhang C, Soda P, Bi J et al (2023) An empirical study on the joint impact of feature selection and data resampling on imbalance classification. Appl Intell 53(5):5449–5461. https://doi.org/10.1007/s10489-022-03772-1
Han F, Wang T, Ling Q (2023) An improved feature selection method based on angle-guided multi-objective pso and feature-label mutual information. Appl Intell 53(3):3545–3562. https://doi.org/10.1007/s10489-022-03465-9
Singh D, Singh B (2019) Hybridization of feature selection and feature weighting for high dimensional data. Appl Intell 49:1580–1596. https://doi.org/10.1007/s10489-018-1348-2
Liu J, Zio E (2019) Integration of feature vector selection and support vector machine for classification of imbalanced data. Appl Soft Comput 75:702–711. https://doi.org/10.1016/j.asoc.2018.11.045
Alelyani S, Tang J, Liu H (2018) Feature selection for clustering: A review. Data Clustering 29–60. https://doi.org/10.1016/j.neucom.2017.11.077
Sheikhpour R, Sarram MA, Gharaghani S et al (2017) A survey on semi-supervised feature selection methods. Pattern Recognit 64:141–158. https://doi.org/10.1016/j.patcog.2016.11.003
Shi S, Xiong H (2022) A hybrid immune genetic algorithm with tabu search for minimizing the tool switch times in cnc milling batch-processing. Appl Intell 1–15. https://doi.org/10.1007/s10489-021-02869-3
Lu T, Zhang L, Fu Y (2018) A novel immune-inspired shellcode detection algorithm based on hyperellipsoid detectors. Sec Commun Netw 2018. https://doi.org/10.1155/2018/2063089
Fan Z, Wen C, Tao L, et al (2019) An antigen space triangulation coverage based real-value negative selection algorithm. IEEE Access 7:51,886–51,898. https://doi.org/10.1109/ACCESS.2019.2911660
Ren Y, Wang X, Zhang C (2021) A novel fault diagnosis method based on improved negative selection algorithm. IEEE Trans Instrument Measure 70:1–8. https://doi.org/10.1109/TIM.2020.3031166
Kim YJ, Nam W, Lee J (2022) Multiclass anomaly detection for unsupervised and semi-supervised data based on a combination of negative selection and clonal selection algorithms. Appl Soft Comput 122(108):838. https://doi.org/10.1016/j.asoc.2022.108838
Zhu Y, Li W, Li T (2023) A hybrid artificial immune optimization for high-dimensional feature selection. Knowl-Based Syst 260(110):111. https://doi.org/10.1016/j.knosys.2022.110111
Forrest S, Perelson AS, Allen L et al. (1994) Self-nonself discrimination in a computer. In: Proceedings of 1994 IEEE computer society symposium on research in security and privacy, Ieee, pp 202–212, https://doi.org/10.1109/RISP.1994.296580
Ho TK (1995) Random decision forests. In: Proceedings of 3rd international conference on document analysis and recognition, IEEE, pp 278–282, https://doi.org/10.1109/ICDAR.1995.598994
Chen T, Guestrin C (2016) Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp 785–794, https://doi.org/10.1145/2939672.2939785
Zhang T, Ramakrishnan R, Livny M (1997) Birch: A new data clustering algorithm and its applications. Data Mining Knowl Discov 1:141–182. https://doi.org/10.1023/A:1009783824328
Gonzalez F, Dasgupta D, Niño LF (2003) A randomized real-valued negative selection algorithm. In: Artificial immune systems: second international conference, ICARIS 2003, Edinburgh, UK, September 1-3, 2003. Proceedings 2, Springer, pp 261–272, https://doi.org/10.1007/978-3-540-45192-1_25
Ji Z, Dasgupta D (2004) Real-valued negative selection algorithm with variable-sized detectors. In: Genetic and evolutionary computation conference, Springer, pp 287–298, https://doi.org/10.1007/978-3-540-24854-5_30
Wen C, Xiaoming D, Tao L et al (2014) Negative selection algorithm based on grid file of the feature space. Knowl-Based Syst 56:26–35. https://doi.org/10.1016/j.knosys.2013.10.018
Stibor T, Mohr P, Timmis J et al. (2005) Is negative selection appropriate for anomaly detection. Association for Computing Machinery, New York, NY, USA, GECCO ’05, p 321-328, https://doi.org/10.1145/1068009.1068061
Tavallaee M, Bagheri E, Lu W et al. (2009) A detailed analysis of the kdd cup 99 data set. In: 2009 IEEE Symposium on computational intelligence for security and defense applications, pp 1–6, https://doi.org/10.1109/CISDA.2009.5356528
Moustafa N (2017) Designing an online and reliable statistical anomaly detection framework for dealing with large high-speed network traffic. PhD thesis, UNSW Sydney, https://doi.org/10.26190/unsworks/3298
Sharafaldin I, Lashkari AH, Ghorbani AA (2018) Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp 1:108–116. https://doi.org/10.5220/0006639801080116
Acknowledgements
This work was supported by the National Key Research and Development Program of China (2020YFB1805400), the National Natural Science Foundation of China (61876134).
Author information
Authors and Affiliations
Contributions
Xiaowen Liu: Methodology, Writing – original draft, Writing – review & editing. Geying Yang: Conceptualization, Writing – review & editing. Lina Wang: Conceptualization, Supervision, Writing – review. Jie Fu: Software, Data curation, Validation. Qinghao Wang: Investigation, Writing – review.
Corresponding author
Ethics declarations
Conflict of Interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Ethical and informed consent for data used
All datasets used in this paper are public datasets, which can be downloaded through public channels upon request.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Liu, X., Yang, G., Wang, L. et al. A novel immune detector training method for network anomaly detection. Appl Intell 54, 2009–2030 (2024). https://doi.org/10.1007/s10489-024-05288-2
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-024-05288-2