Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

A comprehensive survey on network anomaly detection

  • Published:
Telecommunication Systems Aims and scope Submit manuscript

Abstract

Nowadays, there is a huge and growing concern about security in information and communication technology among the scientific community because any attack or anomaly in the network can greatly affect many domains such as national security, private data storage, social welfare, economic issues, and so on. Therefore, the anomaly detection domain is a broad research area, and many different techniques and approaches for this purpose have emerged through the years. In this study, the main objective is to review the most important aspects pertaining to anomaly detection, covering an overview of a background analysis as well as a core study on the most relevant techniques, methods, and systems within the area. Therefore, in order to ease the understanding of this survey’s structure, the anomaly detection domain was reviewed under five dimensions: (1) network traffic anomalies, (2) network data types, (3) intrusion detection systems categories, (4) detection methods and systems, and (5) open issues. The paper concludes with an open issues summary discussing presently unsolved problems, and final remarks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Hashim, F., Munasinghe, K. S., & Jamalipour, A. (2010). Biologically inspired anomaly detection and security control frameworks for complex heterogeneous networks. IEEE Transactions on Network and Service Management, 7, 268–281. https://doi.org/10.1109/TNSM.2010.1012.0360.

    Article  Google Scholar 

  2. Xiao, X., Zhang, S., Mercaldo, F., Hu, G., & Sangaiah, A. K. (2017). Android malware detection based on system call sequences and LSTM. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-017-5104-0.

  3. Balakrishnan, S. M., & Sangaiah, A. K. (2017). MIFIM—Middleware solution for service centric anomaly in future internet models. Future Generation Computer Systems, 74, 349–365. https://doi.org/10.1016/j.future.2016.08.006.

    Article  Google Scholar 

  4. Carvalho, L. F., Abrão, T., Mendes, L. S., & Proença, M. L. (2018). An ecosystem for anomaly detection and mitigation in software-defined networking. Expert Systems with Applications, 104, 121–133. https://doi.org/10.1016/J.ESWA.2018.03.027.

    Article  Google Scholar 

  5. Lu, S., Wang, X., & Mao, L. (2014). Network security situation awareness based on network simulation. In 2014 IEEE workshop on electronics, computer and applications (pp. 512–517). https://doi.org/10.1109/IWECA.2014.6845671.

  6. Hosseini Bamakan, S. M., Wang, H., & Shi, Y. (2017). Ramp loss K-support vector classification-regression: A robust and sparse multi-class approach to the intrusion detection problem. Knowledge-Based Systems, 126, 113–126. https://doi.org/10.1016/j.knosys.2017.03.012.

    Article  Google Scholar 

  7. Lof, A., & Nelson, R. (2014). Annotating network trace data for anomaly detection research. In 2014 IEEE 39th conference on local computer networks workshops (LCN workshops) (pp. 679–684). https://doi.org/10.1109/LCNW.2014.6927720.

  8. Barnett, V., & Lewis, T. (1994). Outliers in statistical data (3rd ed.). New York: Wiley.

    Google Scholar 

  9. Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys, 41, 1–58. https://doi.org/10.1145/1541880.1541882.

    Article  Google Scholar 

  10. Lakhina, A., Crovella, M., & Diot, C. (2004). Diagnosing network-wide traffic anomalies. In ACM SIGCOMM computer communication review (Vol. 34, p. 219). https://doi.org/10.1145/1030194.1015492.

  11. Hoque, N., Bhuyan, M. H., Baishya, R. C., Bhattacharyya, D. K., & Kalita, J. K. (2014). Network attacks: Taxonomy, tools and systems. Journal of Network and Computer Applications, 40, 307–324. https://doi.org/10.1016/j.jnca.2013.08.001.

    Article  Google Scholar 

  12. Thottan, M., Liu, G., & Ji, C. (2010). Anomaly detection approaches for communication networks. In G. Cormode & M. Thottan (Eds.), Algorithms for next generation networks (pp. 239–261). London: Springer. https://doi.org/10.1007/978-1-84882-765-3_11.

    Chapter  Google Scholar 

  13. Patcha, A., & Park, J.-M. (2007). An overview of anomaly detection techniques: Existing solutions and latest technological trends. Computer Networks, 51, 3448–3470. https://doi.org/10.1016/j.comnet.2007.02.001.

    Article  Google Scholar 

  14. Yu, Y. (2012). A survey of anomaly intrusion detection techniques. Journal of Computing Sciences in Colleges, 28, 9–17.

    Google Scholar 

  15. Weiyu, Z., Qingbo, Y., & Yushui, G. (2009). A survey of anomaly detection methods in networks. In International symposium on computer network and multimedia technology, 2009. CNMT 2009 (pp. 1–3). https://doi.org/10.1109/CNMT.2009.5374676.

  16. Marnerides, A. K., Schaeffer-Filho, A., & Mauthe, A. (2014). Traffic anomaly diagnosis in Internet backbone networks: A survey. Computer Networks, 73, 224–243. https://doi.org/10.1016/j.comnet.2014.08.007.

    Article  Google Scholar 

  17. Bhuyan, M. H., Bhattacharyya, D. K., & Kalita, J. K. (2014). Network anomaly detection: Methods, systems and tools. IEEE Communications Surveys & Tutorials, 16, 303–336. https://doi.org/10.1109/SURV.2013.052213.00046.

    Article  Google Scholar 

  18. Ahmed, M., Naser Mahmood, A., & Hu, J. (2016). A survey of network anomaly detection techniques. Journal of Network and Computer Applications, 60, 19–31. https://doi.org/10.1016/j.jnca.2015.11.016.

    Article  Google Scholar 

  19. Xiuyao, S., Mingxi, W., Jermaine, C., & Ranka, S. (2007). Conditional anomaly detection. IEEE Transactions on Knowledge and Data Engineering, 19, 631–644. https://doi.org/10.1109/TKDE.2007.1009.

    Article  Google Scholar 

  20. Barford, P., Kline, J., Plonka, D., & Ron, A. (2002). A signal analysis of network traffic anomalies. In Proceedings of the 2nd ACM SIGCOMM workshop on internet measurment—IMW ’02. ACM Press, New York, NY, USA (p. 71). https://doi.org/10.1145/637201.637210.

  21. Barford, P., & Plonka, D. (2001). Characteristics of network traffic flow anomalies. Proceedings of the 1st ACM SIGCOMM workshop on internet measurement (pp. 69–73). https://doi.org/10.1145/505202.505211.

  22. Jung, J., Krishnamurthy, B., & Rabinovich, M. (2002). Flash crowds and denial of service attacks. In Proceedings of the 11th international conference on World Wide Web—WWW ’02 (p. 293). https://doi.org/10.1145/511446.511485.

  23. Pan, J., Hu, H., & Liu, Y. (2014). Human behavior during Flash Crowd in web surfing. Physica A: Statistical Mechanics and Its Applications, 413, 212–219. https://doi.org/10.1016/j.physa.2014.06.085.

    Article  Google Scholar 

  24. Ghorbani, A. A., Lu, W., & Tavallaee, M. (2010). Network attacks. Advances in Information Security, 47, 1–25. https://doi.org/10.1007/978-0-387-88771-5_1.

    Google Scholar 

  25. Mouton, F., Malan, M. M., & Venter, H. S. (2013). Social engineering from a normative ethics perspective. In Information security for South Africa, 2013 (pp. 1–8). https://doi.org/10.1109/ISSA.2013.6641064.

  26. Maxion, R. A., & Townsend, T. N. (2002). Masquerade detection using truncated command lines. In International conference on dependable systems and networks, 2002. DSN 2002. Proceedings (pp. 219–228). https://doi.org/10.1109/DSN.2002.1028903.

  27. Szor, P. (2005). The art of computer virus research and defense. Reading: Addison-Wesley.

    Google Scholar 

  28. Weaver, N., Paxson, V., Staniford, S., & Cunningham, R., (2003). A taxonomy of computer worms. In Proceedings of the 2003 ACM workshop on Rapid malcode (pp. 11–18). https://doi.org/10.1145/948187.948190.

  29. Peng, T., Leckie, C., & Ramamohanarao, K. (2007). Survey of network-based defense mechanisms countering the DoS and DDoS problems. ACM Computing Surveys, 39, 3. https://doi.org/10.1145/1216370.1216373.

    Article  Google Scholar 

  30. Mohana Priya, P., Akilandeswari, V., Mercy Shalinie, S., Lavanya, V., & Shanmuga Priya, M. (2014). The protocol independent detection and classification (PIDC) system for DRDoS attack. In 2014 International conference on recent trends in information technology (ICRTIT) (pp. 1–7). IEEE. https://doi.org/10.1109/ICRTIT.2014.6996154.

  31. Muller, T., & Freiling, F. C. (2014). A systematic assessment of the security of full disk encryption. IEEE Transactions on Dependable and Secure Computing. https://doi.org/10.1109/TDSC.2014.2369041.

  32. Raza, M., & Haider, W. (2012). A survey of password attacks and comparative analysis on methods for secure authentication. World Applied Sciences Journal, 19, 439–444. https://doi.org/10.5829/idosi.wasj.2012.19.04.1837.

    Google Scholar 

  33. Shimonski, R., Zenir, J., & Bishop, A. (2015). Chapter 2: Information gathering. In R. S. Z. Bishop (Ed.), Cyber Reconnaissance, surveillance and defense (pp. 45–84). Boston: Syngress. https://doi.org/10.1016/B978-0-12-801308-3.00002-0.

    Chapter  Google Scholar 

  34. Harrington, D., Presuhn, R., & Wijnen, B. (2002). RFC 3411: An architecture for describing simple network management protocol (SNMP) management frameworks (pp. 1–64). https://tools.ietf.org/html/rfc3411. Accessed 23 Oct 2017.

  35. Thottan, M., & Ji, C. (2003). Anomaly detection in IP networks. IEEE Transactions on Signal Processing, 51, 2191–2204. https://doi.org/10.1109/TSP.2003.814797.

    Article  Google Scholar 

  36. Cabrera, J. B. D., Lewis, L., Qin, X., Lee, W., Prasanth, R. K., Ravichandran, B., & Mehra, R. K. (2001). Proactive detection of distributed denial of service attacks using MIB traffic variables: A feasibility study. In 2001 IEEE/IFIP International symposium on integrated network management proceedings. VII. Integr. Manag. Strateg. New Millenn. (Cat. No. 01EX470) (pp. 609–622). IEEE. https://doi.org/10.1109/INM.2001.918069.

  37. Yu, J., Lee, H., Kim, M.-S., & Park, D. (2008). Traffic flooding attack detection with SNMP MIB using SVM. Computer Communications, 31, 4212–4219. https://doi.org/10.1016/j.comcom.2008.09.018.

    Article  Google Scholar 

  38. Lima, M. F., Sampaio, L. D. H., Zarpelao, B. B., Rodrigues, J. J. P. C., Abrao, T., & Proenca, M. L., Jr. (2010). Networking anomaly detection using DSNs and particle swarm optimization with re-clustering. In 2010 IEEE global telecommunications conference GLOBECOM 2010 (pp. 1–6). IEEE. https://doi.org/10.1109/GLOCOM.2010.5683910.

  39. Zarpelao, B. B., Mendes, L. S., Proenca Jr., M. L., & Rodrigues, J. J. P. C. (2009). Parameterized anomaly detection system with automatic configuration. In GLOBECOM 2009—2009 IEEE global telecommunications conference (pp. 1–6). IEEE. https://doi.org/10.1109/GLOCOM.2009.5426189.

  40. Duffield, N., Haffner, P., Krishnamurthy, B., & Ringberg, H. (2009). Rule-based anomaly detection on IP flows. In IEEE INFOCOM 2009—28th Conference on Computer Communications (pp. 424–432). IEEE. https://doi.org/10.1109/INFCOM.2009.5061947.

  41. Fontugne, R., & Fukuda, K. (2011). A Hough-transform-based anomaly detector with an adaptive time interval. ACM SIGAPP Applied Computing Review, 11, 41–51. https://doi.org/10.1145/2034594.2034598.

    Article  Google Scholar 

  42. Introduction to Cisco IOS®NetFlow (White Paper), (2012) 1–16. http://www.cisco.com/c/en/us/products/collateral/ios-nx-os software/iosnetflow/prod_white_paper0900aecd80406232.pdf. Accessed 10 Dec 2017.

  43. Claise, B. (2004). RFC 3954: Cisco systems netflow services export version 9 (pp. 1–33). https://tools.ietf.org/html/rfc3954. Accessed September 2, 2016.

  44. Trammell, B., & Claise, B. (2013). RFC 7011: Specification of the IP flow information export (IPFIX) protocol for the exchange of flow information, 1–53. 2070-1721.

  45. Chapman, C. (2016). Chapter 10: Traffic performance testing in the network. In Network performance and security (pp. 295–317). https://doi.org/10.1016/B978-0-12-803584-9.00010-X.

  46. NfSen: NetFlow sensor. (2011). http://nfsen.sourceforge.net/. Accessed September 2, 2016.

  47. nTop. (2016). http://www.ntop.org/. Accessed September 2, 2016.

  48. Panchen, S., Phaal, P., & McKee, N. (2001). RFC 3176: InMon Corporation’s sFlow: A method for monitoring traffic in switched and routed networks, 1–31. https://tools.ietf.org/html/rfc3176. Accessed September 2, 2016.

  49. Duffield, N. (2004). Sampling for passive internet measurement: A review. Statistical Science, 19, 472–498. https://doi.org/10.1214/088342304000000206.

    Article  Google Scholar 

  50. Cisco NetFlow-Lite Solution Overview, Cisco. (2016). http://www.cisco.com/c/en/us/products/collateral/switches/catalyst-2960-x-series-switches/solution_overview_c22-728776.html. Accessed September 2, 2016.

  51. Deri, L., Chou, E., Cherian, Z., Karmarkar, K., & Patterson, M. (2011). Increasing data center network visibility with cisco NetFlow-Lite. In International conference on network and service management (pp. 1–6).

  52. Jadidi, Z., Muthukkumarasamy, V., Sithirasenan, E., & Singh, K. (2015). Flow-based anomaly detection in big data. In Network big data (pp. 257–279). Chapman and Hall/CRC. https://doi.org/10.1201/b18772-17.

  53. Sperotto, A., Schaffrath, G., Sadre, R., Morariu, C., Pras, A., & Stiller, B. (2010). An overview of IP flow-based intrusion detection. IEEE Communications Surveys and Tutorials, 12, 343–356. https://doi.org/10.1109/SURV.2010.032210.00054.

    Article  Google Scholar 

  54. Winter, P., Hermann, E., & Zeilinger, M. (2011). Inductive intrusion detection in flow-based network data using one-class support vector machines. In 2011 4th IFIP international conference on new technologies, mobility and security (pp. 1–5). IEEE. https://doi.org/10.1109/NTMS.2011.5720582.

  55. Bartos, K., Rehak, M., & Krmicek, V. (2011). Optimizing flow sampling for network anomaly detection. In 2011 7th international wireless communications and mobile computing conference (pp. 1304–1309). IEEE. https://doi.org/10.1109/IWCMC.2011.5982728.

  56. Zhang, Y., Fang, B., & Luo, H. (2010). Identifying high-rate flows based on sequential sampling. IEICE Transactions on Information and Systems, E93–D, 1162–1174. https://doi.org/10.1587/transinf.E93.D.1162.

    Article  Google Scholar 

  57. Silva, J. M. C., Carvalho, P., & Lima, S. R. (2015). Analysing traffic flows through sampling: A comparative study. In 2015 IEEE symposium on computers and communications (pp. 341–346). https://doi.org/10.1109/ISCC.2015.7405538.

  58. Kemmerer, R. A., & Vigna, G. (2002). Intrusion detection: A brief history and overview. Computer, 35, 27–30. https://doi.org/10.1109/MC.2002.1012428.

    Article  Google Scholar 

  59. Lee, W., & Stolfo, S. (1998). Data mining approaches for intrusion detection. In Proceedings of 7th {USENIX} security symposium, USENIX Association (pp. 6–6). https://dl.acm.org/citation.cfm?id=1267555. Accessed November 1, 2017.

  60. Bul’ajoul, W., James, A., & Pannu, M. (2015). Improving network intrusion detection system performance through quality of service configuration and parallel technology. Journal of Computer and System Sciences, 81, 981–999. https://doi.org/10.1016/j.jcss.2014.12.012.

    Article  Google Scholar 

  61. Bostani, H., & Sheikhan, M. (2017). Hybrid of anomaly-based and specification-based IDS for Internet of Things using unsupervised OPF based on MapReduce approach. Computer Communications, 98, 52–71. https://doi.org/10.1016/j.comcom.2016.12.001.

    Article  Google Scholar 

  62. Milenkoski, A., Vieira, M., Kounev, S., Avritzer, A., & Payne, B. D. (2015). Evaluating computer intrusion detection systems: A survey of common practices. ACM Computing Surveys, 48, 1–41. https://doi.org/10.1145/2808691.

    Article  Google Scholar 

  63. Butun, I., Morgera, S. D., & Sankar, R. (2014). A survey of intrusion detection systems in wireless sensor networks. IEEE Communications Surveys & Tutorials, 16, 266–282. https://doi.org/10.1109/SURV.2013.050113.00191.

    Article  Google Scholar 

  64. Debar, H., Dacier, M., & Wespi, A. (1999). Towards a taxonomy of intrusion–detection systems. Computer Networks, 31, 805–822. https://doi.org/10.1016/S1389-1286(98)00017-6.

    Article  Google Scholar 

  65. Meng Hui, L., & Jones, A. (2008). Network anomaly detection system: The state of art of network behaviour analysis. In International conference on convergence and hybrid information technology 2008. ICHIT ’08 (pp. 459–465). https://doi.org/10.1109/ICHIT.2008.249.

  66. Sobh, T. S. (2006). Wired and wireless intrusion detection system: Classifications, good characteristics and state-of-the-art. Computer Standards & Interfaces, 28, 670–694. https://doi.org/10.1016/j.csi.2005.07.002.

    Article  Google Scholar 

  67. de Assis, M. V. O., Rodrigues, J. J. P. C., & Proença, M. L. (2014). A seven-dimensional flow analysis to help autonomous network management. Information Sciences, 278, 900–913. https://doi.org/10.1016/j.ins.2014.03.102.

    Article  Google Scholar 

  68. Stakhanova, N., Basu, S., & Wong, J. (2010). On the symbiosis of specification-based and anomaly-based detection. Computers & Security, 29, 253–268. https://doi.org/10.1016/j.cose.2009.08.007.

    Article  Google Scholar 

  69. Lakhina, A., Papagiannaki, K., Crovella, M., Diot, C., Kolaczyk, E. D., & Taft, N. (2004). Structural analysis of network traffic flows. ACM SIGMETRICS Performance Evaluation Review, 32, 61. https://doi.org/10.1145/1012888.1005697.

    Article  Google Scholar 

  70. Lakhina, A., Crovella, M., & Diot, C. (2005). Mining anomalies using traffic feature distributions. ACM SIGCOMM Computer Communication Review, 35, 217. https://doi.org/10.1145/1090191.1080118.

    Article  Google Scholar 

  71. Callegari, C., Giordano, S., Pagano, M., & Pepe, T. (2011). Combining sketches and wavelet analysis for multi time-scale network anomaly detection. Computers & Security, 30, 692–704. https://doi.org/10.1016/j.cose.2011.08.006.

    Article  Google Scholar 

  72. Hamdi, M., & Boudriga, N. (2007). Detecting Denial-of-Service attacks using the wavelet transform. Computer Communications, 30, 3203–3213. https://doi.org/10.1016/j.comcom.2007.05.061.

    Article  Google Scholar 

  73. Jolliffe, I. T. (2002). Principal component analysis. Berlin: Springer.

    Google Scholar 

  74. Jackson, J. E. (2005). A user’s guide to principal components. New York: Wiley.

    Google Scholar 

  75. Ringberg, H., Soule, A., Rexford, J., & Diot, C. (2007). Sensitivity of PCA for traffic anomaly detection. SIGMETRICS Performance Evaluation Review, 35, 109–120. https://doi.org/10.1145/1269899.1254895.

    Article  Google Scholar 

  76. Wright, J., Ganesh, A., Rao, S., Peng, Y., & Ma, Y. (2009). Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization. In Y. Bengio, D. Schuurmans, J. D. Lafferty, C. K. I. Williams, & A. Culotta (Eds.), Advances in neural information processing systems (Vol. 22, pp. 2080–2088). Curran Associates, Inc. http://papers.nips.cc/paper/3704-robust-principal-component-analysis-exact-recovery-of-corrupted-low-rank-matrices-via-convex-optimization.pdf.

  77. Candès, E. J., Li, X., Ma, Y., & Wright, J. (2011). Robust principal component analysis? Journal of the ACM, 58, 11:1–11:37. https://doi.org/10.1145/1970392.1970395.

    Article  Google Scholar 

  78. Pascoal, C., Rosario de Oliveira, M., Valadas, R., Filzmoser, P., Salvador, P., & Pacheco, A. (2012). Robust feature selection and robust PCA for internet traffic anomaly detection. In INFOCOM, 2012 Proceedings of IEEE (pp. 1755–1763). https://doi.org/10.1109/INFCOM.2012.6195548.

  79. Kanda, Y., Fontugne, R., Fukuda, K., & Sugawara, T. (2013). ADMIRE: Anomaly detection method using entropy-based PCA with three-step sketches. Computer Communications, 36, 575–588. https://doi.org/10.1016/j.comcom.2012.12.002.

    Article  Google Scholar 

  80. OReilly, C., Gluhak, A., & Imran, M. A. (2016). Distributed anomaly detection using minimum volume elliptical principal component analysis. IEEE Transactions on Knowledge and Data Engineering, 28, 2320–2333. https://doi.org/10.1109/TKDE.2016.2555804.

    Article  Google Scholar 

  81. Camacho, J., Pérez-Villegas, A., García-Teodoro, P., & Maciá-Fernández, G. (2016). PCA-based multivariate statistical network monitoring for anomaly detection. Computers & Security, 59, 118–137. https://doi.org/10.1016/j.cose.2016.02.008.

    Article  Google Scholar 

  82. Fernandes, G., Carvalho, L. F., Rodrigues, J. J. P. C., & Proença, M. L. (2016). Network anomaly detection using IP flows with principal component analysis and ant colony optimization. Journal of Network and Computer Applications, 64, 1–11. https://doi.org/10.1016/j.jnca.2015.11.024.

    Article  Google Scholar 

  83. Fernandes, G., Rodrigues, J. J. P. C., & Proença, M. L. (2015). Autonomous profile-based anomaly detection system using principal component analysis and flow analysis. Applied Soft Computing, 34, 513–525. https://doi.org/10.1016/j.asoc.2015.05.019.

    Article  Google Scholar 

  84. Fernandes, G., Zacaron, A. M., Rodrigues, J. J. P. C., & Proenca, M. L. (2013). Digital signature to help network management using principal component analysis and K-means clustering. In 2013 IEEE international conference on communications (pp. 2519–2523). IEEE. https://doi.org/10.1109/ICC.2013.6654912.

  85. Yeung, D. S., Shuyuan, J., & Xizhao, W. (2007). Covariance-matrix modeling and detecting various flooding attacks. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 37, 157–169. https://doi.org/10.1109/TSMCA.2006.889480.

    Article  Google Scholar 

  86. Xie, M., Hu, J., & Guo, S. (2015). Segment-based anomaly detection with approximated sample covariance matrix in wireless sensor networks. IEEE Transactions on Parallel and Distributed Systems, 26, 574–583. https://doi.org/10.1109/TPDS.2014.2308198.

    Article  Google Scholar 

  87. Huang, T., Sethu, H., & Kandasamy, N. (2016). A new approach to dimensionality reduction for anomaly detection in data traffic. IEEE Transactions on Network and Service Management, 13, 651–665. https://doi.org/10.1109/TNSM.2016.2597125.

    Article  Google Scholar 

  88. Kalkan, K., & Alagöz, F. (2016). A distributed filtering mechanism against DDoS attacks: ScoreForCore. Computer Networks, 108, 199–209. https://doi.org/10.1016/j.comnet.2016.08.023.

    Article  Google Scholar 

  89. Ozkan, H., Ozkan, F., & Kozat, S. S. (2016). Online anomaly detection under Markov statistics with controllable type-I error. IEEE Transactions on Signal Processing, 64, 1435–1445. https://doi.org/10.1109/TSP.2015.2504345.

    Article  Google Scholar 

  90. Proença, M. L., Coppelmans, C., Bottoli, M., Alberti, A., & Mendes, L. S. (2004). The hurst parameter for digital signature of network segment. In J. N. de Souza, P. Dini, & P. Lorenz (Eds.), Telecommunications and networking—ICT 2004 11th international conference on telecommunications, Fortaleza, Brazil, August 1–6, 2004. Proceedings (pp. 772–781). Berlin: Springer. https://doi.org/10.1007/978-3-540-27824-5_103.

  91. Pena, E. H. M., Carvalho, L. F., Barbon, S, Jr., Rodrigues, J. J. P. C., & Proença, M. L, Jr. (2017). Anomaly detection using the correlational paraconsistent machine with digital signatures of network segment. Information Sciences, 420, 313–328. https://doi.org/10.1016/j.ins.2017.08.074.

    Article  Google Scholar 

  92. Pena, E. H. M., Carvalho, L. F., Barbon, S., Rodrigues, J. J. P. C., & Proenca, M. L. (2014). Correlational paraconsistent machine for anomaly detection. In 2014 IEEE global communications conference (pp. 551–556). IEEE. https://doi.org/10.1109/GLOCOM.2014.7036865.

  93. Bang, J., Cho, Y.-J., & Kang, K. (2017). Anomaly detection of network-initiated LTE signaling traffic in wireless sensor and actuator networks based on a Hidden semi-Markov Model. Computers & Security, 65, 108–120. https://doi.org/10.1016/j.cose.2016.11.008.

    Article  Google Scholar 

  94. Ren, H., Ye, Z., & Li, Z. (2017). Anomaly detection based on a dynamic Markov model. Computers & Security. https://doi.org/10.1016/j.ins.2017.05.021.

  95. Jazi, H. H., Gonzalez, H., Stakhanova, N., & Ghorbani, A. A. (2017). Detecting HTTP-based application layer DoS attacks on web servers in the presence of sampling. Computer Networks, 121, 25–36. https://doi.org/10.1016/j.comnet.2017.03.018.

    Article  Google Scholar 

  96. Han, J., Kamber, M., & Pei, J. (2012). 10: Cluster analysis: Basic concepts and methods. In J. H. Kamber, & J. Pei (Eds.), Data mininig (3d edn., pp. 443–495). Boston: Morgan Kaufmann. https://doi.org/10.1016/B978-0-12-381479-1.00010-1.

  97. Rajasegarar, S., Leckie, C., & Palaniswami, M. (2014). Hyperspherical cluster based distributed anomaly detection in wireless sensor networks. Journal of Parallel and Distributed Computing, 74, 1833–1847. https://doi.org/10.1016/j.jpdc.2013.09.005.

    Article  Google Scholar 

  98. Mazel, J., Casas, P., Labit, Y., & Owezarski, P. (2011). Sub-space clustering, inter-clustering results association and anomaly correlation for unsupervised network anomaly detection. In CNSM ’11 Proceedings of the 7th international conference on network and services management (pp. 73–80). http://dl.acm.org/citation.cfm?id=2147683.

  99. Karami, A., & Guerrero-Zapata, M. A. (2015). Fuzzy anomaly detection system based on hybrid PSO-K means algorithm in content-centric networks. Neurocomputing, 149, 1253–1269. https://doi.org/10.1016/j.neucom.2014.08.070.

    Article  Google Scholar 

  100. Carvalho, L. F., Barbon, S., Mendes, L. S., & Proença, M. L. (2016). Unsupervised learning clustering and self-organized agents applied to help network management. Expert Systems with Applications, 54, 29–47. https://doi.org/10.1016/j.eswa.2016.01.032.

    Article  Google Scholar 

  101. Dromard, J., Roudiere, G., & Owezarski, P. (2017). Online and scalable unsupervised network anomaly detection method. IEEE Transactions on Network and Service Management, 14, 34–47. https://doi.org/10.1109/TNSM.2016.2627340.

    Article  Google Scholar 

  102. He, D., Chan, S., Ni, X., & Guizani, M. (2017). Software-defined-networking-enabled traffic anomaly detection and mitigation. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2017.2694702.

  103. Bigdeli, E., Mohammadi, M., Raahemi, B., & Matwin, S. (2018). Incremental anomaly detection using two-layer cluster-based structure. Information Sciences, 429, 315–331. https://doi.org/10.1016/j.ins.2017.11.023.

    Article  Google Scholar 

  104. Estevez-Tapiador, J. M., Garcia-Teodoro, P., & Diaz-Verdejo, J. E. (2003). Stochastic protocol modeling for anomaly based network intrusion detection. In Information assurance. 2003. IWIAS 2003. Proceedings. First IEEE international workshop on (pp. 3–12). https://doi.org/10.1109/IWIAS.2003.1192454.

  105. Su, M.-Y. (2010). Discovery and prevention of attack episodes by frequent episodes mining and finite state machines. Journal of Network and Computer Applications, 33, 156–167. https://doi.org/10.1016/j.jnca.2009.10.003.

    Article  Google Scholar 

  106. Hammerschmidt, C., Marchal, S., State, R., Pellegrino, G., & Verwer, S., (2016). Efficient learning of communication profiles from IP flow records. In 2016 IEEE 41st conference on local computer networks (pp. 559–562). IEEE. https://doi.org/10.1109/LCN.2016.92.

  107. Duda, R. O., Hart, P. E., & Stork, D. G. (2012). Pattern classification. New York: Wiley.

    Google Scholar 

  108. Klassen, M., & Ning, Y. (2012). Anomaly based intrusion detection in wireless networks using Bayesian classifier. In 2012 IEEE fifth international conference on advanced computational intelligence (ICACI) (pp. 257–264). https://doi.org/10.1109/ICACI.2012.6463163.

  109. Tao, L., Ailing, Q., Yuanbin, H., & Xintan, C. (2008). Method for network anomaly detection based on Bayesian statistical model with time slicing. In 7th world congress on intelligent control and automation, 2008. WCICA 2008 (pp. 3359–3362). https://doi.org/10.1109/WCICA.2008.4593458.

  110. Swarnkar, M., & Hubballi, N. (2016). OCPAD: One class Naive Bayes classifier for payload based anomaly detection. Expert Systems with Applications, 64, 330–339. https://doi.org/10.1016/j.eswa.2016.07.036.

    Article  Google Scholar 

  111. Vapnik, V. N. (1995). The nature of statistical learning theory. New York: Springer.

    Book  Google Scholar 

  112. Catania, C. A., Bromberg, F., & Garino, C. G. (2012). An autonomous labeling approach to support vector machines algorithms for network traffic anomaly detection. Expert Systems with Applications, 39, 1822–1829. https://doi.org/10.1016/j.eswa.2011.08.068.

    Article  Google Scholar 

  113. Amer, M., Goldstein, M., & Abdennadher, S. (2013). Enhancing one-class support vector machines for unsupervised anomaly detection. In Proceedings of the ACM SIGKDD workshop on outlier detection and description (pp. 8–15). https://doi.org/10.1145/2500853.2500857.

  114. Erfani, S. M., Rajasegarar, S., Karunasekera, S., & Leckie, C. (2016). High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning. Pattern Recognition, 58, 121–134. https://doi.org/10.1016/j.patcog.2016.03.028.

    Article  Google Scholar 

  115. Wang, H., Gu, J., & Wang, S. (2017). An effective intrusion detection framework based on SVM with feature augmentation. Knowledge-Based Systems, 136, 130–139. https://doi.org/10.1016/j.knosys.2017.09.014.

    Article  Google Scholar 

  116. Kabir, E., Hu, J., Wang, H., & Zhuo, G. (2017). A novel statistical technique for intrusion detection systems. Future Generation Computer Systems. https://doi.org/10.1016/j.future.2017.01.029.

  117. Subba, B., Biswas, S., & Karmakar, S. (2016). A neural network based system for intrusion detection and attack classification. In 2016 Twenty second national conference on communication (pp. 1–6). IEEE. https://doi.org/10.1109/NCC.2016.7561088.

  118. Saeed, A., Ahmadinia, A., Javed, A., & Larijani, H. (2016). Intelligent intrusion detection in low-power IoTs. ACM Transactions on Internet Technology, 16, 1–25. https://doi.org/10.1145/2990499.

    Article  Google Scholar 

  119. Brown, J., Anwar, M., & Dozier, G. (2016). An evolutionary general regression neural network classifier for intrusion detection. In 2016 25th International conference on computer communication and networks (ICCCN) (pp. 1–5). IEEE. https://doi.org/10.1109/ICCCN.2016.7568493.

  120. Ashfaq, R. A. R., Wang, X.-Z., Huang, J. Z., Abbas, H., & He, Y.-L. (2017). Fuzziness based semi-supervised learning approach for intrusion detection system. Information Sciences, 378, 484–497. https://doi.org/10.1016/j.ins.2016.04.019.

    Article  Google Scholar 

  121. Peddabachigari, S., Abraham, A., Grosan, C., & Thomas, J. (2007). Modeling intrusion detection system using hybrid intelligent systems. Journal of Network and Computer Applications, 30, 114–132. https://doi.org/10.1016/j.jnca.2005.06.003.

    Article  Google Scholar 

  122. Aburomman, A. A., & Bin Ibne Reaz, M. (2016). A novel SVM-kNN-PSO ensemble method for intrusion detection system. Applied Soft Computing, 38, 360–372. https://doi.org/10.1016/j.asoc.2015.10.011.

    Article  Google Scholar 

  123. Sornsuwit, P., & Jaiyen, S. (2015). Intrusion detection model based on ensemble learning for U2R and R2L attacks. In 2015 7th international conference on information technology and electrical engineering (ICITEE) (pp. 354–359). IEEE. https://doi.org/10.1109/ICITEED.2015.7408971.

  124. Bukhtoyarov, V., & Zhukov, V. (2014). Ensemble-distributed approach in classification problem solution for intrusion detection systems. In E. Corchado, J. A. Lozano, H. Quintián, & H. Yin (Eds.), 2014 15th International conference on intelligent data engineering automated learning—IDEAL, Salamanca, Spain, September 10–12, 2014. Proceedings (pp. 255–265). Cham: Springer. https://doi.org/10.1007/978-3-319-10840-7_32.

  125. Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x.

    Article  Google Scholar 

  126. Cover, T. M., & Thomas, J. A. (2006). Elements of information theory (2nd ed.). New York: Wiley.

    Google Scholar 

  127. Lee, W., & Xiang, D. (2001). Information-theoretic measures for anomaly detection. In Proceedings of 2001 IEEE symposium on security and privacy, S&P 2001 (pp. 130–143). IEEE Comput. Soc, n.d. https://doi.org/10.1109/SECPRI.2001.924294.

  128. David, J., & Thomas, C. (2015). DDoS attack detection using fast entropy approach on flow- based network traffic. Procedia Computer Science, 50, 30–36. https://doi.org/10.1016/j.procs.2015.04.007.

    Article  Google Scholar 

  129. Amaral, A. A., Mendes, L. S., Zarpelão, B. B., & Junior, M. L. P. (2017). Deep IP flow inspection to detect beyond network anomalies. Computer Communications, 98, 80–96. https://doi.org/10.1016/j.comcom.2016.12.007.

    Article  Google Scholar 

  130. Bhuyan, M. H., Bhattacharyya, D. K., & Kalita, J. K. (2016). A multi-step outlier-based anomaly detection approach to network-wide traffic. Information Sciences, 348, 243–271. https://doi.org/10.1016/j.ins.2016.02.023.

    Article  Google Scholar 

  131. Bereziński, P., Jasiul, B., & Szpyrka, M. (2015). An entropy-based network anomaly detection method. Entropy, 17, 2367–2408. https://doi.org/10.3390/e17042367.

    Article  Google Scholar 

  132. Behal, S., & Kumar, K. (2017). Detection of DDoS attacks and flash events using novel information theory metrics. Computer Networks, 116, 96–110. https://doi.org/10.1016/j.comnet.2017.02.015.

    Article  Google Scholar 

  133. Xie, M., Hu, J., Guo, S., & Zomaya, A. Y. (2017). Distributed segment-based anomaly detection with Kullback–Leibler divergence in wireless sensor networks. IEEE Transactions on Information Forensics and Security, 12, 101–110. https://doi.org/10.1109/TIFS.2016.2603961.

    Article  Google Scholar 

  134. Li, G., & Wang, Y. (2012). Differential Kullback–Leibler divergence based anomaly detection scheme in sensor networks. In 2012 IEEE 12th international conference on computer and information technology (pp. 966–970). IEEE. https://doi.org/10.1109/CIT.2012.197.

  135. Kar, A. K. (2016). Bio inspired computing: A review of algorithms and scope of applications. Expert Systems with Applications, 59, 20–32. https://doi.org/10.1016/j.eswa.2016.04.018.

    Article  Google Scholar 

  136. Firdaus, A., Anuar, N. B., Razak, M. F. A., & Sangaiah, A. K. (2017). Bio-inspired computational paradigm for feature investigation and malware detection: Interactive analytics. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-017-4586-0.

  137. Buczak, A. L., & Guven, E. (2016). A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Communications Surveys & Tutorials, 18, 1153–1176. https://doi.org/10.1109/COMST.2015.2494502.

    Article  Google Scholar 

  138. Sen, S. (2015). A survey of intrusion detection systems using evolutionary computation. In Bio-inspired computation in telecommunications (pp. 73–94). Elsevier. https://doi.org/10.1016/B978-0-12-801538-4.00004-5.

  139. de Castro, L. N., & Timmis, J. (2002). Artificial immune systems: A new computational intelligence approach. London: Springer.

    Google Scholar 

  140. Saurabh, P., & Verma, B. (2016). An efficient proactive artificial immune system based anomaly detection and prevention system. Expert Systems with Applications, 60, 311–320. https://doi.org/10.1016/j.eswa.2016.03.042.

    Article  Google Scholar 

  141. Igbe, O., Darwish, I., & Saadawi, T. (2016). Distributed network intrusion detection systems: An artificial immune system approach. In 2016 IEEE First international conference on connected health: applications, systems and engineering technologies (pp. 101–106). IEEE. https://doi.org/10.1109/CHASE.2016.36.

  142. Shamshirband, S., Anuar, N. B., Kiah, M. L. M., Rohani, V. A., Petković, D., Misra, S., et al. (2014). Co-FAIS: Cooperative fuzzy artificial immune system for detecting intrusion in wireless sensor networks. Journal of Network and Computer Applications, 42, 102–117. https://doi.org/10.1016/j.jnca.2014.03.012.

    Article  Google Scholar 

  143. Aslahi-Shahri, B. M., Rahmani, R., Chizari, M., Maralani, A., Eslami, M., Golkar, M. J., et al. (2016). A hybrid method consisting of GA and SVM for intrusion detection system. Neural Computing and Applications, 27, 1669–1676. https://doi.org/10.1007/s00521-015-1964-2.

    Article  Google Scholar 

  144. Singh, S., & Kushwah, R. S. (2016). Energy efficient approach for intrusion detection system for WSN by applying optimal clustering and genetic algorithm. In Proceedings of the international conference on advances in information communication technology & computing—AICTC ’16 (pp. 1–6). New York, NY: ACM Press. https://doi.org/10.1145/2979779.2979840.

  145. Hamamoto, A. H., Carvalho, L. F., Sampaio, L. D. H., Abrão, T., & Proença, M. L. (2018). Network anomaly detection system using genetic algorithm and fuzzy logic. Expert Systems with Applications, 92, 390–402. https://doi.org/10.1016/j.eswa.2017.09.013.

    Article  Google Scholar 

  146. Elsayed, S., Sarker, R., & Slay, J. (2015). Evaluating the performance of a differential evolution algorithm in anomaly detection. In 2015 IEEE congress on evolutionary computation (pp. 2490–2497). IEEE. https://doi.org/10.1109/CEC.2015.7257194.

  147. Huang, C.-L., & Dun, J.-F. (2008). A distributed PSO-SVM hybrid system with feature selection and parameter optimization. Applied Soft Computing, 8, 1381–1391. https://doi.org/10.1016/j.asoc.2007.10.007.

    Article  Google Scholar 

  148. Lin, S.-W., Ying, K.-C., Chen, S.-C., & Lee, Z.-J. (2008). Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Systems with Applications, 35, 1817–1824. https://doi.org/10.1016/j.eswa.2007.08.088.

    Article  Google Scholar 

  149. Hosseini Bamakan, S. M., Wang, H., Yingjie, T., & Shi, Y. (2016). An effective intrusion detection framework based on MCLP/SVM optimized by time-varying chaos particle swarm optimization. Neurocomputing, 199, 90–102. https://doi.org/10.1016/j.neucom.2016.03.031.

    Article  Google Scholar 

  150. de Assis, M. V. O., Hamamoto, A. H., Abrao, T., & Proenca, M. L. (2017). A game theoretical based system using holt-winters and genetic algorithm with fuzzy logic for DoS/DDoS mitigation on SDN networks. IEEE Access. https://doi.org/10.1109/ACCESS.2017.2702341.

  151. Grill, M., & Pevný, T. (2016). Learning combination of anomaly detectors for security domain. Computer Networks, 107, 55–63. https://doi.org/10.1016/j.comnet.2016.05.021.

    Article  Google Scholar 

  152. Al-Yaseen, W. L., Othman, Z. A., & Nazri, M. Z. A. (2017). Multi-level hybrid support vector machine and extreme learning machine based on modified K-means for intrusion detection system. Expert Systems with Applications, 67, 296–303. https://doi.org/10.1016/j.eswa.2016.09.041.

    Article  Google Scholar 

  153. Forestiero, A. (2016). Self-organizing anomaly detection in data streams. Information Sciences, 373, 321–336. https://doi.org/10.1016/j.ins.2016.09.007.

    Article  Google Scholar 

  154. Salem, O., Guerassimov, A., Mehaoua, A., Marcus, A., & Furht, B. (2014). Anomaly detection in medical wireless sensor networks using SVM and linear regression models. International Journal of E-Health and Medical Communications, 5, 20–45. https://doi.org/10.4018/ijehmc.2014010102.

    Google Scholar 

  155. Wang, W., Liu, J., Pitsilis, G., & Zhang, X. (2016). Abstracting massive data for lightweight intrusion detection in computer networks. Information Sciences. https://doi.org/10.1016/j.ins.2016.10.023.

  156. Adaniya, M. H. A. C., Abrão, T., & Proença, M. L, Jr. (2013). Anomaly detection using metaheuristic firefly harmonic clustering. Journal of Networks, 8, 82–91. https://doi.org/10.4304/jnw.8.1.82-91.

    Article  Google Scholar 

  157. Proenca, M. L., Zarpelao, B. B., & Mendes, L. S. (2005). Anomaly detection for network servers using digital signature of network segment. In Advanced industrial conference on telecommunications/service assurance with partial and intermittent resources conference/e-learning on telecommunications workshop (pp. 290–295). IEEE. https://doi.org/10.1109/AICT.2005.26.

  158. Chen, M.-H., Chang, P.-C., & Wu, J.-L. (2016). A population-based incremental learning approach with artificial immune system for network intrusion detection. Engineering Applications of Artificial Intelligence, 51, 171–181. https://doi.org/10.1016/j.engappai.2016.01.020.

    Article  Google Scholar 

  159. Grill, M., Pevný, T., & Rehak, M. (2017). Reducing false positives of network anomaly detection by local adaptive multivariate smoothing. Journal of Computer and System Sciences, 83, 43–57. https://doi.org/10.1016/j.jcss.2016.03.007.

    Article  Google Scholar 

  160. Guo, C., Ping, Y., Liu, N., & Luo, S.-S. (2016). A two-level hybrid approach for intrusion detection. Neurocomputing, 214, 391–400. https://doi.org/10.1016/j.neucom.2016.06.021.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by Brazilian National Council for Scientific and Technological Development (CNPq) via Grant Nos. 249794/2013-6 and 309335/2017-5, and under Grant of Project 308348/2016-8; by National Funding from the FCT—Fundação para a Ciência e a Tecnologia through the UID/EEA/500008/2013 Project; by the Government of the Russian Federation, Grant 08-08; by Finep, with resources from Funttel, Grant No. 01.14.0231.00, under the Radiocommunication Reference Center (Centro de Referência em Radiocomunicações—CRR) project of the National Institute of Telecommunications (Instituto Nacional de Telecomunicações—Inatel), Brazil; and by the Research Center of the College of Computer and Information Sciences, King Saud University. The authors are grateful for this support.

Author information

Authors and Affiliations

  1. Instituto de Telecomunicações, Universidade da Beira Interior, Covilhã, Portugal

    Gilberto Fernandes Jr. & Joel J. P. C. Rodrigues

  2. National Institute of Telecommunications (Inatel), Av. João de Camargo, 510 – Center, Santa Rita do Sapucaì, 37540-000, Brazil

    Joel J. P. C. Rodrigues

  3. ITMO University, St. Petersburg, Russia

    Joel J. P. C. Rodrigues

  4. University of Fortaleza (UNIFOR), Fortaleza, Brazil

    Joel J. P. C. Rodrigues

  5. State University of Campinas (UNICAMP), Campinas, Brazil

    Luiz Fernando Carvalho

  6. College of Computer and Information Sciences (CCIS), King Saud University (KSU), Riyadh, 12372, Saudi Arabia

    Joel J. P. C. Rodrigues & Jalal F. Al-Muhtadi

  7. Computer Science Department, State University of Londrina (UEL), Londrina, Brazil

    Mario Lemes Proença Jr.

Authors
  1. Gilberto Fernandes Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joel J. P. C. Rodrigues
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luiz Fernando Carvalho
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jalal F. Al-Muhtadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mario Lemes Proença Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joel J. P. C. Rodrigues.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fernandes, G., Rodrigues, J.J.P.C., Carvalho, L.F. et al. A comprehensive survey on network anomaly detection. Telecommun Syst 70, 447–489 (2019). https://doi.org/10.1007/s11235-018-0475-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11235-018-0475-8

Keywords

Search

Navigation