Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3407023.3409189acmotherconferencesArticle/Chapter ViewAbstractPublication PagesaresConference Proceedingsconference-collections
research-article

Real-time stream processing tool for detecting suspicious network patterns using machine learning

Published: 25 August 2020 Publication History

Abstract

In this paper, the performance of stream processing and accuracy in the prediction of suspicious flows in simulated network traffic is investigated. In addition, concepts of an engine that integrates with novel solutions like the Elastic-search database and Apache Kafka that allows easy definition of streams and implementation of any machine learning algorithm are presented.

References

[1]
Hamzah Al Najada, Imad Mahgoub, and Imran Mohammed. 2018. Cyber Intrusion Prediction and Taxonomy System Using Deep Learning And Distributed Big Data Processing. In 2018 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, 631--638.
[2]
Mustapha Belouch, Salah El Hadaj, and Mohamed Idhammad. 2018. Performance evaluation of intrusion detection based on machine learning using Apache Spark. Procedia Computer Science 127 (2018), 1--6.
[3]
J. Chen, K. Li, Z. Tang, K. Bilal, S. Yu, C. Weng, and K. Li. 2017. A Parallel Random Forest Algorithm for Big Data in a Spark Cloud Computing Environment. IEEE Transactions on Parallel and Distributed Systems 28, 4 (April 2017), 919--933.
[4]
Sanket Chintapalli, Derek Dagit, Bobby Evans, Reza Farivar, Thomas Graves, Mark Holderbaugh, Zhuo Liu, Kyle Nusbaum, Kishorkumar Patil, Boyang Jerry Peng, et al. 2016. Benchmarking streaming computation engines: Storm, flink and spark streaming. In 2016 IEEE international parallel and distributed processing symposium workshops (IPDPSW). IEEE, 1789--1792.
[5]
Michał Choraś and Rafał Kozik. 2015. Machine learning techniques applied to detect cyber attacks on web applications. Logic Journal of the IGPL 23, 1 (2015), 45--56.
[6]
S. Gupta and R. Rani. 2016. A comparative study of elasticsearch and CouchDB document oriented databases. In 2016 International Conference on Inventive Computation Technologies (ICICT), Vol. 1. 1--4.
[7]
B. R. Hiraman, C. Viresh M., and K. Abhijeet C. 2018. A Study of Apache Kafka in Big Data Stream Processing. In 2018 International Conference on Information, Communication, Engineering and Technology (ICICET). 1--3.
[8]
H. Isah, T. Abughofa, S. Mahfuz, D. Ajerla, F. Zulkernine, and S. Khan. 2019. A Survey of Distributed Data Stream Processing Frameworks. IEEE Access 7 (2019), 154300--154316.
[9]
Jeyhun Karimov, Tilmann Rabl, Asterios Katsifodimos, Roman Samarev, Henri Heiskanen, and Volker Markl. 2018. Benchmarking distributed stream data processing systems. In 2018 IEEE 34th International Conference on Data Engineering (ICDE). IEEE, 1507--1518.
[10]
Rafał Kozik and Michał Choraś. 2019. Protecting the application layer in the public domain with machine learning methods. Logic Journal of the IGPL 27, 2 (2019), 149--159.
[11]
Sheeraz Niaz et al. Lighari. 2018. The Efficient Way of Detecting Anomalies in Large Scale Streaming Data. University of Sindh Journal of Information and Communication Technology 2, 3 (2018), 156--161.
[12]
Nour Moustafa and Jill Slay. 2015. UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set).
[13]
Hamid Nasiri, Saeed Nasehi, and Maziar Goudarzi. 2019. Evaluation of distributed stream processing frameworks for IoT applications in Smart Cities. Journal of Big Data 6, 1 (11 Jun 2019), 52.
[14]
M. T. Tun, D. E. Nyaung, and M. P. Phyu. 2019. Performance Evaluation of Intrusion Detection Streaming Transactions Using Apache Kafka and Spark Streaming. In 2019 International Conference on Advanced Information Technologies (ICAIT). 25--30.
[15]
Melissa J. M. Turcotte, Alexander D. Kent, and Curtis Hash. 2018. Unified Host and Network Data Set. World Scientific, Chapter Chapter 1, 1--22. arXiv:https://www.worldscientific.com/doi/pdf/10.1142/9781786345646001
[16]
Petr Zapletal. [n. d.]. Comparison of Apache Stream Processing Frameworks: Part 1. https://cloud.tencent.com/developer/article/1088152. (Accessed on 01/20/2020).

Cited By

View all
  • (2023)Modern NetFlow network dataset with labeled attacks and detection methodsProceedings of the 18th International Conference on Availability, Reliability and Security10.1145/3600160.3605094(1-8)Online publication date: 29-Aug-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ARES '20: Proceedings of the 15th International Conference on Availability, Reliability and Security
August 2020
1073 pages
ISBN:9781450388337
DOI:10.1145/3407023
  • Program Chairs:
  • Melanie Volkamer,
  • Christian Wressnegger
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 August 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. anomaly detection
  2. big data
  3. cybersecurity
  4. machine learning
  5. random forest
  6. stream data

Qualifiers

  • Research-article

Funding Sources

Conference

ARES 2020

Acceptance Rates

Overall Acceptance Rate 228 of 451 submissions, 51%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)59
  • Downloads (Last 6 weeks)3
Reflects downloads up to 12 Sep 2024

Other Metrics

Citations

Cited By

View all
  • (2023)Modern NetFlow network dataset with labeled attacks and detection methodsProceedings of the 18th International Conference on Availability, Reliability and Security10.1145/3600160.3605094(1-8)Online publication date: 29-Aug-2023

View Options

Get Access

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media