research-article

Public Access

Automatic Detection of Network Traffic Anomalies and Changes

Authors:

Kesheng WuAuthors Info & Claims

SNTA '19: Proceedings of the ACM Workshop on Systems and Network Telemetry and Analytics

Pages 3 - 10

https://doi.org/10.1145/3322798.3329255

Published: 17 June 2019 Publication History

Abstract

Accurately predicting network behavior is beneficial for TCP congestion control, and can help improve routing, allocating network resources, and optimizing network designs.This task is challenging because many factors could affect network traffic, such as the number of network sessions and synthetic reordering. There are also many ways to measure the network state, such as the number of retransmissions per flow and packet duplication. For this work, we use a set of passive TCP flow measurements collected at a major computer center on multiple data transfer nodes (DTN). To assist the operations of the computer network, we propose to detect abnormally slow network transfers in real-time. The proposed system breaks the network monitoring logs into fixed-size chunks and employs a state of art classifier to identify the slow time windows. This method will be validated on real large datasets collected from several DTNs. The proposed method is able to generate models to quickly detect large intervals of low performing network transfers, which require attention from network engineers.

References

[1]

Sebastian Abt and Harald Baier. 2014. Are we missing labels? A study of the availability of ground-truth in network security research. In 2014 Third International Workshop on Building Analysis Datasets and Gathering Experience Returns for Security (BADGERS) . IEEE, 40--55.

Digital Library

[2]

Daniele Apiletti, Elena Baralis, Tania Cerquitelli, Paolo Garza, Danilo Giordano, Marco Mellia, and Luca Venturini. 2016. SeLINA: a self-learning insightful network analyzer. IEEE Transactions on Network and Service Management, Vol. 13, 3 (2016), 696--710.

Digital Library

[3]

Raouf Boutaba, Mohammad A Salahuddin, Noura Limam, Sara Ayoubi, Nashid Shahriar, Felipe Estrada-Solano, and Oscar M Caicedo. 2018. A comprehensive survey on machine learning for networking: evolution, applications and research opportunities. Journal of Internet Services and Applications, Vol. 9, 1 (June 2018), 16.

[4]

Zhitang Chen, Jiayao Wen, and Yanhui Geng. 2016. Predicting future traffic using hidden markov models. In 2016 IEEE 24th International Conference on Network Protocols (ICNP). IEEE, 1--6.

[5]

Cecilia Dao, Xinyu Liu, Alex Sim, Craig Tull, and Kesheng Wu. 2018. Modeling data transfers: change point and anomaly detection. In 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS). IEEE, 1589--1594.

[6]

Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: A library for large linear classification. Journal of machine learning research, Vol. 9, Aug (2008), 1871--1874.

Digital Library

[7]

Alessandro Finamore, Marco Mellia, Michela Meo, Maurizio M Munafo, Politecnico Di Torino, and Dario Rossi. 2011. Experiences of internet traffic monitoring with tstat. IEEE Network, Vol. 25, 3 (2011), 8--14.

[8]

Anna Giannakou, Daniel Gunter, and Sean Peisert. 2018. Flowzilla: A Methodology for Detecting Data Transfer Anomalies in Research Networks. In 2018 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS). IEEE, 1--9.

[9]

Jinoh Kim and Alex Sim. 2017. A New Approach to Online, Multivariate Network Traffic Analysis. In Computer Communication and Networks (ICCCN), 2017 26th International Conference on. IEEE, 1--6.

[10]

Jinoh Kim, Alex Sim, Sang C Suh, and Ikkyun Kim. 2017a. An approach to online network monitoring using clustered patterns. In Computing, Networking and Communications (ICNC), 2017 International Conference on. IEEE, 656--661.

[11]

Jinoh Kim, Alex Sim, Brian Tierney, Sang Suh, and Ikkyun Kim. 2018. Multivariate network traffic analysis using clustered patterns. Computing (2018), 1--23.

Digital Library

[12]

Jinoh Kim, Wucherl Yoo, Alex Sim, Sang C Suh, and Ikkyun Kim. 2017b. A lightweight network anomaly detection technique. In Computing, Networking and Communications (ICNC), 2017 International Conference on. IEEE, 896--900.

[13]

Alina Lazar, Kesheng Wu, and Alex Sim. 2018. Predicting Network Traffic Using TCP Anomalies. In 2018 IEEE International Conference on Big Data (Big Data). IEEE, 5369--5371.

[14]

Zhengchun Liu, Rajkumar Kettimuthu, Ian Foster, and Yuanlai Liu. 2018. A comprehensive study of wide area data movement at a scientific computing facility. In 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS). IEEE, 1604--1611.

[15]

Zhengyang Liu, Malathi Veeraraghavan, Jianhui Zhou, Jason Hick, and Yee-Ting Li. 2013. On causes of GridFTP transfer throughput variance. In Proceedings of the Third International Workshop on Network-Aware Data Management. ACM, 5.

Digital Library

[16]

Leland McInnes and John Healy. 2018. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426 (2018).

[17]

Marco Mellia, Michela Meo, Luca Muscariello, and Dario Rossi. 2008a. Passive analysis of TCP anomalies. Computer Networks, Vol. 52, 14 (2008), 2663--2676.

Digital Library

[18]

Marco Mellia, Michela Meo, Luca Muscariello, and Dario Rossi. 2008b. Passive analysis of TCP anomalies. Computer Networks, Vol. 52, 14 (2008), 2663--2676.

Digital Library

[19]

Nageswara S Rao, Mariam Kiran, Cong Wang, and Anirban Mandal. 2018. Detecting Outliers in Network Transfers with Feature Extraction . Technical Report. Oak Ridge National Lab.(ORNL), Oak Ridge, TN (United States).

[20]

M Wang, Y Cui, X Wang, S Xiao, and J Jiang. 2018. Machine Learning for Networking: Workflow, Advances and Opportunities. IEEE Netw., Vol. 32, 2 (March 2018), 92--99.

[21]

Keith Winstein and Hari Balakrishnan. 2013. TCP Ex Machina: Computer-generated Congestion Control. In Proceedings of the ACM SIGCOMM 2013 Conference on SIGCOMM (SIGCOMM '13). ACM, New York, NY, USA, 123--134.

Digital Library

Cited By

Shao RSim AWu KKim J(2023)Leveraging History to Predict Infrequent Abnormal Transfers in Distributed WorkflowsSensors10.3390/s2312548523:12(5485)Online publication date: 10-Jun-2023
https://doi.org/10.3390/s23125485
Shao RKim JSim AWu KCafaro MChou JKim JSim A(2022)Predicting Slow Network Transfers in Scientific ComputingFifth International Workshop on Systems and Network Telemetry and Analytics10.1145/3526064.3534112(13-20)Online publication date: 30-Jun-2022
https://dl.acm.org/doi/10.1145/3526064.3534112
Lazar ASim AWu KCafaro MKim JSim A(2021)GPU-based Classification for Wireless Intrusion DetectionProceedings of the 2021 on Systems and Network Telemetry and Analytics10.1145/3452411.3464445(27-31)Online publication date: 21-Jun-2021
https://dl.acm.org/doi/10.1145/3452411.3464445
Show More Cited By

Index Terms

Automatic Detection of Network Traffic Anomalies and Changes
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
        Dimensionality reduction and manifold learning
    2. Machine learning approaches
      1. Classification and regression trees
2. Networks
  1. Network performance evaluation
    1. Network performance analysis

Recommendations

A modification of TCP flow control for improving end-to-end TCP performance over networks with wireless links

End-to-end Transmission Control Protocol (TCP) performance is one of the more important issues in wireless Internet services. This paper proposes the improvement of end-to-end TCP performance via a TCP-aware link layer protocol called Adaptive TCP (A-...
Logarithmic window increase for TCP Westwood+ for improvement in high speed, long distance networks

The majority of current Internet applications uses Transmission Control Protocol (TCP) for ensuring reliable end-to-end delivery of data over IP networks. The resulting path is, generally speaking, characterized by fairly large propagation delays (of ...
Empirical Mode Decomposition-empowered Network Traffic Anomaly Detection for Secure Multipath TCP Communications
Abstract
The development of new technologies such as the Internet of Things and cloud computing tests the transmission capabilities of communication networks. With the widespread application of multiple wireless access technologies, it has become popular ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SNTA '19: Proceedings of the ACM Workshop on Systems and Network Telemetry and Analytics

June 2019

58 pages

ISBN:9781450367615

DOI:10.1145/3322798

General Chairs:
Jinoh Kim
Texas A&M University, Commerce, USA
,
Alex Sim
Lawrence Berkeley National Laboratory, USA

Copyright © 2019 ACM.

© 2019 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the United States Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

University of Arizona: University of Arizona
SIGHPC: ACM Special Interest Group on High Performance Computing, Special Interest Group on High Performance Computing
SIGARCH: ACM Special Interest Group on Computer Architecture

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 June 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

U.S. Department of Energy

Conference

HPDC '19

Sponsor:

University of Arizona
SIGHPC
SIGARCH

HPDC '19: The 28th International Symposium on High-Performance Parallel and Distributed Computing

June 25, 2019

AZ, Phoenix, USA

Acceptance Rates

SNTA '19 Paper Acceptance Rate 22 of 106 submissions, 21%;

Overall Acceptance Rate 22 of 106 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
493
Total Downloads

Downloads (Last 12 months)100
Downloads (Last 6 weeks)21

Reflects downloads up to 09 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shao RSim AWu KKim J(2023)Leveraging History to Predict Infrequent Abnormal Transfers in Distributed WorkflowsSensors10.3390/s2312548523:12(5485)Online publication date: 10-Jun-2023
https://doi.org/10.3390/s23125485
Shao RKim JSim AWu KCafaro MChou JKim JSim A(2022)Predicting Slow Network Transfers in Scientific ComputingFifth International Workshop on Systems and Network Telemetry and Analytics10.1145/3526064.3534112(13-20)Online publication date: 30-Jun-2022
https://dl.acm.org/doi/10.1145/3526064.3534112
Lazar ASim AWu KCafaro MKim JSim A(2021)GPU-based Classification for Wireless Intrusion DetectionProceedings of the 2021 on Systems and Network Telemetry and Analytics10.1145/3452411.3464445(27-31)Online publication date: 21-Jun-2021
https://dl.acm.org/doi/10.1145/3452411.3464445
Bhandari SKukreja ALazar ASim AWu KCafaro MKim JSim A(2020)Feature Selection Improves Tree-based Classification for Wireless Intrusion DetectionProceedings of the 3rd International Workshop on Systems and Network Telemetry and Analytics10.1145/3391812.3396274(19-26)Online publication date: 23-Jun-2020
https://dl.acm.org/doi/10.1145/3391812.3396274
Nakashima MSim AKim JCafaro MKim JSim A(2020)Evaluation of Deep Learning Models for Network Performance Prediction for Scientific FacilitiesProceedings of the 3rd International Workshop on Systems and Network Telemetry and Analytics10.1145/3391812.3396272(53-56)Online publication date: 23-Jun-2020
https://dl.acm.org/doi/10.1145/3391812.3396272
Papadimitriou GKiran MWang CMandal ADeelman E(2019)Training Classifiers to Identify TCP Signatures in Scientific Workflows2019 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)10.1109/INDIS49552.2019.00012(61-68)Online publication date: Nov-2019
https://doi.org/10.1109/INDIS49552.2019.00012

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten