research-article

Cluster-Based Spatiotemporal Background Traffic Generation for Network Simulation

Authors:

Jason LiuAuthors Info & Claims

ACM Transactions on Modeling and Computer Simulation (TOMACS), Volume 25, Issue 1

Article No.: 4, Pages 1 - 25

https://doi.org/10.1145/2667222

Published: 13 November 2014 Publication History

Abstract

To reduce the computational complexity of large-scale network simulation, one needs to distinguish foreground traffic generated by the target applications one intends to study from background traffic that represents the bulk of the network traffic generated by other applications. Background traffic competes with foreground traffic for network resources and consequently plays an important role in determining the behavior of network applications. Existing background traffic models either operate only at coarse time granularity or focus only on individual links. There is little insight on how to meaningfully apply realistic background traffic over the entire network. In this article, we propose a method for generating background traffic with spatial and temporal characteristics observed from real traffic traces. We apply data clustering techniques to describe the behavior of end hosts as a function of multidimensional attributes and group them into distinct classes, and then map the classes to simulated routers so that we can generate traffic in accordance with the cluster-level statistics. The proposed traffic generator makes no assumption on the target network topology. It is also capable of scaling the generated traffic so that the traffic intensity can be varied accordingly in order to test applications under different and yet realistic network conditions. Experiments show that our method is able to generate traffic that maintains the same spatial and temporal characteristics as in the observed traffic traces.

References

[1]

P. Abry and D. Veitch. 1998. Wavelet analysis of long-range dependent network traffic. IEEE Transactions on Information Theory 44, 1 (1998), 2--15.

Digital Library

[2]

J. S. Ahn and P. B. Danzig. 1996. Packet network simulation: Speedup and accuracy versus timing granularity. IEEE ACM TON 4 (1996), 743--757.

Digital Library

[3]

A. Anand, V. Sekar, and A. Akella. 2009. SmartRE: An architecture for coordinated network-wide redundancy elimination. In Proceedings of SIGCOMM’09. 87--98.

Digital Library

[4]

P. Barford and M. Crovella. 1998. Generating representative web workloads for network and server performance. In Proceedings of SIGMETRICS’98, Vol. 26. 151--160.

Digital Library

[5]

S. Bhandarkar, A. L. N. Reddy, Y. Zhang, and D. Loguinov. 2007. Emulating AQM from end hosts. In Proceedings of SIGCOMM’07. 349--360.

Digital Library

[6]

V. Bharti, P. Kankar, L. Setia, G. Gursun, A. Lakhina, and M. Crovella. 2010. Inferring invisible traffic. In Proceedings of Co-NEXT’10. 1--12.

Digital Library

[7]

CAIDA. 2011. CAIDA Internet Traces. Retrieved from http://www.caida.org/data/passive/passive_2011_dataset.xml.

[8]

A. Dainotti, A. Pescapè, P. Salvo Rossi, F. Palmieri, and G. Ventre. 2008. Internet traffic modeling by means of hidden markov models. Computer Networks: The International Journal of Computer and Telecommunications Networking 52, 14 (2008), 2645--2662.

Digital Library

[9]

A. Dainotti, A. Pescapè, and G. Ventre. 2009. A cascade architecture for DoS attacks detection based on the wavelet transform. Journal of Computer Security 17 (2009), 945--968.

Digital Library

[10]

M. A. Erazo and J. Liu. 2013. Leveraging symbiotic relationship between simulation and emulation for scalable network experimentation. In Proceedings of PADS’13. 79--90.

Digital Library

[11]

V. Erramilli, M. Crovella, and N. Taft. 2006. An independent-connection model for traffic matrices. In Proceedings of IMC’06. 251--256.

Digital Library

[12]

A. Feldmann, A. Gilbert, P. Huang, and W. Willinger. 1999. Dynamics of IP traffic: A study of the role of variability and the impact of control. In Proceedings of SIGCOMM’99. 301--313.

Digital Library

[13]

Y. Guo, W. Gong, and D. Towsley. 2000. Time-stepped hybrid simulation (TSHS) for large scale networks. In Proceedings of INFOCOM’00, Vol. 2. 441--450.

[14]

M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten. 2009. The WEKA data mining software: An update. SIGKDD Explorations 11 (2009), 10--18.

Digital Library

[15]

D. Han, R. Grandl, A. Akella, and S. Seshan. 2013. FCP: A flexible transport framework for accommodating diversity. In Proceedings of SIGCOMM’13. 135--146.

Digital Library

[16]

T. Karagiannis, K. Papagiannaki, and M. Faloutsos. 2005. BLINC: Multilevel traffic classification in the dark. In Proceedings of SIGCOMM’05. 229--240.

Digital Library

[17]

G. Kesidis, A. Singh, D. Cheung, and W. W. Kwok. 1996. Feasibility of fluid-driven simulation for ATM network. In Proceedings of GLOBECOM’96, Vol. 3. 2013--2017.

[18]

C. Kim, M. Caesar, and J. Rexford. 2008. Floodless in Seattle: A scalable ethernet architecture for large enterprises. In Proceedings of SIGCOMM’08. 3--14.

Digital Library

[19]

N. Kothari, R. Mahajan, T. Millstein, R. Govindan, and M. Musuvathi. 2011. Finding protocol manipulation attacks. In Proceedings of SIGCOMM’11. 26--37.

Digital Library

[20]

A. Lakhina, K. Papagiannaki, M. Crovella, C. Diot, E. D. Kolaczyk, and N. Taft. 2004. Structural analysis of network traffic flows. In Proceedings of SIGMETRICS’04. 61--72.

Digital Library

[21]

M. Lee, N. Duffield, and R. R. Kompella. 2010. Not all microseconds are equal: Fine-grained per-flow measurements with reference latency interpolation. In Proceedings of SIGCOMM’10. 27--38.

Digital Library

[22]

T. Li, N. Van Vorst, and J. Liu. 2013. A rate-based TCP traffic model to accelerate network simulation. Transactions of the Society for Modeling and Simulation International 89, 4 (2013), 466--480.

Digital Library

[23]

B. Liu, D. R. Figueiredo, Y. Guo, J. F. Kurose, and D. Towsley. 2001. A study of networks simulation efficiency: Fluid simulation vs. packet-level simulation. In Proceedings of INFOCOM’01, Vol. 3. 1244--1253.

[24]

X. Liu, X. Yang, and Y. Xia. 2010. NetFence: Preventing internet denial of service from inside out. In Proceedings of SIGCOMM’10. 255--266.

Digital Library

[25]

J. B. MacQueen. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1. 281--297.

[26]

MAWI. 2013. MAWI Traffic Archive. Retrieved http://tracer.csl.sony.co.jp/mawi/.

[27]

A. McGregor, M. Hall, P. Lorier, J. Brunskill, A. Mcgregor, M. Hall, P. Lorier, and J. Brunskill. 2004. Flow clustering using machine learning techniques. In Proceedings of PAM’04. 205--214.

[28]

A. Medina, N. Taft, K. Salamatian, S. Bhattacharyya, and C. Diot. 2002. Traffic matrix estimation: Existing techniques and new directions. In Proceedings of SIGCOMM’02. 161--174.

Digital Library

[29]

V. Misra, W. Gong, and D. Towsley. 2000. Fluid-based analysis of a network of AQM routers supporting TCP flows with an application to RED. In Proceedings of SIGCOMM’00. 151--160.

Digital Library

[30]

V. Misra, W. Gong, and D. Towsley. 2003. Fluid models and solutions for large-scale IP networks. In Proceedings of SIGMETRICS’03. 91--101.

Digital Library

[31]

D. M. Nicol. 2001. Discrete event fluid modeling of TCP. In Proceedings of n WSC’01.

Digital Library

[32]

D. M. Nicol and G. Yan. 2004. Discrete event fluid modeling of background TCP traffic. TOMACS 14, 3 (2004), 211--250.

Digital Library

[33]

A. Nucci, A. Sridharan, and N. Taft. 2005. The problem of synthetically generating IP traffic matrices: Initial recommendations. SIGCOMM Computer Communication Review 35, 3 (2005), 19--32.

Digital Library

[34]

P. Papageorge, J. Mccann, and M. Hicks. 2009. Passive aggressive measurement with MGRP. In Proceedings of SIGCOMM’09. 255--266.

Digital Library

[35]

K. Papagiannaki, N. Taft, Z. Zhang, and C. Diot. 2003. Long-term forecasting of internet backbone traffic: Observations and initial models. In Proceedings of INFOCOM’03. 1178--1188.

Digital Library

[36]

M. Podlesny and S. Gorinsky. 2008. RD network services: Differentiation through performance incentives. In Proceedings of SIGCOMM’08. 255--266.

Digital Library

[37]

M. Roughan, A. Greenberg, C. Kalmanek, M. Rumsewicz, J. Yates, and Y. Zhang. 2002. Experience in measuring backbone traffic variability: Models, metrics, measurements and meaning. In Proceedings of IMW’02. 91--92.

Digital Library

[38]

M. Roughan, S. Sen, O. Spatscheck, and N. Duffield. 2004. Class-of-service mapping for QoS: A statistical signature-based approach to IP traffic classification. In Proceedings of IMC’04. 135--148.

Digital Library

[39]

J. Sommers and P. Barford. 2004. Self-configuring network traffic generation. In Proceedings of IMC’04. 68--81.

Digital Library

[40]

J. Sommers, R. A. Bowden, B. Eriksson, P. Barford, M. Roughan, and N. G. Duffield. 2011. Efficient network-wide flow record generation. In Proceedings of INFOCOM’11.

[41]

N. Spring, R. Mahajan, D. Wetherall, and T. Anderson. 2004. Measuring ISP topologies with Rocketfuel. IEEE ACM TON 12, 1 (2004), 2--16.

Digital Library

[42]

P. Tune and M. Roughan. 2003. Internet traffic matrices: A primer. ACM SIGCOMM eBook: Recent Advances in Networking (2003).

[43]

V. C. Valgenti and M. S. Kim. 2012. An application-level content generative model for network applications. In Proceedings of SIMUTools’02. 47--56.

Digital Library

[44]

N. Vegh. 2013. NTools traffic generator/analyzer and network emulator package. http://norvegh.com/ntools. (2013).

[45]

K. V. Vishwanath and A. Vahdat. 2006. Realistic and responsive network traffic generation. In Proceedings of SIGCOMM’06. 111--122.

Digital Library

[46]

K. V. Vishwanath and A. Vahdat. 2008. Evaluating distributed systems: Does background traffic matter. In Proceedings of the 2008 USENIX Technical Conference. 227--240.

Digital Library

[47]

Y. Wang, H. Wang, A. Mahimkar, R. Alimi, Y. Zhang, L. Qiu, and Y. R. Yang. 2010. R3: Resilient routing reconfiguration. In Proceedings of SIGCOMM’10. 291--302.

Digital Library

[48]

S. Wei, J. Mirkovic, and E. Kissel. 2006. Profiling and clustering internet hosts. In DMIN. 269--275.

[49]

M. C. Weigle, P. Adurthi, F. Hernández-Campos, K. Jeffay, and F. D. Smith. 2006. Tmix: A tool for generating realistic TCP application workloads in ns-2. CCR 36, 3 (2006), 67--76.

Digital Library

[50]

K. Xu, Z. Zhang, and S. Bhattacharyya. 2005. Profiling Internet backbone traffic: Behavior models and applications. In Proceedings of SIGCOMM’05. 169--180.

Digital Library

[51]

Y. Zhang, Z. Ge, A. Greenberg, and M. Roughan. 2005a. Network anomography. In Proceedings of IMC’05. 317--330.

Digital Library

[52]

Y. Zhang, M. Roughan, N. Duffield, and A. Greenberg. 2003a. Fast accurate computation of large-scale IP traffic matrices from link loads. In Proceedings of SIGMETRICS’03. 206--217.

Digital Library

[53]

Y. Zhang, M. Roughan, C. Lund, and D. Donoho. 2003b. An information-theoretic approach to traffic matrix estimation. In Proceedings of SIGCOMM’03. 301--312.

Digital Library

[54]

Y. Zhang, M. Roughan, C. Lund, and D. Donoho. 2005b. Estimating point-to-point and point-to-multipoint traffic matrix: An information-theoretic approach. IEEE ACM TON 13 (2005), 947--960.

Digital Library

[55]

Y. Zhang, M. Roughan, W. Willinger, and L. Qui. 2009. Spatio-temporal compressive sensing and internet traffic matrices. In Proceedings of SIGCOMM’09. 267--278.

Digital Library

Cited By

Dai BCao YWu ZXu Y(2022)IQoR-LSE: An Intelligent QoS On-Demand Routing Algorithm With Link State EstimationIEEE Systems Journal10.1109/JSYST.2022.314999016:4(5821-5830)Online publication date: Dec-2022
https://doi.org/10.1109/JSYST.2022.3149990
Sun YWang H(2021)Study on Model Adaptability Based on K-means Clustering Method and Exponential Continuous Time - Varying Characteristics2021 33rd Chinese Control and Decision Conference (CCDC)10.1109/CCDC52312.2021.9602077(1081-1084)Online publication date: 22-May-2021
https://doi.org/10.1109/CCDC52312.2021.9602077
Lin ZJain AWang CFanti GSekar V(2020)Using GANs for Sharing Networked Time Series DataProceedings of the ACM Internet Measurement Conference10.1145/3419394.3423643(464-483)Online publication date: 27-Oct-2020
https://dl.acm.org/doi/10.1145/3419394.3423643
Show More Cited By

Index Terms

Cluster-Based Spatiotemporal Background Traffic Generation for Network Simulation
1. Computing methodologies
  1. Modeling and simulation
    1. Model development and analysis

Recommendations

QoS protocols for end-to-end delay guarantee of instantaneous jobs on computer networks

QoS (Quality of Service) guarantee is highly desirable for many service-oriented computer and network applications on the Internet. This paper focuses on the timeliness aspect of QoS, especially the end-to-end delay guarantee. Resource reSerVation ...
A fluid background traffic model
ICC'09: Proceedings of the 2009 IEEE international conference on Communications

Background traffic has a significant impact on the behavior of network services and protocols. However, a detailed model of the background traffic can be extremely time consuming in simulation. In this paper, we extend our previous hybrid model that ...
The Implementation of Cluster-Based Traffic Simulation System
PAAP '11: Proceedings of the 2011 Fourth International Symposium on Parallel Architectures, Algorithms and Programming

Self-similarity is one of the most important characteristics of real network traffic. Constructing an accurate model can reflect the real network environment precisely. It is also an effective measure to predict the upcoming network traffic, thus ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Modeling and Computer Simulation

ACM Transactions on Modeling and Computer Simulation Volume 25, Issue 1

January 2015

141 pages

ISSN:1049-3301

EISSN:1558-1195

DOI:10.1145/2661171

Editor:
Adelinde M. Uhrmacher
University of Rostock, Germany

Issue’s Table of Contents

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 November 2014

Accepted: 01 August 2014

Revised: 01 May 2014

Received: 01 January 2014

Published in TOMACS Volume 25, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
374
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)1

Reflects downloads up to 12 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dai BCao YWu ZXu Y(2022)IQoR-LSE: An Intelligent QoS On-Demand Routing Algorithm With Link State EstimationIEEE Systems Journal10.1109/JSYST.2022.314999016:4(5821-5830)Online publication date: Dec-2022
https://doi.org/10.1109/JSYST.2022.3149990
Sun YWang H(2021)Study on Model Adaptability Based on K-means Clustering Method and Exponential Continuous Time - Varying Characteristics2021 33rd Chinese Control and Decision Conference (CCDC)10.1109/CCDC52312.2021.9602077(1081-1084)Online publication date: 22-May-2021
https://doi.org/10.1109/CCDC52312.2021.9602077
Lin ZJain AWang CFanti GSekar V(2020)Using GANs for Sharing Networked Time Series DataProceedings of the ACM Internet Measurement Conference10.1145/3419394.3423643(464-483)Online publication date: 27-Oct-2020
https://dl.acm.org/doi/10.1145/3419394.3423643
Premaratne UPremarathne U(2020)A Sum of Bernoulli Sources Approximation for Packet Switched Network Traffic in Backbone LinksIEEE Communications Letters10.1109/LCOMM.2019.294706024:1(141-145)Online publication date: Jan-2020
https://doi.org/10.1109/LCOMM.2019.2947060
Cao YDai BMo YXu Y(2020)IQoR: An Intelligent QoS-aware Routing Mechanism with Deep Reinforcement Learning2020 IEEE 45th Conference on Local Computer Networks (LCN)10.1109/LCN48667.2020.9314768(329-332)Online publication date: 16-Nov-2020
https://doi.org/10.1109/LCN48667.2020.9314768
Deng JTyson GCuadrado FUhlig S(2019)KeddahACM Transactions on Modeling and Computer Simulation10.1145/330150329:3(1-25)Online publication date: 19-Jun-2019
https://dl.acm.org/doi/10.1145/3301503
Fernandes SFernandes S(2017)Internet Traffic ProfilingPerformance Evaluation for Network Services, Systems and Protocols10.1007/978-3-319-54521-9_4(113-152)Online publication date: 22-Mar-2017
https://doi.org/10.1007/978-3-319-54521-9_4
Uhrmacher ABrailsford SLiu JRabe MTolk AHuschka TChick S(2016)Reproducible research in discrete event simulationProceedings of the 2016 Winter Simulation Conference10.5555/3042094.3042264(1301-1315)Online publication date: 11-Dec-2016
https://dl.acm.org/doi/10.5555/3042094.3042264
Uhrmacher ABrailsford SLiu JRabe MTolk A(2016)Panel — Reproducible research in discrete event simulation — A must or rather a maybe?2016 Winter Simulation Conference (WSC)10.1109/WSC.2016.7822185(1301-1315)Online publication date: Dec-2016
https://doi.org/10.1109/WSC.2016.7822185
Sihotang JBandung Y(2016)The impact of background traffic towards multimedia content delivery Quality on limited capacity network2016 10th International Conference on Telecommunication Systems Services and Applications (TSSA)10.1109/TSSA.2016.7871082(1-6)Online publication date: Oct-2016
https://doi.org/10.1109/TSSA.2016.7871082
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents