research-article

Towards predictable datacenter networks

Authors:

Hitesh Ballani,

Thomas Karagiannis,

Ant RowstronAuthors Info & Claims

ACM SIGCOMM Computer Communication Review, Volume 41, Issue 4

Pages 242 - 253

https://doi.org/10.1145/2043164.2018465

Published: 15 August 2011 Publication History

Abstract

The shared nature of the network in today's multi-tenant datacenters implies that network performance for tenants can vary significantly. This applies to both production datacenters and cloud environments. Network performance variability hurts application performance which makes tenant costs unpredictable and causes provider revenue loss. Motivated by these factors, this paper makes the case for extending the tenant-provider interface to explicitly account for the network. We argue this can be achieved by providing tenants with a virtual network connecting their compute instances. To this effect, the key contribution of this paper is the design of virtual network abstractions that capture the trade-off between the performance guarantees offered to tenants, their costs and the provider revenue.

To illustrate the feasibility of virtual networks, we develop Oktopus, a system that implements the proposed abstractions. Using realistic, large-scale simulations and an Oktopus deployment on a 25-node two-tier testbed, we demonstrate that the use of virtual networks yields significantly better and more predictable tenant performance. Further, using a simple pricing model, we find that the our abstractions can reduce tenant costs by up to 74% while maintaining provider revenue neutrality.

Supplementary Material

JPG File (sigcomm_8_1.jpg)

Download
17.86 KB

MP4 File (sigcomm_8_1.mp4)

Download
165.13 MB

References

[1]

Amazon EC2 Spot Instances. http://aws.amazon.com/ec2/spot-instances/.

[2]

Amazon Cluster Compute, Jan. 2011. http://aws.amazon.com/ec2/hpc-applications/.

[3]

Amazon's EC2 Generating 220M, Jan. 2011. http://bit.ly/8rZdu.

[4]

Traffic Control API, Jan. 2011. http://msdn.microsoft.com/en-us/library/aa374468%28v=VS.85%29.aspx.

[5]

M. Al-Fares, A. Loukissas, and A. Vahdat. A scalable, commodity data center network architecture. In Proc. of ACM SIGCOMM, 2008.

Digital Library

[6]

M. Al-Fares, S. Radhakrishnan, B. Raghavan, N. Huang, and A. Vahdat. Hedera: Dynamic Flow Scheduling for Data Center Networks. In Proc. of USENIX NSDI, 2010.

Digital Library

[7]

G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha, and E. Harris. Reining in the Outliers in Map-Reduce Clusters using Mantri. In Proc. of USENIX OSDI, 2010.

Digital Library

[8]

H. Ballani, P. Costa, T. Karagiannis, and A. Rowstron. Towards Predictable Datacenter Networks. Technical Report MSR-TR-2011-72, Microsoft Research, May 2011.

Digital Library

[9]

R. Black, A. Donnelly, and C. Fournet. Ethernet Topology Discovery without Network Assistance. In Proc. of ICNP, 2004.

Digital Library

[10]

B. Craybrook. Comparing cloud risks and virtualization risks for data center apps, 2011. http://bit.ly/fKjwzW.

[11]

J. Duato, S. Yalamanchili, and L. Ni. Interconnection Networks: An Engineering Approach. Elsevier, 2003.

Digital Library

[12]

N. G. Duffield, P. Goyal, A. Greenberg, P. Mishra, K. K. Ramakrishnan, and J. E. van der Merive. A flexible model for resource management in virtual private networks. In Proc. of ACM SIGCOMM, 1999.

Digital Library

[13]

A. Giurgiu. Network performance in virtual infrastrucures, Feb. 2010. http://staff.science.uva.nl/ delaat/sne-2009-2010/p29/presentation.pdf.

[14]

A. Greenberg, J. R. Hamilton, N. Jain, S. Kandula, C. Kim, P. Lahiri, D. A. Maltz, P. Patel, and S. Sengupta. VL2: a scalable and flexible data center network. In Proc. of ACM SIGCOMM, 2009.

Digital Library

[15]

C. Guo, G. Lu, H. J. Wang, S. Yang, C. Kong, P. Sun, W. Wu, and Y. Zhang. SecondNet: A Data Center Network Virtualization Architecture with Bandwidth Guarantees. In Proc. of ACM CoNext, 2010.

Digital Library

[16]

M. Hajjat, X. Sun, Y.-W. E. Sung, D. Maltz, S. Rao, K. Sripanidkulchai, and M. Tawarmalani. Cloudward bound: Planning for beneficial migration of enterprise applications to the cloud. In Proc. of SIGCOMM, 2010.

Digital Library

[17]

Q. He, S. Zhou, B. Kobler, D. Duffy, and T. McGlynn. Case study for running HPC applications in public clouds. In Proc. of ACM Symposium on High Performance Distributed Computing, 2010.

Digital Library

[18]

A. Iosup, N. Yigitbasi, and D. Epema. On the Performance Variability of Production Cloud Services. Technical Report PDS-2010-002, Delft University of Technology, Jan. 2010.

[19]

S. Kandula, J. Padhye, and P. Bahl. Flyways To Decongest Data Center Networks. In Proc. of HotNets, 2005.

[20]

S. Kandula, S. Sengupta, A. Greenberg, P. Patel, and R. Chaiken. The Nature of Data Center Traffic: Measurements & Analysis. In Proc. of ACM IMC, 2009.

Digital Library

[21]

D. Kossmann, T. Kraska, and S. Loesing. An Evaluation of Alternative Architectures for Transaction Processing in the Cloud. In Proc. of international conference on Management of data (SIGMOD), 2010.

Digital Library

[22]

T. Lam, S. Radhakrishnan, A. Vahdat, and G. Varghese. NetShare: Virtualizing Data Center Networks across Services. Technical Report CS2010-0957, University of California, San Deigo, May 2010.

[23]

A. Li, X. Yang, S. Kandula, and M. Zhang. CloudCmp: comparing public cloud providers. In Proc. of conference on Internet measurement (IMC), 2010.

Digital Library

[24]

D. Mangot. Measuring EC2 system performance, May 2009. http://bit.ly/48Wui.

[25]

X. Meng, V. Pappas, and L. Zhang. Improving the Scalability of Data Center Networks with Traffic-aware Virtual Machine Placement. In Proc. of Infocom, 2010.

Digital Library

[26]

Michael Armburst et al. Above the Clouds: A Berkeley View of Cloud Computing. Technical report, University of California, Berkeley, 2009.

[27]

J. Mudigonda, P. Yalagandula, M. Al-Fares, and J. Mogul. SPAIN: COTS Data-Center Ethernet for Multipathing over Arbitrary Topologies. In Proc of NSDI, 2010.

Digital Library

[28]

B. Raghavan, K. Vishwanath, S. Ramabhadran, K. Yocum, and A. C. Snoeren. Cloud control with distributed rate limiting. In Proc. of ACM SIGCOMM, 2007.

Digital Library

[29]

R. Ricci, C. Alfeld, and J. Lepreau. A Solver for the Network Testbed Mapping problem. SIGCOMM CCR, 33, 2003.

Digital Library

[30]

J. Schad, J. Dittrich, and J.-A. Quiané-Ruiz. Runtime measurements in the cloud: observing, analyzing, and reducing variance. In Proc. of VLDB, 2010.

Digital Library

[31]

A. Shieh, S. Kandula, A. Greenberg, and C. Kim. Sharing the Datacenter Network. In Proc. of USENIX NSDI, 2011.

Digital Library

[32]

P. Soares, J. Santos, N. Tolia, and D. Guedes. Gatekeeper: Distributed Rate Control for Virtualized Datacenters. Technical Report HP-2010-151, HP Labs, 2010.

[33]

E. Walker. Benchmarking Amazon EC2 for high-performance scientific computing. Usenix Login, 2008.

[34]

G. Wang, D. G. Andersen, M. Kaminsky, K. Papagiannaki, T. S. E. Ng, M. Kozuch, and M. Ryan. c-Through: Part-time Optics in Data Centers. In Proc. of ACM SIGCOMM, 2010.

Digital Library

[35]

G. Wang and T. S. E. Ng. The Impact of Virtualization on Network Performance of Amazon EC2 Data Center. In Proc. of IEEE Infocom, 2010.

Digital Library

[36]

H. Wang, Q. Jing, S. Jiao, R. Chen, B. He, Z. Qian, and L. Zhou. Distributed Systems Meet Economics: Pricing in the Cloud. In Proc. of USENIX HotCloud, 2010.

Digital Library

[37]

M. Yu, Y. Yi, J. Rexford, and M. Chiang. Rethinking Virtual Network Embedding: substrate support for path splitting and migration. SIGCOMM CCR, 38, 2008.

Digital Library

[38]

M. Zaharia, A. Konwinski, A. D. Joseph, Y. Katz, and I. Stoica. Improving MapReduce Performance in Heterogeneous Environments. In Proc. of OSDI, 2008.

Digital Library

Cited By

You F(2024)The Static Virtual Machine Placement and Routing Spectrum Assignment for Multi-tenant in Elastic Optical NetworksComputer Science and Education. Computer Science and Technology10.1007/978-981-97-0730-0_8(72-84)Online publication date: 26-Feb-2024
https://doi.org/10.1007/978-981-97-0730-0_8
Liu AChen ZZhang YQin J(2023)HFFC: A High-Fan-Out and Flexible-Cluster Optical Network Architecture for Data CentersIEEE Photonics Journal10.1109/JPHOT.2023.333381615:6(1-12)Online publication date: Dec-2023
https://doi.org/10.1109/JPHOT.2023.3333816
Pathak SPoddar PLavanya V(2023)Swasthify : Health as a Service2023 International Conference on Networking and Communications (ICNWC)10.1109/ICNWC57852.2023.10127352(1-7)Online publication date: 5-Apr-2023
https://doi.org/10.1109/ICNWC57852.2023.10127352
Show More Cited By

Index Terms

Towards predictable datacenter networks
1. Networks
  1. Network services

Recommendations

Towards predictable datacenter networks
SIGCOMM '11: Proceedings of the ACM SIGCOMM 2011 conference

The shared nature of the network in today's multi-tenant datacenters implies that network performance for tenants can vary significantly. This applies to both production datacenters and cloud environments. Network performance variability hurts ...
Live Migration Impact on Virtual Datacenter Performance: Vmware vMotion Based Study
FICLOUD '14: Proceedings of the 2014 International Conference on Future Internet of Things and Cloud

Cloud computing is the future wave of information technology that provides infrastructure, platform and application as on demand services with low cost and rapid scalability. Infrastructure resources virtualization is the backbone of cloud computing to ...
Dynamic resource allocation in hybrid optical-electrical datacenter networks

Joint problem of bandwidth allocation and VM-placement in optical-electrical DC.Challenges: (i) limited reachability due to the size of the optical switch in the datacenter netework, (ii) dynamic creation of optical paths lead to the creation of ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGCOMM Computer Communication Review

ACM SIGCOMM Computer Communication Review Volume 41, Issue 4

SIGCOMM '11

August 2011

480 pages

ISSN:0146-4833

DOI:10.1145/2043164

Issue’s Table of Contents

SIGCOMM '11: Proceedings of the ACM SIGCOMM 2011 conference
August 2011
502 pages
ISBN:9781450307970
DOI:10.1145/2018436
General Chairs:
Srinivasan Keshav
University of Waterloo, Canada
,
Jörg Liebeherr
University of Toronto, Canada
,
Program Chairs:
John Byers
Boston University, USA
,
Jeffrey Mogul
HP Labs, USA

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 August 2011

Published in SIGCOMM-CCR Volume 41, Issue 4

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

651
Total Citations
View Citations
3,650
Total Downloads

Downloads (Last 12 months)250
Downloads (Last 6 weeks)42

Reflects downloads up to 06 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

You F(2024)The Static Virtual Machine Placement and Routing Spectrum Assignment for Multi-tenant in Elastic Optical NetworksComputer Science and Education. Computer Science and Technology10.1007/978-981-97-0730-0_8(72-84)Online publication date: 26-Feb-2024
https://doi.org/10.1007/978-981-97-0730-0_8
Liu AChen ZZhang YQin J(2023)HFFC: A High-Fan-Out and Flexible-Cluster Optical Network Architecture for Data CentersIEEE Photonics Journal10.1109/JPHOT.2023.333381615:6(1-12)Online publication date: Dec-2023
https://doi.org/10.1109/JPHOT.2023.3333816
Pathak SPoddar PLavanya V(2023)Swasthify : Health as a Service2023 International Conference on Networking and Communications (ICNWC)10.1109/ICNWC57852.2023.10127352(1-7)Online publication date: 5-Apr-2023
https://doi.org/10.1109/ICNWC57852.2023.10127352
Liu AChen ZKochan OHamula P(2023)An Adaptive Spectrum Time Assignment in OFDM Datacenter Optical Networks2023 IEEE 5th International Conference on Advanced Information and Communication Technologies (AICT)10.1109/AICT61584.2023.10452424(23-28)Online publication date: 21-Nov-2023
https://doi.org/10.1109/AICT61584.2023.10452424
Chen MHou JSheng YWu YWang SLu JFan Q(2023)HA-D3QN: Embedding virtual private cloud in cloud data centers with heuristic assisted deep reinforcement learningFuture Generation Computer Systems10.1016/j.future.2023.05.025148(1-14)Online publication date: Nov-2023
https://doi.org/10.1016/j.future.2023.05.025
Hbaieb AKhemakhem M(2023)Optimal network‐aware virtual data center embeddingInternational Journal of Network Management10.1002/nem.222333:2Online publication date: 19-Jan-2023
https://doi.org/10.1002/nem.2223
Blocher MWang LEugster PSchmidt M(2022)Holistic Resource Scheduling for Data Center In-Network ComputingIEEE/ACM Transactions on Networking10.1109/TNET.2022.317478330:6(2448-2463)Online publication date: Dec-2022
https://doi.org/10.1109/TNET.2022.3174783
Mostafaei HAfridi SAbawajy J(2022)Network-aware worker placement for wide-area streaming analyticsFuture Generation Computer Systems10.1016/j.future.2022.06.009136:C(270-281)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1016/j.future.2022.06.009
Di Stefano ADi Stefano AMorana G(2022)Improving QoS through network isolation in PaaSFuture Generation Computer Systems10.1016/j.future.2022.01.010131:C(91-105)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1016/j.future.2022.01.010
Blenk AKellerer WWebster J(2022)Network Virtualization and Network HypervisorsWiley Encyclopedia of Electrical and Electronics Engineering10.1002/047134608X.W8428(1-11)Online publication date: 17-Feb-2022
https://doi.org/10.1002/047134608X.W8428
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents