research-article

Performance Analysis of Load Balancing Policies with Memory

Authors:

Benny Van HoudtAuthors Info & Claims

VALUETOOLS '20: Proceedings of the 13th EAI International Conference on Performance Evaluation Methodologies and Tools

Pages 27 - 34

https://doi.org/10.1145/3388831.3388839

Published: 29 May 2020 Publication History

Abstract

Joining the shortest or least loaded queue among d randomly selected queues are two fundamental load balancing policies. Under both policies the dispatcher does not maintain any information on the queue length or load of the servers. In this paper we analyze the performance of these policies when the dispatcher has some memory available to store the ids of some of the idle servers. We consider methods where the dispatcher discovers idle servers as well as methods where idle servers inform the dispatcher about their state.

We focus on large-scale systems and our analysis uses the cavity method. The main insight provided is that the performance measures obtained via the cavity method for a load balancing policy with memory reduce to the performance measures for the same policy without memory provided that the arrival rate is properly scaled. Thus, we can study the performance of load balancers with memory in the same manner as load balancers without memory. In particular this entails closed form solutions for joining the shortest or least loaded queue among d randomly selected queues with memory in case of exponential job sizes.

We present simulation results that support our belief that the approximation obtained by the cavity method becomes exact as the number of servers tends to infinity.

References

[1]

R. Aghajani, X. Li, and K. Ramanan. 2017. The PDE Method for the Analysis of Randomized Load Balancing Networks. Proceedings of the ACM on Measurement and Analysis of Computing Systems 1, 2 (2017), 38.

Digital Library

[2]

Jonatha Anselmi and Francois Dufour. 2020. Power-of-d-choices with memory: Fluid limit and optimality. Mathematics of Operations Research (2020).

[3]

M. Benaim and J. Le Boudec. 2008. A class of mean field interaction models for computer and communication systems. Performance evaluation 65, 11-12 (2008), 823--838.

[4]

M. Bramson, Y. Lu, and B. Prabhakar. 2010. Randomized load balancing with general service time distributions. In ACM SIGMETRICS 2010. 275--286. https://doi.org/10.1145/1811039.1811071

Digital Library

[5]

M. Bramson, Y. Lu, and B. Prabhakar. 2012. Asymptotic independence of queues under randomized load balancing. Queueing Syst. 71, 3 (2012), 247--292. https://doi.org/10.1007/s11134--012--9311--0

[6]

A. Braverman. 2018. Steady-state analysis of the Join the Shortest Queue model in the Halfin-Whitt regime. arXiv preprint arXiv:1801.05121 (2018).

[7]

S. Foss and L. Stolyar, Alexander. 2017. Large-scale join-idle-queue system with general service times. Journal of Applied Probability 54, 4 (2017), 995--1007.

[8]

D. Gamarnik, J. N. Tsitsiklis, and M. Zubeldia. 2016. Delay, memory, and messaging tradeoffs in distributed service systems. ACM SIGMETRICS Performance Evaluation Review 44, 1 (2016), 1--12.

Digital Library

[9]

T. Hellemans and B. Van Houdt. 2018. On the Power-of-d-choices with Least Loaded Server Selection. Proceedings of the ACM on Measurement and Analysis of Computing Systems 2, 2 (2018), 27.

Digital Library

[10]

T Hellemans and B Van Houdt. 2020. Performance Analysis of Load Balancing Policies with Memory. arXiv preprint arXiv:2002.06906 (2020).

[11]

J. Kriege and P. Buchholz. 2014. PH and MAP fitting with aggregated traffic traces. In Measurement, Modelling, and Evaluation of Computing Systems and Dependability and Fault Tolerance. Springer, 1--15.

[12]

Y. Lu, Q. Xie, G. Kliot, A. Geller, James R L., and A. Greenberg. 2011. Join-Idle-Queue: A novel load balancing algorithm for dynamically scalable web services. Performance Evaluation 68, 11 (2011), 1056--1071.

Digital Library

[13]

M. Mitzenmacher. 2001. The Power of Two Choices in Randomized Load Balancing. IEEE Trans. Parallel Distrib. Syst. 12 (October 2001), 1094--1104. Issue 10.

Digital Library

[14]

M. Mitzenmacher. 2019. The Supermarket Model with Known and Predicted Service Times. arXiv preprint arXiv:1905.12155 (2019).

[15]

Michael Mitzenmacher, Balaji Prabhakar, and Devavrat Shah. 2002. Load balancing with memory. In The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings. IEEE, 799--808.

[16]

K. Ousterhout, P. Wendell, M. Zaharia, and I. Stoica. 2013. Sparrow: Distributed, Low Latency Scheduling. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles (Farminton, Pennsylvania) (SOSP '13). ACM, New York, NY, USA, 69--84. https://doi.org/10.1145/2517349.2522716

[17]

A. Panchenko and A. Thümmler. 2007. Efficient Phase-type Fitting with Aggregated Traffic Traces. Perform. Eval. 64, 7-8 (Aug. 2007), 629--645. https://doi.org/10.1016/j.peva.2006.09.002

Digital Library

[18]

A. L. Stolyar. 2015. Pull-based load distribution in large-scale heterogeneous service systems. Queueing Systems 80, 4 (2015), 341--361. https://doi.org/10.1007/s11134-015-9448-8

Digital Library

[19]

Mark van der Boor, Sem Borst, and Johan van Leeuwaarden. 2019. Hyper-scalable JSQ with sparse feedback. Proceedings of the ACM on Measurement and Analysis of Computing Systems 3, 1 (2019), 1--37.

Digital Library

[20]

N.D. Vvedenskaya, R.L. Dobrushin, and F.I. Karpelevich. 1996. Queueing System with Selection of the Shortest of Two Queues: an Asymptotic Approach. Problemy Peredachi Informatsii 32 (1996), 15--27.

Cited By

Niu JWang CFeng CXu H(2022)When Power-of-d-Choices Meets Priority2022 IEEE/ACM 30th International Symposium on Quality of Service (IWQoS)10.1109/IWQoS54832.2022.9812880(1-10)Online publication date: 10-Jun-2022
https://doi.org/10.1109/IWQoS54832.2022.9812880
Hellemans TVan Houdt B(2021)Mean Waiting Time in Large-Scale and Critically Loaded Power of d Load Balancing SystemsProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/34600865:2(1-34)Online publication date: 4-Jun-2021
https://dl.acm.org/doi/10.1145/3460086
van der Boor MBorst Svan Leeuwaarden J(2021)Optimal hyper-scalable load balancing with a strict queue limitPerformance Evaluation10.1016/j.peva.2021.102217(102217)Online publication date: Jun-2021
https://doi.org/10.1016/j.peva.2021.102217

Index Terms

Performance Analysis of Load Balancing Policies with Memory
1. Mathematics of computing
  1. Probability and statistics
2. Networks
  1. Network performance evaluation
    1. Network performance modeling

Recommendations

Mean Waiting Time in Large-Scale and Critically Loaded Power of d Load Balancing Systems
POMACS

Mean field models are a popular tool used to analyse load balancing policies. In some exceptional cases the waiting time distribution of the mean field limit has an explicit form. In other cases it can be computed as the solution of a set of differential ...
Performance analysis of load balancing policies with memory
Abstract
Joining the shortest or least loaded queue among d randomly selected queues are two fundamental load balancing policies. Under both policies the dispatcher does not maintain any information on the queue length or load of the servers. ...
Performance Analysis of Workload Dependent Load Balancing Policies

Load balancing plays a crucial role in achieving low latency in large distributed systems. Recent load balancing strategies often rely on replication or use placeholders to further improve latency. However assessing the performance and stability of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

VALUETOOLS '20: Proceedings of the 13th EAI International Conference on Performance Evaluation Methodologies and Tools

May 2020

217 pages

ISBN:9781450376464

DOI:10.1145/3388831

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

EAI: The European Alliance for Innovation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 May 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

VALUETOOLS '20

VALUETOOLS '20: 13th EAI International Conference on Performance Evaluation Methodologies and Tools

May 18 - 20, 2020

Tsukuba, Japan

Acceptance Rates

Overall Acceptance Rate 90 of 196 submissions, 46%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
80
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Niu JWang CFeng CXu H(2022)When Power-of-d-Choices Meets Priority2022 IEEE/ACM 30th International Symposium on Quality of Service (IWQoS)10.1109/IWQoS54832.2022.9812880(1-10)Online publication date: 10-Jun-2022
https://doi.org/10.1109/IWQoS54832.2022.9812880
Hellemans TVan Houdt B(2021)Mean Waiting Time in Large-Scale and Critically Loaded Power of d Load Balancing SystemsProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/34600865:2(1-34)Online publication date: 4-Jun-2021
https://dl.acm.org/doi/10.1145/3460086
van der Boor MBorst Svan Leeuwaarden J(2021)Optimal hyper-scalable load balancing with a strict queue limitPerformance Evaluation10.1016/j.peva.2021.102217(102217)Online publication date: Jun-2021
https://doi.org/10.1016/j.peva.2021.102217

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents