Article

Simple efficient load balancing algorithms for peer-to-peer systems

Authors:

David R. Karger and

Matthias RuhlAuthors Info & Claims

SPAA '04: Proceedings of the sixteenth annual ACM symposium on Parallelism in algorithms and architectures

June 2004

Pages 36 - 43

https://doi.org/10.1145/1007912.1007919

Published: 27 June 2004 Publication History

Abstract

Load balancing is a critical issue for the efficient operation of peer-to-peer networks. We give two new load-balancing protocols whose provable performance guarantees are within a constant factor of optimal. Our protocols refine the consistent hashing data structure that underlies the Chord (and Koorde) P2P network. Both preserve Chord's logarithmic query time and near-optimal data migration cost.Consistent hashing is an instance of the distributed hash table (DHT) paradigm for assigning items to nodes in a peer-to-peer system: items and nodes are mapped to a common address space, and nodes have to store all items residing closeby in the address space.Our first protocol balances the distribution of the key address space to nodes, which yields a load-balanced system when the DHT maps items "randomly" into the address space. To our knowledge, this yields the first P2P scheme simultaneously achieving O(log n) degree, O(log n) look-up cost, and constant-factor load balance (previous schemes settled for any two of the three).Our second protocol aims to directly balance the distribution of items among the nodes. This is useful when the distribution of items in the address space cannot be randomized. We give a simple protocol that balances load by moving nodes to arbitrary locations "where they are needed." As an application, we use the last protocol to give an optimal implementation of a distributed data structure for range searches on ordered data.

References

[1]

M. Adler, E. Halperin, R. M. Karp, and V. V. Vazirani. A Stochastic Process on the Hypercube with Applications to Peer-to-Peer Networks. In Proceedings STOC, pages 575--584, June 2003.

Digital Library

[2]

A. Anagnostopoulos, A. Kirsch, and E. Upfal. Stability and Efficiency of a Random Local Load Balancing Protocol. In Proceedings FOCS, pages 472--481, Oct. 2003.

Digital Library

[3]

J. Aspnes and G. Shah. Skip Graphs. In Proceedings SODA, pages 384--393, Jan. 2003.

Digital Library

[4]

P. Ganesan and M. Bawa. Distributed Balanced Tables: Not Making a Hash of it all. Technical Report 2003-71, Stanford University, Database Group, Nov. 2003.

[5]

M. Harren, J. M. Hellerstein, R. Huebsch, B. T. Loo, S. Shenker, and I. Stoica. Complex Queries in DHT-based Peer-to-Peer Networks. In Proceedings IPTPS, pages 242--250, Mar. 2002.

Digital Library

[6]

R. Huebsch, J. M. Hellerstein, N. Lanham, B. T. Loo, S. Shenker, and I. Stoica. Querying the Internet with PIER. In Proceedings VLDB, pages 321--332, Sept. 2003.

Digital Library

[7]

F. Kaashoek and D. R. Karger. Koorde: A Simple Degree-optimal Hash Table. In Proceedings IPTPS, Feb. 2003.

[8]

D. Karger, E. Lehman, T. Leighton, M. Levine, D. Lewin, and R. Panigrahy. Consistent Hashing and Random Trees: Tools for Relieving Hot Spots on the World Wide Web. In Proceedings STOC, pages 654--663, May 1997.

Digital Library

[9]

J. Kubiatowicz, D. Bindel, Y. Chen, S. Czerwinski, P. Eaton, D. Geels, R. Gummadi, S. Rhea, H. Weatherspoon, W. Weimer, C. Wells, and B. Zhao. OceanStore: An Architecture for Global-Scale Persistent Storage. In Proceedings ASPLOS, pages 190--201, Nov. 2000.

Digital Library

[10]

D. M. Lewin. Consistent Hashing and Random Trees: Algorithms for Caching in Distributed Networks. Master's thesis, Massachusetts Institute of Technology, May 1998.

[11]

D. Liben-Nowell, H. Balakrishnan, and D. Karger. Analysis of the Evolution of Peer-to-Peer Systems. In Proceedings PODC, pages 233--242, July 2002.

Digital Library

[12]

D. Malkhi, M. Naor, and D. Ratajczak. Viceroy: A Scalable and Dynamic Emulation of the Butterfly. In Proceedings PODC, pages 183--192, July 2002.

Digital Library

[13]

P. Maymounkov and D. Mazières. Kademlia: A Peer-to-peer Information System Based on the XOR Metric. In Proceedings IPTPS, pages 53--65, Mar. 2002.

Digital Library

[14]

M. Naor and U. Wieder. Novel Architectures for P2P Applications: the Continuous-Discrete Approach. In Proceedings SPAA, pages 50--59, June 2003.

Digital Library

[15]

A. Rao, K. Lakshminarayanan, S. Surana, R. Karp, and I. Stoica. Load Balancing in Structured P2P Systems. In Proceedings IPTPS, Feb. 2003.

[16]

S. Ratnasamy, P. Francis, M. Handley, R. Karp, and S. Shenker. A Scalable Content-Addressable Network. In Proceedings ACM SIGCOMM, pages 161--172, Aug. 2001.

Digital Library

[17]

A. Rowstron and P. Druschel. Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems. In Proceedings IFIP/ACM International Conference on Distributed Systems Platforms (Middleware), pages 329--350, Nov. 2001.

Digital Library

[18]

M. Ruhl. Efficient Algorithms for New Computational Models. PhD thesis, Massachusetts Institute of Technology, Sept. 2003.

Digital Library

[19]

J. P. Schmidt, A. Siegel, and A. Srinivasan. Chernoff-Hoeffding bounds for applications with limited independence. In Proceedings SODA, pages 331--340, Jan. 1993.

Digital Library

[20]

R. G. Seidel and C. R. Aragon. Randomized Search Trees. Algorithmica, 16(4/5):464--497, Oct./Nov. 1996.

[21]

I. Stoica, R. Morris, D. Karger, F. Kaashoek, and H. Balakrishnan. Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications. In Proceedings ACM SIGCOMM, pages 149--160, Aug. 2001.

Digital Library

Cited By

Im SKumar RLi SPetety APurohit MAgrawal KPetrank E(2024)Online Load and Graph Balancing for Random Order InputsProceedings of the 36th ACM Symposium on Parallelism in Algorithms and Architectures10.1145/3626183.3659983(491-497)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3626183.3659983
Balatsouras CKarras AKarras CKarydis ISioutas S(2023)WiCHORD+: A Scalable, Sustainable, and P2P Chord-Based Ecosystem for Smart Agriculture ApplicationsSensors10.3390/s2323948623:23(9486)Online publication date: 28-Nov-2023
https://doi.org/10.3390/s23239486
Kubo TKawakami T(2023)An Adaptive Virtual Node Management Method for Overlay Networks Based on Multiple Time IntervalsComplex, Intelligent and Software Intensive Systems10.1007/978-3-031-35734-3_22(222-232)Online publication date: 19-Jun-2023
https://doi.org/10.1007/978-3-031-35734-3_22
Show More Cited By

Index Terms

Simple efficient load balancing algorithms for peer-to-peer systems
1. Software and its engineering
  1. Software organization and properties
    1. Software system structures
      1. Distributed systems organizing principles
        Organizing principles for web applications
2. Theory of computation
  1. Design and analysis of algorithms

Recommendations

Randomized load balancing strategies with churn resilience in peer-to-peer networks

The objective of load balancing in peer-to-peer (P2P) networks is to balance the workload of peer nodes in proportion to their capacity so as to eliminate performance bottlenecks. It is challenging because of the dynamic nature in overlay networks, the ...
Read More
An Efficient Hybrid Peer-to-Peer System for Distributed Data Sharing

Peer-to-peer overlay networks are widely used in distributed systems. Based on whether a regular topology is maintained among peers, peer-to-peer networks can be divided into two categories: structured peer-to-peer networks in which peers are connected ...
Read More
Locality-Aware and Churn-Resilient Load-Balancing Algorithms in Structured Peer-to-Peer Networks

Structured peer-to-peer overlay networks, like distributed hash tables (DHTs), map data items to the network based on a consistent hashing function. Such mapping for data distribution has an inherent load balance problem. Data redistribution algorithms ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SPAA '04: Proceedings of the sixteenth annual ACM symposium on Parallelism in algorithms and architectures

June 2004

332 pages

ISBN:1581138407

DOI:10.1145/1007912

General Chair:
Phil Gibbons
Intel Research
,
Program Chair:
Micah Adler
University of Massachusetts

Copyright © 2004 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 June 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SPAA04

Sponsor:

SPAA04: 16th ACM Symposium on Parallelism in Algorithms and Architectures 2004

June 27 - 30, 2004

Barcelona, Spain

Acceptance Rates

Overall Acceptance Rate 447 of 1,461 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

224
Total Citations
View Citations
2,873
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)4

Other Metrics

View Author Metrics

Citations

Cited By

Im SKumar RLi SPetety APurohit MAgrawal KPetrank E(2024)Online Load and Graph Balancing for Random Order InputsProceedings of the 36th ACM Symposium on Parallelism in Algorithms and Architectures10.1145/3626183.3659983(491-497)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3626183.3659983
Balatsouras CKarras AKarras CKarydis ISioutas S(2023)WiCHORD+: A Scalable, Sustainable, and P2P Chord-Based Ecosystem for Smart Agriculture ApplicationsSensors10.3390/s2323948623:23(9486)Online publication date: 28-Nov-2023
https://doi.org/10.3390/s23239486
Kubo TKawakami T(2023)An Adaptive Virtual Node Management Method for Overlay Networks Based on Multiple Time IntervalsComplex, Intelligent and Software Intensive Systems10.1007/978-3-031-35734-3_22(222-232)Online publication date: 19-Jun-2023
https://doi.org/10.1007/978-3-031-35734-3_22
Liu ZZhao YFan ZYang TLi XZhang RYang KZhong ZHuang YLiu CHu JXie GCui B(2022)BurstBalancer: Do Less, Better Balance for Large-scale Data Center Traffic2022 IEEE 30th International Conference on Network Protocols (ICNP)10.1109/ICNP55882.2022.9940372(1-13)Online publication date: 30-Oct-2022
https://doi.org/10.1109/ICNP55882.2022.9940372
Aliev AIsmayilova N(2022)Graph-Based Load Balancing Model for Exascale Computing Systems11th International Conference on Theory and Application of Soft Computing, Computing with Words and Perceptions and Artificial Intelligence - ICSCCW-202110.1007/978-3-030-92127-9_33(229-236)Online publication date: 4-Jan-2022
https://doi.org/10.1007/978-3-030-92127-9_33
Kubo TKawakami T(2021)An Enhanced Routing Method for Overlay Networks Based on Multiple Different Time Intervals2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC51774.2021.00232(1559-1564)Online publication date: Jul-2021
https://doi.org/10.1109/COMPSAC51774.2021.00232
DMs SN BV ST RG S(2020)SERVER LOAD BALANCING USING LOAD REBALANCING ALGORITHMINTERNATIONAL JOURNAL OF ADVANCED INFORMATION AND COMMUNICATION TECHNOLOGY10.46532/ijaict-2020006(24-26)Online publication date: 5-Feb-2020
https://doi.org/10.46532/ijaict-2020006
Safara FSouri ADeiman S(2020)Super peer selection strategy in peer‐to‐peer networks based on learning automataInternational Journal of Communication Systems10.1002/dac.429633:6Online publication date: 6-Jan-2020
https://doi.org/10.1002/dac.4296
Fan WHu CLiu MLu PYin QZhou J(2019)Dynamic scaling for parallel graph computationsProceedings of the VLDB Endowment10.14778/3324301.332430512:8(877-890)Online publication date: 1-Apr-2019
https://dl.acm.org/doi/10.14778/3324301.3324305
Archer AAydin KBateni MMirrokni VSchild AYang RZhuang R(2019)Cache-aware load balancing of data center applicationsProceedings of the VLDB Endowment10.14778/3311880.331188712:6(709-723)Online publication date: 1-Feb-2019
https://dl.acm.org/doi/10.14778/3311880.3311887
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents