article

Gossip-based peer sampling

Authors:

Márk Jelasity,

Spyros Voulgaris,

Rachid Guerraoui,

Anne-Marie Kermarrec,

Maarten van SteenAuthors Info & Claims

ACM Transactions on Computer Systems (TOCS), Volume 25, Issue 3

Pages 8 - es

https://doi.org/10.1145/1275517.1275520

Published: 01 August 2007 Publication History

Abstract

Gossip-based communication protocols are appealing in large-scale distributed applications such as information dissemination, aggregation, and overlay topology management. This paper factors out a fundamental mechanism at the heart of all these protocols: the peer-sampling service. In short, this service provides every node with peers to gossip with. We promote this service to the level of a first-class abstraction of a large-scale distributed system, similar to a name service being a first-class abstraction of a local-area system. We present a generic framework to implement a peer-sampling service in a decentralized manner by constructing and maintaining dynamic unstructured overlays through gossiping membership information itself. Our framework generalizes existing approaches and makes it easy to discover new ones. We use this framework to empirically explore and compare several implementations of the peer-sampling service. Through extensive simulation experiments we show that---although all protocols provide a good quality uniform random stream of peers to each node locally---traditional theoretical assumptions about the randomness of the unstructured overlays as a whole do not hold in any of the instances. We also show that different design decisions result in severe differences from the point of view of two crucial aspects: load balancing and fault tolerance. Our simulations are validated by means of a wide-area implementation.

References

[1]

Albert, R., Jeong, H., and Barabási, A.-L. 2000. Error and attack tolerance of complex networks. Nature 406, 378--382.

[2]

Allavena, A., Demers, A., and Hopcroft, J. E. 2005. Correctness of a gossip based membership protocol. In Proceedings of the 24th annual ACM symposium on principles of distributed computing (PODC'05). ACM Press.

Digital Library

[3]

Barabási, A.-L. 2002. Linked: The New Science of Networks. Perseus, Cambridge, MA.

[4]

Bhagwan, R., Savage, S., and Voelker, G. 2003. Understanding Availability. In 2nd International Workshop on Peer-to-Peer Systems. Lecture Notes in Computer Science. Springer-Verlag, Berlin, Germany.

[5]

Birman, K. P., Hayden, M., Ozkasap, O., Xiao, Z., Budiu, M., and Minsky, Y. 1999. Bimodal multicast. ACM Trans. Comput. Syst. 17, 2 (May), 41--88.

Digital Library

[6]

Dabek, F., Zhao, B., Druschel, P., Kubiatowicz, J., and Stoica, I. 2003. Towards a common API for structured peer-to-peer overlays. In Proceedings of the 2nd International Workshop on Peer-to-Peer Systems (IPTPS'03). Berkeley, CA.

[7]

DAS2. The distributed ASCI supercomputer 2 (DAS-2). http://www.cs.vu.nl/das2/.

[8]

Demers, A., Greene, D., Hauser, C., Irish, W., Larson, J., Shenker, S., Sturgis, H., Swinehart, D., and Terry, D. 1987. Epidemic algorithms for replicated database maintenance. In Proceedings of the 6th Annual ACM Symposium on Principles of Distributed Computing (PODC'87). ACM Press, 1--12.

Digital Library

[9]

Dorogovtsev, S. N. and Mendes, J. F. F. 2002. Evolution of networks. Advan. Phys. 51, 1079--1187.

[10]

Eugster, P. T., Guerraoui, R., Handurukande, S. B., Kermarrec, A.-M., and Kouznetsov, P. 2003. Lightweight probabilistic broadcast. ACM Trans. Comput. Syst. 21, 4, 341--374.

Digital Library

[11]

Eugster, P. T., Guerraoui, R., Kermarrec, A.-M., and Massoulié, L. 2004. Epidemic information dissemination in distributed systems. IEEE Comput. 37, 5 (May), 60--67.

Digital Library

[12]

Ganesh, A. J., Kermarrec, A.-M., and Massoulié, L. 2003. Peer-to-peer membership management for gossip-based protocols. IEEE Trans. Comput. 52, 2 (Feb.).

Digital Library

[13]

Gupta, I., Birman, K. P., and van Renesse, R. 2002. Fighting fire with fire: using randomized gossip to combat stochastic scalability limits. Qual. Reliabi. Engin. Int. 18, 3, 165--184.

[14]

Jelasity, M., Guerraoui, R., Kermarrec, A.-M., and van Steen, M. 2004. The peer sampling service: Experimental evaluation of unstructured gossip-based implementations. In Middleware 2004, H.-A. Jacobsen, Ed. Lecture Notes in Computer Science, vol. 3231. Springer-Verlag, 79--98.

Digital Library

[15]

Jelasity, M., Kowalczyk, W., and van Steen, M. 2003. Newscast computing. Tech. rep. IR-CS-006 (Nov.), Department of Computer Science, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands.

[16]

Jelasity, M., Kowalczyk, W., and van Steen, M. 2004. An approach to massively distributed aggregate computing on peer-to-peer networks. In Proceedings of the 12th Euromicro Conference on Parallel, Distributed and Network-Based Processing (PDP'04). IEEE Computer Society, 200--207.

[17]

Jelasity, M., Montresor, A., and Babaoglu, O. 2004. A modular paradigm for building self-organizing peer-to-peer applications. In Engineering Self-Organising Systems, G. Di Marzo Serugendo, A. Karageorgos, O. F. Rana, and F. Zambonelli, Eds. Lecture Notes in Artificial Intelligence, vol. 2977. Springer, 265--282.

[18]

Jelasity, M., Montresor, A., and Babaoglu, O. 2005. Gossip-based aggregation in large dynamic networks. ACM Trans. Comput. Syst. 23, 2 (May).

Digital Library

[19]

Karp, R. M., Schindelhauer, C., Shenker, S., and Vöcking, B. 2000. Randomized Rumor Spreading. In 14th Symposium on the Foundations of Computer Science. IEEE Computer Society Press, Los Alamitos, CA., 565--574.

Digital Library

[20]

Kempe, D., Dobra, A., and Gehrke, J. 2003. Gossip-based computation of aggregate information. In Proceedings of the 44th Annual IEEE Symposium on Foundations of Computer Science (FOCS'03). IEEE Computer Society, 482--491.

Digital Library

[21]

Kermarrec, A.-M., Massoulié, L., and Ganesh, A. J. 2003. Probablistic reliable dissemination in large-scale systems. IEEE Trans. Parall. Distrib. Syst. 14, 3 (March).

Digital Library

[22]

King, V. and Saia, J. 2004. Choosing a random peer. In Proceedings of the 23rd annual ACM Symposium on Principles of Distributed Computing (PODC'04). ACM Press, 125--130.

Digital Library

[23]

Kostić, D., Rodriguez, A., Albrecht, J., Bhirud, A., and Vahdat, A. 2003. Using random subsets to build scalable network services. In Proceedings of the USENIX Symposium on Internet Technologies and Systems (USITS'03).

Digital Library

[24]

Kowalczyk, W. and Vlassis, N. 2005. Newscast EM. In 17th Advances in Neural Information Processing Systems (NIPS). L. K. Saul, Y. Weiss, and L. Bottou, Eds. MIT Press, Cambridge, MA, 713--720.

[25]

Law, C. and Siu, K.-Y. 2003. Distributed construction of random expander graphs. In Proceedings of The 22nd Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM'03). San Francisco, CA.

[26]

Li, M. and Vitányi, P. 1997. An Introduction to Kolmogorov Complexity and its Applications 2nd Ed. Springer Verlag.

Digital Library

[27]

Loguinov, D., Kumar, A., Rai, V., and Ganesh, S. 2003. Graph-theoretic analysis of structured peer-to-peer systems: Routing distances and fault resilience. In Proceedings of ACM SIGCOMM 2003. ACM Press, 395--406.

Digital Library

[28]

Marsaglia, G. 1995. The Marsaglia random number CDROM including the Diehard battery of tests of randomness. Florida State University. http://www.stat.fsu.edu/pub/diehard.

[29]

Marsaglia, G. and Tsang, W. W. 2002. Some difficult-to-pass tests of randomness. J. Statis. Softw. 7, 3, 1--8.

[30]

Montresor, A., Jelasity, M., and Babaoglu, O. 2004. Robust aggregation protocols for large-scale overlay networks. In Proceedings of the International Conference on Dependable Systems and Networks (DSN). IEEE Computer Society, 19--28.

Digital Library

[31]

Newman, M. E. J. 2002. Random graphs as models of networks. In Handbook of Graphs and Networks: From the Genome to the Internet, S. Bornholdt and H. G. Schuster Eds., John Wiley, New York, NY (Chapter 2)

[32]

Pandurangan, G., Raghavan, P., and Upfal, E. 2003. Building low-diameter peer-to-peer networks. IEEE J. Selec. Areas Comm. 21, 6 (Aug.) 995--1002.

Digital Library

[33]

Pastor-Satorras, R. and Vespignani, A. 2001. Epidemic dynamics and endemic states in complex networks. Phys. Rev. E 63, 066117.

[34]

PeerSim. PeerSim. http://peersim.sourceforge.net/.

[35]

Pittel, B. 1987. On spreading a rumor. SIAM J. Appl. Math. 47, 1 (Feb.) 213--223.

Digital Library

[36]

Ratnasamy, S., Francis, P., Handley, M., Karp, R., and Schenker, S. 2001. A scalable content-addressable network. In Proceedings of the Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM). ACM Press, 161--172.

Digital Library

[37]

Rowstron, A. and Druschel, P. 2001. Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems. In Middleware 2001. R. Guerraoui Ed. Lecture Notes in Computer Science, vol. 2218. Springer-Verlag, 329--350.

Digital Library

[38]

Saroiu, S., Gummadi, P. K., and Gribble, S. D. 2003. Measuring and analyzing the characteristics of Napster and Gnutella hosts. Multim. Syst. J. 9, 2 (Aug.) 170--184.

Digital Library

[39]

Sen, S. and Wang, J. 2004. Analyzing Peer-to-Peer Traffic Across Large Networks. IEEE/ACM Trans. Network. 12, 2 (April) 219--232.

Digital Library

[40]

Stavrou, A., Rubenstein, D., and Sahu, S. 2004. A Lightweight, Robust P2P System to Handle Flash Crowds. IEEE J. Select. Areas Comm. 22, 1 (Jan.) 6--17.

Digital Library

[41]

Stoica, I., Morris, R., Karger, D., Kaashoek, M. F., and Balakrishnan, H. 2001. Chord: A scalable peer-to-peer lookup service for internet applications. In Proceedings of the Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM). ACM Press, 149--160.

Digital Library

[42]

Sun, Q. and Sturman, D. 2000. A Gossip-Based Reliable Multicast for Large-Scale High-Throughput Applications. In International Conference on Dependable Systems and Networks. IEEE Computer Society Press, Los Alamitos, CA., 347--358.

Digital Library

[43]

van Renesse, R., Birman, K. P., and Vogels, W. 2003. Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining. ACM Trans. Comput. Syst. 21, 2 (May) 164--206.

Digital Library

[44]

van Renesse, R., Minsky, Y., and Hayden, M. 1998. A gossip-style failure detection service. In Middleware 1998. 55--70.

Digital Library

[45]

Voulgaris, S., Gavidia, D., and van Steen., M. 2005. CYCLON: Inexpensive membership management for unstructured P2P overlays. J. Netw. Syst. Manag. 13, 2, 197--217.

[46]

Voulgaris, S. and van Steen, M. 2003. An Epidemic Protocol for Managing Routing Tables in very large Peer-to-Peer Networks. In 14th Workshop on Distributed Systems: Operations and Management. Lecture Notes in Computer Science, vol. 2867. Springer-Verlag, Berlin, 41--54.

[47]

Watts, D. J. and Strogatz, S. H. 1998. Collective dynamics of 'small-world' networks. Nature 393, 440--442.

[48]

Zhong, M., Shen, K., and Seiferas, J. 2005. Non-uniform random membership management in peer-to-peer networks. In Proceedings of the IEEE INFOCOM.

Cited By

Schmid SHarth AHong JPark J(2024)Enabling Adaptation in Dynamic Manufacturing Environments with Decentralized Agent-Based Systems and Local PerceptionProceedings of the 39th ACM/SIGAPP Symposium on Applied Computing10.1145/3605098.3635967(235-242)Online publication date: 8-Apr-2024
https://dl.acm.org/doi/10.1145/3605098.3635967
Zhang XWang Q(2024)A Graph-Assisted Framework for Multiple Graph LearningIEEE Transactions on Signal and Information Processing over Networks10.1109/TSIPN.2024.335223610(162-178)Online publication date: 2024
https://doi.org/10.1109/TSIPN.2024.3352236
Król MAscigil ORene SSonnino APigaglio MSadre RLange FRivière E(2024)DISC-NG: Robust Service Discovery in the Ethereum Global Network2024 IEEE 9th European Symposium on Security and Privacy (EuroS&P)10.1109/EuroSP60621.2024.00019(193-215)Online publication date: 8-Jul-2024
https://doi.org/10.1109/EuroSP60621.2024.00019
Show More Cited By

Index Terms

Gossip-based peer sampling
1. Computer systems organization
  1. Dependable and fault-tolerant systems and networks
2. Software and its engineering
  1. Software organization and properties
    1. Software system structures
      1. Distributed systems organizing principles
        Organizing principles for web applications

Recommendations

NAT-resilient Gossip Peer Sampling
ICDCS '09: Proceedings of the 2009 29th IEEE International Conference on Distributed Computing Systems

Gossip peer sampling protocols now represent a solid basis to build and maintain peer to peer (p2p) overlay networks. They provide peers with a random sample of the network and maintain connectivity in highly dynamic settings. They rely on the ...
A Probabilistic Gossip-based Secure Protocol for Unstructured P2P Networks

Gossip based communication is an efficient tool to manage unstructured decentralized Peer-to-Peer (P2P) overlay connectivity. Such unstructured overlay connectivity is prone to security risks. Few malicious nodes are able to isolate all peer of the ...
Proactive gossip-based management of semantic overlay networks: Research Articles
Parallel and Distributed Computing (EuroPar 2005)

Much research on content-based P2P searching for file-sharing applications has focused on exploiting semantic relations between peers to facilitate searching. Current methods suggest reactive ways to manage semantic relations: they rely on the usage of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer Systems

ACM Transactions on Computer Systems Volume 25, Issue 3

August 2007

121 pages

ISSN:0734-2071

EISSN:1557-7333

DOI:10.1145/1275517

Issue’s Table of Contents

Copyright © 2007 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 August 2007

Published in TOCS Volume 25, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

390
Total Citations
View Citations
2,299
Total Downloads

Downloads (Last 12 months)67
Downloads (Last 6 weeks)4

Reflects downloads up to 12 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Schmid SHarth AHong JPark J(2024)Enabling Adaptation in Dynamic Manufacturing Environments with Decentralized Agent-Based Systems and Local PerceptionProceedings of the 39th ACM/SIGAPP Symposium on Applied Computing10.1145/3605098.3635967(235-242)Online publication date: 8-Apr-2024
https://dl.acm.org/doi/10.1145/3605098.3635967
Zhang XWang Q(2024)A Graph-Assisted Framework for Multiple Graph LearningIEEE Transactions on Signal and Information Processing over Networks10.1109/TSIPN.2024.335223610(162-178)Online publication date: 2024
https://doi.org/10.1109/TSIPN.2024.3352236
Król MAscigil ORene SSonnino APigaglio MSadre RLange FRivière E(2024)DISC-NG: Robust Service Discovery in the Ethereum Global Network2024 IEEE 9th European Symposium on Security and Privacy (EuroS&P)10.1109/EuroSP60621.2024.00019(193-215)Online publication date: 8-Jul-2024
https://doi.org/10.1109/EuroSP60621.2024.00019
Ferraz de Arruda GAleta AMoreno Y(2024)Contagion dynamics on higher-order networksNature Reviews Physics10.1038/s42254-024-00733-06:8(468-482)Online publication date: 5-Jul-2024
https://doi.org/10.1038/s42254-024-00733-0
Majumdar SPournaras E(2024)Consensus-Based Participatory Budgeting for Legitimacy: Decision Support via Multi-agent Reinforcement LearningMachine Learning, Optimization, and Data Science10.1007/978-3-031-53969-5_1(1-14)Online publication date: 16-Feb-2024
https://doi.org/10.1007/978-3-031-53969-5_1
KANEKO TSHUDO K(2023)Broadcast with Tree Selection from Multiple Spanning Trees on an Overlay NetworkIEICE Transactions on Communications10.1587/transcom.2022EBP3007E106.B:2(145-155)Online publication date: 1-Feb-2023
https://doi.org/10.1587/transcom.2022EBP3007
Mc Donnell NDuggan JHowley E(2023)A Genetic Programming-based Framework for Semi-automated Multi-agent Systems EngineeringACM Transactions on Autonomous and Adaptive Systems10.1145/358473118:2(1-30)Online publication date: 28-May-2023
https://dl.acm.org/doi/10.1145/3584731
Dhasade AKermarrec APires RSharma RVujasinovic MYoneki ENardi L(2023)Decentralized Learning Made Easy with DecentralizePyProceedings of the 3rd Workshop on Machine Learning and Systems10.1145/3578356.3592587(34-41)Online publication date: 8-May-2023
https://dl.acm.org/doi/10.1145/3578356.3592587
Frey DMostefaoui APerrin MRoman PTaïani F(2023)Differentiated Consistency for Worldwide GossipsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2022.320915034:1(1-15)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TPDS.2022.3209150
Femminella MReali G(2023)Gossip-Based Monitoring Protocol for 6G NetworksIEEE Transactions on Network and Service Management10.1109/TNSM.2023.326354220:4(4126-4140)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TNSM.2023.3263542
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents