Communication Cost for Permissionless Distributed Consensus at Internet Scale
Pages 28 - 35
Abstract
The diffusion of information that evolves a distributed computing state is a fundamental operation of a permissionless distributed consensus system (DCS). This permissionless participation decentralized the consensus over the distributed computing state, e.g., in cryptocurrencies and voting systems. For this, a permissionless DCS implements protocols to establish relationships among peers, which is then used to diffuse information. The relation establishment constitutes the control plane of the DCS, while the state diffusion is the data plane. The prevalent mechanism to realize both is a randomized peer-centric iterative diffusion. In this paper, we contrast this approach against a multicast-based design, focusing our comparison on the costs (bytes transmitted) for maintaining the relations, the control plane. We develop suitable models to account for those costs, parameterized through Internet-scale experimental insights we derived from existing DCS deployments. Our results show that the communication costs can be reduced by 30 times.
References
[1]
Amazon. 2022. Amazon Compute Service Level Agreement. https://aws.amazon.com/compute/sla/ Retrieved October, 2024 from
[2]
T. Bartczak and P. Zwierzykowski. 2012. Performance evaluation of Source-Specific Multicast routing protocols for IP networks. In 2012 8th International Symposium on Communication Systems, Networks and Digital Signal Processing (CSNDSP). 1--6. https://doi.org/10.1109/CSNDSP.2012.6292693
[3]
Inc. Brand Media. 2023. Online Tools. https://tools.iplocation.net/ Retrieved October, 2024 from
[4]
Miguel Castro and Barbara Liskov. 1999. Practical Byzantine Fault Tolerance. In Proceedings of the Third Symposium on Operating Systems Design and Implementation (New Orleans, Louisiana, USA) (OSDI '99). USENIX Association, USA, 173--186.
[5]
Yi-Ching Chiu, Brandon Schlinker, Abhishek Balaji Radhakrishnan, Ethan Katz-Bassett, and Ramesh Govindan. 2015. Are We One Hop Away from a Better Internet?. In Proceedings of the 2015 Internet Measurement Conference (Tokyo, Japan) (IMC '15). Association for Computing Machinery, New York, NY, USA, 523--529. https://doi.org/10.1145/2815675.2815719
[6]
D.D. Clark. 1989. Policy routing in Internet protocols. RFC 1102. IETF. http://tools.ietf.org/rfc/rfc1102.txt
[7]
Bram Cohen and Krzysztof Pietrzak. 2019. The Chia Network Blockchain., Vol. 1 (2019), 1--44.
[8]
Étienne Coulouma, Emmanuel Godard, and Joseph Peters. 2015. A characterization of oblivious message adversaries for which Consensus is solvable. Theoretical Computer Science, Vol. 584 (2015), 80--90. https://doi.org/10.1016/j.tcs.2015.01.024
[9]
S.E. Deering and D.R. Cheriton. 1985. Host groups: A multicast extension to the Internet Protocol. RFC 966. IETF. http://tools.ietf.org/rfc/rfc0966.txt
[10]
Xavier Défago, André Schiper, and Péter Urbán. 2004. Total Order Broadcast and Multicast Algorithms: Taxonomy and Survey. ACM Comput. Surv., Vol. 36, 4 (dec 2004), 372--421. https://doi.org/10.1145/1041680.1041682
[11]
Nicola Dimitri. 2022. Proof-of-Stake in Algorand. Distrib. Ledger Technol., Vol. 1, 2, Article 9 (dec 2022), 17 pages.
[12]
Ethereum. 2022. RLPx Go Ethereum Official Client, Accessed Jun 2024. https://github.com/ethereum/devp2p/blob/master/rlpx.md Retrieved October, 2024 from
[13]
Ethereum. 2023. Go Ethereum Official Client. https://github.com/ethereum/
[14]
Peter Fishburn. 1973. The Theory of Social Choice. Vol. 1. Princeton University Press.
[15]
Matthias Grundmann, Max Baumstark, and Hannes Hartenstein. 2022. On the Peer Degree Distribution of the Bitcoin P2P Network. IEEE International Conference on Blockchain and Cryptocurrency, ICBC 2022 July 2021 (2022). https://doi.org/10.1109/ICBC54727.2022.9805511
[16]
David Guzman, Dirk Trossen, Mike McBride, and Xinxin Fan. 2022. Insights on Impact of Distributed Ledgers on Provider Networks. In Blockchain -- ICBC 2022, Shiping Chen, Rudrapatna K. Shyamasundar, and Liang-Jie Zhang (Eds.). Springer Nature Switzerland, Cham, 3--17.
[17]
David Guzman, Dirk Trossen, and Joerg Ott. 2024. Distributed Consensus through Network Support. 2024 14th IFIP International Conference (2024).
[18]
David Guzman, Dirk Trossen, and Joerg Ott. 2024. Proliferation of the Service-centric Distributed Consensus Model and its Impact on Ethereum. IEEE International Conference on Blockchain and Cryptocurrency, ICBC 2024 (2024).
[19]
Lioba Heimbach, Lucianna Kiffer, Christof Ferreira Torres, and Roger Wattenhofer. 2023. Ethereum's Proposer-Builder Separation: Promises and Realities. In Proceedings of the 2023 ACM on Internet Measurement Conference (IMC '23). Association for Computing Machinery, New York, NY, USA, 406--420. https://doi.org/10.1145/3618257.3624824
[20]
Hetzner. 2021. Terms and Conditions Version 2.0.0. https://www.hetzner.com/assets/Uploads/downloads/AGB-en.pdf Retrieved October, 2024 from
[21]
Heidi Howard. 2014. Distributed consensus revised. University of Cambridge UCAM-CL-TR-935 (2014).
[22]
XRP Ledger. 2021. Message Routing Optimizations Proposal and Validation Relaying. Accessed June 2024. https://xrpl.org/blog/2021/message-routing-optimizations-pt-1-proposal-validation-relaying/ Retrieved October, 2024 from
[23]
Antunes Leit. 2012. Topology Management for Unstructured Overlay Networks. Technical University of Lisbon (2012).
[24]
Ang Li, Xiaowei Yang, Srikanth Kandula, and Ming Zhang. 2010. CloudCmp: Comparing public cloud providers. Proceedings of the ACM SIGCOMM Internet Measurement Conference, IMC (2010), 1--14. https://doi.org/10.1145/1879141.1879143
[25]
Yifan Mao, Soubhik Deb, Shaileshh Bojja Venkatakrishnan, Sreeram Kannan, and Kannan Srinivasan. 2020. Perigee: Efficient Peer-to-Peer Network Design for Blockchains. Proceedings of the Annual ACM Symposium on Principles of Distributed Computing (2020), 428--437. https://doi.org/10.1145/3382734.3405704 arxiv: 2006.14186
[26]
Petar Maymounkov and David Mazières. 2002. Kademlia: A Peer-to-peer Information System Based on the XOR Metric. In Peer-to-Peer Systems. https://doi.org/10.1007/3--540--45748--8
[27]
Satoshi Nakamoto. 2008. Bitcoin: A Peer-to-Peer Electronic Cash System. Journal for General Philosophy of Science, Vol. 39, 1 (2008), 53--67. https://doi.org/10.1007/s10838-008--9062-0
[28]
John Von Newman. 1956. Probabilistic Logics and the Synthesis of Reliable Organism from Unreliable Components. Automata Studies, Vol. C. Shannon (1956).
[29]
Haoran Qiu, Tao Ji, Shixiong Zhao, Xusheng Chen, Ji Qi, Heming Cui, and Sen Wang. 2023. A Geography-Based P2P Overlay Network for Fast and Robust Blockchain Systems. IEEE Transactions on Services Computing, Vol. 16, 3 (2023), 1572--1588. https://doi.org/10.1109/TSC.2022.3189667
[30]
Mohammadreza Rasolroveicy and Marios Fokaefs. 2022. Impact of DDoS Attacks on the Performance of Blockchain Consensus as an loT Data Registry: An Empirical Study. In Proceedings of the 32nd Annual International Conference on Computer Science and Software Engineering (Toronto, Canada) (CASCON '22). IBM Corp., USA, 71--80.
[31]
Elias Rohrer and Florian Tschorsch. 2019. Kadcast: A Structured Approach to Broadcast in Blockchain Networks. In Proceedings of the 1st ACM Conference on Advances in Financial Technologies (Zurich, Switzerland) (AFT '19). Association for Computing Machinery, New York, NY, USA, 199--213. https://doi.org/10.1145/3318041.3355469
[32]
Loqman Salamatian, Scott Anderson, Joshua Mathews, Paul Barford, Walter Willinger, and Mark Crovella. 2023. A Manifold View of Connectivity in the Private Backbone Networks of Hyperscalers. Commun. ACM, Vol. 66, 8 (2023), 95--103. https://doi.org/10.1145/3604620
[33]
Loqman Salamatian, Scott Anderson, Joshua Matthews, Paul Barford, Walter Willinger, and Mark Crovella. 2022. Curvature-based Analysis of Network Connectivity in Private Backbone Infrastructures. Performance Evaluation Review, Vol. 50, 1 (2022), 1--2. https://doi.org/10.1145/3489048.3522645
[34]
Loqman Salamatian, Scott Anderson, Joshua Matthews, Paul Barford, Walter Willinger, and Mark Crovella. 2024. Curvature-based Analysis of Network Connectivity in Private Backbone Infrastructures. (May 2024). https://github.com/Burdantes/
[35]
S. Santesson, R. Housley, S. Bajaj, and L. Rosenthol. 2011. Internet X.509 Public Key Infrastructure -- Certificate Image. RFC 6170. IETF. http://tools.ietf.org/rfc/rfc6170.txt
[36]
Yannis Thomas, Nikos Fotiou, Iakovos Pittaras, George Xylomenos, Spyros Voulgaris, and George C. Polyzos. 2023. Peer Clustering for the InterPlanetary File System. In Proceedings of the 2nd ACM SIGCOMM Workshop on Future of Internet Routing & Addressing (New York, NY, USA) (FIRA '23). Association for Computing Machinery, New York, NY, USA, 8--14. https://doi.org/10.1145/3607504.3609289
[37]
TON. 2024. The Open Network. https://ton.tg/ Retrieved October, 2024 from
[38]
TON. 2024. The Open Network. https://docs.ton.org/participate/run-nodes/full-node#recommended-providers Retrieved October, 2024 from
[39]
Dennis Trautwein, Aravindh Raman, Gareth Tyson, Ignacio Castro, Will Scott, Moritz Schubotz, Bela Gipp, and Yiannis Psaras. 2022. Design and evaluation of ipfs: A storage layer for the decentralizedweb. SIGCOMM 2022 - Proceedings of the ACM SIGCOMM 2022 Conference (2022), 739--752. https://doi.org/10.1145/3544216.3544232 arxiv: 2208.05877
[40]
Dirk Trossen, David Guzman, Mike McBride, and Xinxin Fan. 2021. Impact of Distributed Ledgers on Provider Networks. 935 (2021).
[41]
Vytautas Tumas, Sean Rivera, Damien Magoni, and Radu State. 2023. Topology Analysis of the XRP Ledger. In Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing (Tallinn, Estonia) (SAC '23). Association for Computing Machinery, New York, NY, USA, 1277--1284. https://doi.org/10.1145/3555776.3577611
[42]
Taotao Wang, Chonghe Zhao, Qing Yang, Shengli Zhang, and Soung Chang Liew. 2021. Ethna : Analyzing the Underlying Peer-to-Peer Network of Ethereum Blockchain. (2021), 1--15. arxiv: arXiv:2010.01373v2
[43]
Kyrill Winkler, Ami Paz, Hugo Rincon Galeana, Stefan Schmid, and Ulrich Schmid. 2023. The Time Complexity of Consensus Under Oblivious Message Adversaries., 100:1--100:0 pages. https://doi.org/10.4230/LIPIcs.ITCS.2023.100
[44]
Gavin Wood. 2014. Ethereum: a secure decentralised generalised transaction ledger. Ethereum Project Yellow Paper (2014), 1--32. https://doi.org/10.1017/CBO9781107415324.004 arxiv: arXiv:1011.1669v3
[45]
Mingxun Zhou, Liyi Zeng, Yilin Han, Peilun Li, Fan Long, Dong Zhou, Ivan Beschastnikh, and Ming Wu. 2023. Mercury: Fast Transaction Broadcast in High Performance Blockchain Systems. Proceedings - IEEE INFOCOM, Vol. 2023-May (2023), 1--10. https://doi.org/10.1109/INFOCOM53939.2023.10228972
Index Terms
- Communication Cost for Permissionless Distributed Consensus at Internet Scale
Recommendations
Fast and simple distributed consensus
The problem of fault-tolerant agreement is fundamental to distributed computing. When agreement is to be reached in spite of arbitrary behavior by faulty processors, this problem is called Distributed Consensus. By requiring that the number of faulty ...
On Consensus in Public Blockchains
ICBCT '19: Proceedings of the 2019 International Conference on Blockchain TechnologyIn this paper, we formulate the consensus problem and its solutions as seen in public blockchains and contrast them to those of the traditional distributed consensus. The Proof of Work (PoW) algorithm introduced in Bitcoin provides the first elegant and ...
Consensus Propagation
We propose consensus propagation, an asynchronous distributed protocol for averaging numbers across a network. We establish convergence, characterize the convergence rate for regular graphs, and demonstrate that the protocol exhibits better scaling ...
Comments
Information & Contributors
Information
Published In
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 09 December 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
CoNEXT '24
Sponsor:
CoNEXT '24: The 20th International Conference on emerging Networking EXperiments and Technologies
December 9 - 12, 2024
CA, Los Angeles, USA
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 54Total Downloads
- Downloads (Last 12 months)54
- Downloads (Last 6 weeks)41
Reflects downloads up to 26 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in