research-article

Shard scheduler: object placement and migration in sharded account-based blockchains

Authors:

Alberto Sonnino,

Mustafa Al-Bassam,

Etienne RivièreAuthors Info & Claims

AFT '21: Proceedings of the 3rd ACM Conference on Advances in Financial Technologies

Pages 43 - 56

https://doi.org/10.1145/3479722.3480989

Published: 23 November 2021 Publication History

Abstract

We propose Shard Scheduler, a system for object placement and migration in account-based sharded blockchains. Our system calculates optimal placement and decides on object migrations across shards. It supports complex multi-account transactions caused by smart contracts. Placement and migration decisions made by Shard Scheduler are fully deterministic, verifiable, and can be made part of the consensus protocol. Shard Scheduler reduces the number of costly cross-shard transactions, ensures balanced load distribution and maximizes the number of processed transactions for the blockchain as a whole. To this end, it leverages a novel incentive model motivating miners to maximize the global throughput of the entire blockchain rather than the throughput of a specific shard. In our simulations, Shard Scheduler can reduce the number of costly cross-shard transactions by half while ensuring equal load and increasing throughput more than 2 fold when using 60 shards. We also implement and evaluate Shard Scheduler on Chainspace, more than doubling its throughput and reducing user-perceived latency by 70% when using 10 shards.

References

[1]

Mustafa Al-Bassam, Alberto Sonnino, Shehar Bano, Dave Hrycyszyn, and George Danezis. Chainspace: A sharded smart contracts platform. In Network and Distributed System Security Symposium, NDSS, 2017.

[2]

Mustafa Al-Bassam, Alberto Sonnino, and Vitalik Buterin. Fraud proofs: Maximising light client security and scaling blockchains with dishonest majorities. arXiv preprint arXiv:1809.09044, 160, 2018.

[3]

Shehar Bano, Alberto Sonnino, Mustafa Al-Bassam, Sarah Azouvi, Patrick McCorry, Sarah Meiklejohn, and George Danezis. Sok: Consensus in the age of blockchains. In 1st ACM Conference on Advances in Financial Technologies, AFT, 2019.

Digital Library

[4]

Mathieu Baudet, George Danezis, and Alberto Sonnino. FastPay: High-performance byzantine fault tolerant settlement. In 2nd ACM Conference on Advances in Financial Technologies, AFT, 2020.

Digital Library

[5]

Gong Chen, Wenbo He, Jie Liu, Suman Nath, Leonidas Rigas, Lin Xiao, and Feng Zhao. Energy-aware server provisioning and load dispatching for connection-intensive internet services. In 5th USENIX Symposium on Networked Systems Design & Implementation, NSDI, 2008.

Digital Library

[6]

Ting Chen, Zihao Li, Yuxiao Zhu, Jiachi Chen, Xiapu Luo, John Chi-Shing Lui, Xiaodong Lin, and Xiaosong Zhang. Understanding Ethereum via graph analysis. ACM Transactions on Internet Technology (TOIT), 20(2):1--32, 2020.

[7]

Daniel Collins, Rachid Guerraoui, Jovan Komatovic, Petr Kuznetsov, Matteo Monti, Matej Pavlovic, Yvonne-Anne Pignolet, Dragos-Adrian Seredinschi, Andrei Tonkikh, and Athanasios Xygkis. Online payments by merely broadcasting messages. In 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN, 2020.

[8]

ConsenSys. Ethereum by the numbers. https://media.consensys.net/ethereum-by-the-numbers-3520f44565a9, 2018.

[9]

Inderjit Dhillon, Yuqiang Guan, and Brian Kulis. A fast kernel-based multilevel algorithm for graph clustering. In 11th ACM SIGKDD international conference on Knowledge discovery in data mining, KDD, 2005.

Digital Library

[10]

Cynthia Dwork, Nancy Lynch, and Larry Stockmeyer. Consensus in the presence of partial synchrony. Journal of the ACM (JACM), 35(2):288--323, 1988.

[11]

Aaron J Elmore, Vaibhav Arora, Rebecca Taft, Andrew Pavlo, Divyakant Agrawal, and Amr El Abbadi. Squall: Fine-grained live reconfiguration for partitioned main memory databases. In ACM SIGMOD International Conference on Management of Data, 2015.

[12]

EtherScan. Ethereum charts and statistics. https://etherscan.io/charts, 2021.

[13]

Ethereum Foundation. Ethereum 2.0 phases. https://docs.ethhub.io/ethereum-roadmap/ethereum-2.0/eth-2.0-phases/, 2019.

[14]

Enrique Fynn, AlyssonBessani, and Fernando Pedone. Smart contracts on the move. In 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN, 2020.

[15]

Enrique Fynn and Fernando Pedone. Challenges and pitfalls of partitioning blockchains. In Workshops of the 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops, DSN-W, 2018.

[16]

Jim Gray. Notes on data base operating systems. In Operating Systems, An Advanced Course, 1978.

[17]

Runchao Han, Jiangshan Yu, and Ren Zhang. Analysing and improving shard allocation protocols for sharded blockchains. IACR Cryptol. ePrint Arch., 2020, 2020.

[18]

HyperLedger. Burrow. https://github.com/hyperledger/burrow, 2021.

[19]

George Karypis and Vipin Kumar. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on scientific Computing, 20(1):359--392, 1998.

Digital Library

[20]

Aggelos Kiayias, Alexander Russell, Bernardo David, and Roman Oliynykov. Ouroboros: A provably secure proof-of-stake blockchain protocol. In Annual International Cryptology Conference, pages 357--388. Springer, 2017.

[21]

Jae-Yun Kim, Junmo Lee, Yeonjae Koo, Sanghyeon Park, and Soo-Mook Moon. Ethanos: efficient bootstrapping for full nodes on account-based blockchain. In 16th European Conference on Computer Systems, EuroSys, 2021.

[22]

Eleftherios Kokoris-Kogias, Philipp Jovanovic, Linus Gasser, Nicolas Gailly, Ewa Syta, and Bryan Ford. Omniledger: A secure, scale-out, decentralized ledger via sharding. In IEEE Symposium on Security and Privacy, S&P. IEEE, 2018.

[23]

Eleftherios Kokoris-Kogias, Philipp Jovanovic, Linus Gasser, Nicolas Gailly, Ewa Syta, and Bryan Ford. Omniledger: A secure, scale-out, decentralized ledger via sharding. In 2018 IEEE Symposium on Security and Privacy (SP), pages 583--598. IEEE, 2018.

[24]

Yu-Shan Lin, Shao-Kan Pi, Meng-Kai Liao, Ching Tsai, Aaron Elmore, and Shan-Hung Wu. Mgcrab: transaction crabbing for live migration in deterministic database systems. Proceedings of the VLDB Endowment, 12(5):597--610, 2019.

Digital Library

[25]

Loi Luu, Viswesh Narayanan, Chaodong Zheng, Kunal Baweja, Seth Gilbert, and Prateek Saxena. A secure sharding protocol for open blockchains. In ACM SIGSAC Conference on Computer and Communications Security, CCS, 2016.

Digital Library

[26]

Ralph C Merkle. A digital signature based on a conventional encryption function. In Conference on the theory and application of cryptographic techniques, pages 369--378. Springer, 1987.

[27]

Michael Mirkin, Yan Ji, Jonathan Pang, Ariah Klages-Mundt, Ittay Eyal, and Ari Juels. Bdos: Blockchain denial-of-service. In ACM SIGSAC conference on Computer and Communications Security, CCS, 2020.

[28]

Satoshi Nakamoto. Bitcoin: A peer-to-peer electronic cash system. Technical report, 2008.

[29]

Lan N Nguyen, Truc DT Nguyen, Thang N Dinh, and My T Thai. Optchain: optimal transactions placement for scalable blockchain sharding. In 39th International Conference on Distributed Computing Systems, ICDCS. IEEE, 2019.

[30]

Marco Serafini, Rebecca Taft, Aaron J Elmore, Andrew Pavlo, Ashraf Aboulnaga, and Michael Stonebraker. Clay: Fine-grained adaptive partitioning for general database schemas. Proceedings of the VLDB Endowment, 10(4):445--456, 2016.

Digital Library

[31]

Alberto Sonnino, Shehar Bano, Mustafa Al-Bassam, and George Danezis. Replay attacks and defenses against cross-shard consensus in sharded distributed ledgers. In European Symposium on Security and Privacy, EuroS&P. IEEE, 2020.

[32]

Hanyi Sun, Na Ruan, and Hanqing Liu. Ethereum analysis via node clustering. In International Conference on Network and System Security, NSS. Springer, 2019.

Digital Library

[33]

Rebecca Taft, Nosayba El-Sayed, Marco Serafini, Yu Lu, Ashraf Aboulnaga, Michael Stonebraker, Ricardo Mayerhofer, and Francisco Andrade. P-store: An elastic database system with predictive provisioning. In International Conference on Management of Data, SIGMOD, 2018.

[34]

Rebecca Taft, Essam Mansour, Marco Serafini, Jennie Duggan, Aaron J Elmore, Ashraf Aboulnaga, Andrew Pavlo, and Michael Stonebraker. E-store: Fine-grained elastic partitioning for distributed transaction processing systems. Proceedings of the VLDB Endowment, 8(3):245--256, 2014.

Digital Library

[35]

The Zilliqa Team. Zilliqa whitepaper. https://docs.zilliqa.com/whitepaper.pdf, 2017.

[36]

Gang Wang, Zhijie Jerry Shi, Mark Nixon, and Song Han. SOK: Sharding on blockchain. In 1st ACM Conference on Advances in Financial Technologies, AFT, 2019.

Digital Library

[37]

Jiaping Wang and Hao Wang. Monoxide: Scale out blockchains with asynchronous consensus zones. In 16th USENIX Symposium on Networked Systems Design and Implementation, NSDI, 2019.

[38]

Gavin Wood et al. Ethereum: A secure decentralised generalised transaction ledger. Ethereum project yellow paper, 151(2014):1--32, 2014.

[39]

Mahdi Zamani, Mahnush Movahedi, and Mariana Raykova. Rapidchain: Scaling blockchain via full sharding. In ACM SIGSAC Conference on Computer and Communications Security, CCS, 2018.

Cited By

Kruminis ENavaie KAscigil O(2024)BB-FLoC: A Blockchain-based Targeted Advertisement Scheme with K-AnonymityDistributed Ledger Technologies: Research and Practice10.1145/3672404Online publication date: 13-Jun-2024
https://dl.acm.org/doi/10.1145/3672404
Li PSong MXing MXiao ZDing QGuan SLong JChua TNgo CKa-Wei Lee RKumar RLauw H(2024)SPRING: Improving the Throughput of Sharding Blockchain via Deep Reinforcement Learning Based State PlacementProceedings of the ACM on Web Conference 202410.1145/3589334.3645386(2836-2846)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645386
Cheng FXiao JLiu CZhang SZhou YLi BLi BJin H(2024)SharDAG: Scaling DAG-Based Blockchains Via Adaptive Sharding2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00165(2068-2081)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00165
Show More Cited By

Index Terms

Shard scheduler: object placement and migration in sharded account-based blockchains
1. Security and privacy
  1. Systems security
    1. Distributed systems security

Recommendations

SkyChain: A Deep Reinforcement Learning-Empowered Dynamic Blockchain Sharding System
ICPP '20: Proceedings of the 49th International Conference on Parallel Processing

To overcome the limitations on the scalability of current blockchain systems, sharding is widely considered as a promising solution that divides the network into multiple disjoint groups processing transactions in parallel to improve throughput while ...
Shard Manager: A Generic Shard Management Framework for Geo-distributed Applications
SOSP '21: Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles

Sharding is widely used to scale an application. Despite a decade of effort to build generic sharding frameworks that can be reused across different applications, the extent of their success remains unclear. We attempt to answer a fundamental question: ...
Dual-Quorum: A Highly Available and Consistent Replication System for Edge Services

This paper introduces dual-quorum replication, a novel data replication algorithm designed to support Internet edge services. Edge services allow clients to access Internet services via distributed edge servers that operate on a shared collection of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

AFT '21: Proceedings of the 3rd ACM Conference on Advances in Financial Technologies

September 2021

225 pages

ISBN:9781450390828

DOI:10.1145/3479722

Conference Chairs:
Foteini Baldimtsi
George Mason University
,
Tim Roughgarden
Columbia University

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

ACM: Association for Computing Machinery

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 November 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

AFT '21

Sponsor:

ACM

AFT '21: 3rd ACM Conference on Advances in Financial Technologies

September 26 - 28, 2021

Virginia, Arlington

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
342
Total Downloads

Downloads (Last 12 months)121
Downloads (Last 6 weeks)11

Reflects downloads up to

Other Metrics

View Author Metrics

Citations

Cited By

Kruminis ENavaie KAscigil O(2024)BB-FLoC: A Blockchain-based Targeted Advertisement Scheme with K-AnonymityDistributed Ledger Technologies: Research and Practice10.1145/3672404Online publication date: 13-Jun-2024
https://dl.acm.org/doi/10.1145/3672404
Li PSong MXing MXiao ZDing QGuan SLong JChua TNgo CKa-Wei Lee RKumar RLauw H(2024)SPRING: Improving the Throughput of Sharding Blockchain via Deep Reinforcement Learning Based State PlacementProceedings of the ACM on Web Conference 202410.1145/3589334.3645386(2836-2846)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645386
Cheng FXiao JLiu CZhang SZhou YLi BLi BJin H(2024)SharDAG: Scaling DAG-Based Blockchains Via Adaptive Sharding2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00165(2068-2081)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00165
Jin SXia YXu B(2024)Scalability meets regulation: UTXO-based sharding and zero-knowledge proofs for regulated digital currenciesCluster Computing10.1007/s10586-023-04247-9Online publication date: 6-Feb-2024
https://doi.org/10.1007/s10586-023-04247-9
Liu CWan JLi LYao B(2023)Throughput Optimization for Blockchain System with Dynamic ShardingElectronics10.3390/electronics1224491512:24(4915)Online publication date: 6-Dec-2023
https://doi.org/10.3390/electronics12244915
Set SPark G(2023)Service-Aware Dynamic Sharding Approach for Scalable BlockchainIEEE Transactions on Services Computing10.1109/TSC.2022.323161916:4(2954-2969)Online publication date: 1-Jul-2023
https://doi.org/10.1109/TSC.2022.3231619
Li MWang WZhang J(2023)LB-Chain: Load-Balanced and Low-Latency Blockchain Sharding via Account MigrationIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.323834334:10(2797-2810)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1109/TPDS.2023.3238343
Mu KWei X(2023)EfShard: Toward Efficient State Sharding Blockchain via Flexible and Timely State AllocationIEEE Transactions on Network and Service Management10.1109/TNSM.2023.323643320:3(2817-2829)Online publication date: 12-Jan-2023
https://dl.acm.org/doi/10.1109/TNSM.2023.3236433
Yao BWan JJaffry SLi LMa ZLiu C(2023)Optimal Sharding for Dynamic Throughput Optimization in Blockchain Systems with Deep Reinforcement Learning2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC53992.2023.10394337(51-55)Online publication date: 1-Oct-2023
https://doi.org/10.1109/SMC53992.2023.10394337
Zhang YPan SYu J(2023)TxAllo: Dynamic Transaction Allocation in Sharded Blockchain Systems2023 IEEE 39th International Conference on Data Engineering (ICDE)10.1109/ICDE55515.2023.00390(721-733)Online publication date: Apr-2023
https://doi.org/10.1109/ICDE55515.2023.00390
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents