research-article

Towards In-network Acceleration of Erasure Coding

Authors:

Jun BiAuthors Info & Claims

SOSR '20: Proceedings of the Symposium on SDN Research

Pages 41 - 47

https://doi.org/10.1145/3373360.3380833

Published: 04 March 2020 Publication History

Abstract

In distributed storage systems, erasure coding (EC) is a crucial technology to enable high fault tolerance with lower storage overheads than data replication. EC can reconstruct missing data by downloading parity data from survived machines. However, downloading streams of EC multiplex the available network I/O on the receiving end, leading to a substantially low data reconstruction speed. In this paper, we present NetEC, a novel in-network accelerating system that fully offloads EC to programmable switching ASICs. NetEC prevents multiplexing network I/O through on-switch downloading stream aggregation, thus significantly improving reconstruction speed. NetEC addresses three key challenges: computation offloading of complex EC operations, rate synchronization of multiple downloading streams, and deep payload inspection/assembly. We implement NetEC on hardware programmable switches. Evaluation shows that compared to HDFS-EC, NetEC significantly improves reconstruction rate by 2.7x-9.0x and eliminates CPU overheads, with low switch memory usage.

References

[1]

2018. Apache Hadoop. (2018). https://hadoop.apache.org/.

[2]

2018. Barefoot Capilano. https://barefootnetworks.com/products/brief-capilano/. (2018).

[3]

2018. HDFS Erasure Coding. https://hadoop.apache.org/docs/r3.1.1/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html. (2018).

[4]

2018. Intel Intelligent Storage Acceleration Library (Intel ISA-L). https://software.intel.com/en-us/storage/ISA-L. (2018).

[5]

2018. RFC2018: TCP Selective Acknowledgment Options. https://tools.ietf.org/html/rfc2018. (2018).

[6]

2018. RFC793: TRANSMISSION CONTROL PROTOCOL. https://tools.ietf.org/html/rfc793. (2018).

[7]

Pat Bosshart, Dan Daly, Glen Gibb, Martin Izzard, Nick McKeown, Jennifer Rexford, Cole Schlesinger, Dan Talayco, Amin Vahdat, George Varghese, et al. 2014. P4: Programming protocol-independent packet processors. ACM SIGCOMM Computer Communication Review 44, 3 (2014), 87--95.

Digital Library

[8]

Sharad Chole, Andy Fingerhut, Sha Ma, Anirudh Sivaraman, Shay Vargaftik, Alon Berger, Gal Mendelson, Mohammad Alizadeh, Shang-Tse Chuang, Isaac Keslassy, et al. 2017. drmt: Disaggregated programmable switching. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. ACM, 1--14.

Digital Library

[9]

Juntao Fang, Shenggang Wan, and Xubin He. 2018. RAFI: Risk-Aware Failure Identification to Improve the RAS in Erasure-coded Data Centers. In 2018 USENIX Annual Technical Conference (USENIX ATC 18). 495--506.

[10]

Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung. 2003. The Google file system. Vol. 37. ACM.

[11]

Cheng Huang, Huseyin Simitci, Yikang Xu, Aaron Ogus, Brad Calder, Parikshit Gopalan, Jin Li, Sergey Yekhanin, et al. [n. d.]. Erasure Coding in Windows Azure Storage. In Usenix annual technical conference. Boston, MA, 15--26.

[12]

Theo Jepsen, Daniel Alvarez, Nate Foster, Changhoon Kim, Jeongkeun Lee, Masoud Moshref, and Robert Soulé. 2019. Fast String Searching on PISA. In Proceedings of the 2019 ACM Symposium on SDN Research. ACM, 21--28.

Digital Library

[13]

Xin Jin, Xiaozhou Li, Haoyu Zhang, Nate Foster, Jeongkeun Lee, Robert Soulé, Changhoon Kim, and Ion Stoica. 2018. NetChain: Scale-Free Sub-RTT Coordination. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18). USENIX Association.

[14]

Xin Jin, Xiaozhou Li, Haoyu Zhang, Robert Soulé, Jeongkeun Lee, Nate Foster, Changhoon Kim, and Ion Stoica. 2017. NetCache: Balancing Key-Value Stores with Fast In-Network Caching. In Proceedings of the 26th Symposium on Operating Systems Principles. ACM, 121--136.

Digital Library

[15]

Osama Khan, Randal C Burns, James S Plank, William Pierce, and Cheng Huang. 2012. Rethinking erasure codes for cloud file systems: minimizing I/O for recovery and degraded reads. In FAST. 20.

[16]

Daehyeok Kim, Amirsaman Memaripour, Anirudh Badam, Yibo Zhu, Hongqiang Harry Liu, Jitu Padhye, Shachar Raindel, Steven Swanson, Vyas Sekar, and Srinivasan Seshan. 2018. Hyperloop: group-based NIC-offloading to accelerate replicated transactions in multi-tenant storage systems. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication. ACM, 297--312.

Digital Library

[17]

Zaoxing Liu, Antonis Manousis, Gregory Vorsanger, Vyas Sekar, and Vladimir Braverman. 2016. One sketch to rule them all: Rethinking network flow monitoring with univmon. In Proceedings of the 2016 ACM SIGCOMM Conference. ACM, 101--114.

Digital Library

[18]

Rui Miao, Hongyi Zeng, Changhoon Kim, Jeongkeun Lee, and Minlan Yu. 2017. SilkRoad: Making Stateful Layer-4 Load Balancing Fast and Cheap Using Switching ASICs. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. ACM, 15--28.

Digital Library

[19]

James S Plank. 1997. A tutorial on Reed-Solomon coding for fault-tolerance in RAID-like systems. Software: Practice and Experience 27, 9 (1997), 995--1012.

Digital Library

[20]

KV Rashmi, Preetum Nakkiran, Jingyan Wang, Nihar B Shah, and Kannan Ramchandran. 2015. Having Your Cake and Eating It Too: Jointly Optimal Erasure Codes for I/O, Storage, and Network-bandwidth. In FAST. 81--94.

[21]

KV Rashmi, Nihar B Shah, Dikang Gu, Hairong Kuang, Dhruba Borthakur, and Kannan Ramchandran. 2015. A hitchhiker's guide to fast and efficient data reconstruction in erasure-coded data centers. ACM SIGCOMM Computer Communication Review 44, 4 (2015), 331--342.

Digital Library

[22]

KV Rashmi, Nihar B Shah, and P Vijay Kumar. [n. d.]. Optimal Exact-Regenerating Codes for the MSR and MBR Points via a Product-Matrix Construction. submitted to IEEE Transactions on Information Theory. Available online at arxiv 1005 ([n.d.]).

[23]

Irving S Reed and Gustave Solomon. 1960. Polynomial codes over certain finite fields. Journal of the society for industrial and applied mathematics 8, 2 (1960), 300--304.

[24]

Maheswaran Sathiamoorthy, Megasthenis Asteris, Dimitris Papailiopoulos, Alexandros G Dimakis, Ramkumar Vadali, Scott Chen, and Dhruba Borthakur. 2013. Xoring elephants: Novel erasure codes for big data. In Proceedings of the VLDB Endowment, Vol. 6. VLDB Endowment, 325--336.

Digital Library

[25]

Anirudh Sivaraman, Alvin Cheung, Mihai Budiu, Changhoon Kim, Mohammad Alizadeh, Hari Balakrishnan, George Varghese, Nick McKeown, and Steve Licking. 2016. Packet transactions: High-level programming for line-rate switches. In Proceedings of the 2016 ACM SIGCOMM Conference. ACM, 15--28.

Digital Library

[26]

Vibhaalakshmi Sivaraman, Srinivas Narayana, Ori Rottenstreich, S Muthukrishnan, and Jennifer Rexford. 2017. Heavy-hitter detection entirely in the data plane. In Proceedings of the Symposium on SDN Research. ACM, 164--176.

Digital Library

[27]

John Sonchack, Oliver Michel, Adam J Aviv, Eric Keller, and Jonathan M Smith. 2018. Scaling Hardware Accelerated Network Monitoring to Concurrent and Dynamic Queries With* Flow. In 2018 USENIX Annual Technical Conference (USENIX ATC18). USENIX Association.

[28]

Fang Wang, Yingjie Tang, Yanwen Xie, and Xuehai Tang. 2019. XORInc: Optimizing Data Repair and Update for Erasure-Coded Systems with XOR-Based In-Network Computation. In 2019 35th Symposium on Mass Storage Systems and Technologies (MSST). IEEE, 244--256.

[29]

Dingming Wu, Ang Chen, TS Eugene Ng, Guohui Wang, and Haiyong Wang. 2019. Accelerated Service Chaining on a Single Switch ASIC. (2019).

[30]

Mingyuan Xia, Mohit Saxena, Mario Blaum, and David Pease. 2015. A Tale of Two Erasure Codes in HDFS. In FAST. 213--226.

Cited By

Cong YXie KWen JZhang JYin YZhang QXie GLiang W(2024)Per-Packet Traffic Measurement in Storage, Computation and Bandwidth Limited Data PlaneIEEE/ACM Transactions on Networking10.1109/TNET.2024.340401132:5(3730-3742)Online publication date: Oct-2024
https://doi.org/10.1109/TNET.2024.3404011
Shin DKim J(2023)Cache-Based Matrix Technology for Efficient Write and Recovery in Erasure Coding Distributed File SystemsSymmetry10.3390/sym1504087215:4(872)Online publication date: 6-Apr-2023
https://doi.org/10.3390/sym15040872
KIM EKIM JSHIN H(2022)Cooperative Recording to Increase Storage Efficiency in Networked Home AppliancesIEICE Transactions on Information and Systems10.1587/transinf.2021EDL8077E105.D:3(727-731)Online publication date: 1-Mar-2022
https://doi.org/10.1587/transinf.2021EDL8077
Show More Cited By

Index Terms

Towards In-network Acceleration of Erasure Coding
1. Networks
  1. Network services
    1. In-network processing

Recommendations

Precise Time-synchronization in the Data-Plane using Programmable Switching ASICs
SOSR '19: Proceedings of the 2019 ACM Symposium on SDN Research

Current implementations of time synchronization protocols (e.g. PTP) in standard industry-grade switches handle the protocol stack in the slow-path (control-plane). With new use cases of in-network computing using programmable switching ASICs, global ...
Rethinking Erasure-Coding Libraries in the Age of Optimized Machine Learning
HotStorage '24: Proceedings of the 16th ACM Workshop on Hot Topics in Storage and File Systems

Erasure codes are critical tools for building fault-tolerant and resource-efficient storage systems. However developing and maintaining optimized erasure-coding libraries are challenging. We make the case that the growth of fast machine-learning (ML) ...
Accelerating erasure coding by exploiting multiple repair paths in distributed storage systems
Abstract
High reliability must be ensured in distributed storage systems (DSSs) to maintain the stability of warehouse-scale computing and high-performance computing (HPC) systems. For system-level reliability, a repair operation using redundant storage ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SOSR '20: Proceedings of the Symposium on SDN Research

March 2020

151 pages

ISBN:9781450371018

DOI:10.1145/3373360

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 March 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SOSR '20

Sponsor:

SIGCOMM

SOSR '20: Symposium on SDN Research

March 3, 2020

CA, San Jose, USA

Acceptance Rates

Overall Acceptance Rate 7 of 43 submissions, 16%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
514
Total Downloads

Downloads (Last 12 months)47
Downloads (Last 6 weeks)3

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cong YXie KWen JZhang JYin YZhang QXie GLiang W(2024)Per-Packet Traffic Measurement in Storage, Computation and Bandwidth Limited Data PlaneIEEE/ACM Transactions on Networking10.1109/TNET.2024.340401132:5(3730-3742)Online publication date: Oct-2024
https://doi.org/10.1109/TNET.2024.3404011
Shin DKim J(2023)Cache-Based Matrix Technology for Efficient Write and Recovery in Erasure Coding Distributed File SystemsSymmetry10.3390/sym1504087215:4(872)Online publication date: 6-Apr-2023
https://doi.org/10.3390/sym15040872
KIM EKIM JSHIN H(2022)Cooperative Recording to Increase Storage Efficiency in Networked Home AppliancesIEICE Transactions on Information and Systems10.1587/transinf.2021EDL8077E105.D:3(727-731)Online publication date: 1-Mar-2022
https://doi.org/10.1587/transinf.2021EDL8077
Zhang PPan HLi ZCui PJia RHe PZhang ZTyson GXie G(2022)NetSHa: In-Network Acceleration of LSH-Based Distributed SearchIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2021.313584233:9(2213-2229)Online publication date: 1-Sep-2022
https://doi.org/10.1109/TPDS.2021.3135842
Harkous HJarschel MHe MPries RKellerer W(2021)P8: P4 With Predictable Packet Processing PerformanceIEEE Transactions on Network and Service Management10.1109/TNSM.2020.303010218:3(2846-2859)Online publication date: Sep-2021
https://doi.org/10.1109/TNSM.2020.3030102
Kfoury ECrichigno JBou-Harb E(2021)An Exhaustive Survey on P4 Programmable Data Plane Switches: Taxonomy, Applications, Challenges, and Future TrendsIEEE Access10.1109/ACCESS.2021.30867049(87094-87155)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3086704
Bressana PZilberman NVucinic DSoulé R(2020)Trading Latency for Compute in the NetworkProceedings of the Workshop on Network Application Integration/CoDesign10.1145/3405672.3405807(35-40)Online publication date: 14-Aug-2020
https://dl.acm.org/doi/10.1145/3405672.3405807
Vaucher SYazdani NFelber PLucani DSchiavoni VHan DFeldmann A(2020)ZipLineProceedings of the 16th International Conference on emerging Networking EXperiments and Technologies10.1145/3386367.3431302(399-405)Online publication date: 23-Nov-2020
https://dl.acm.org/doi/10.1145/3386367.3431302
Fu YLiu XShu JShen ZZhang SWu JDuan JMa L(2020)Device and Placement Aware Framework to optimize Single Failure Recoveries and Reads for Erasure Coded Storage System with Heterogeneous Storage Devices2020 International Symposium on Reliable Distributed Systems (SRDS)10.1109/SRDS51746.2020.00030(225-235)Online publication date: Sep-2020
https://doi.org/10.1109/SRDS51746.2020.00030

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents