research-article

Free access

ClickNP: Highly Flexible and High Performance Network Processing with Reconfigurable Hardware

Authors:

Layong (Larry) Luo,

Yongqiang Xiong,

Enhong ChenAuthors Info & Claims

SIGCOMM '16: Proceedings of the 2016 ACM SIGCOMM Conference

Pages 1 - 14

https://doi.org/10.1145/2934872.2934897

Published: 22 August 2016 Publication History

Abstract

Highly flexible software network functions (NFs) are crucial components to enable multi-tenancy in the clouds. However, software packet processing on a commodity server has limited capacity and induces high latency. While software NFs could scale out using more servers, doing so adds significant cost. This paper focuses on accelerating NFs with programmable hardware, i.e., FPGA, which is now a mature technology and inexpensive for datacenters. However, FPGA is predominately programmed using low-level hardware description languages (HDLs), which are hard to code and difficult to debug. More importantly, HDLs are almost inaccessible for most software programmers. This paper presents ClickNP, a FPGA-accelerated platform for highly flexible and high-performance NFs with commodity servers. ClickNP is highly flexible as it is completely programmable using high-level C-like languages, and exposes a modular programming abstraction that resembles Click Modular Router. ClickNP is also high performance. Our prototype NFs show that they can process traffic at up to 200 million packets per second with ultra-low latency ($< 2\mu$s). Compared to existing software counterparts, with FPGA, ClickNP improves throughput by 10x, while reducing latency by 10x. To the best of our knowledge, ClickNP is the first FPGA-accelerated platform for NFs, written completely in high-level language and achieving 40 Gbps line rate at any packet size.

Supplementary Material

MP4 File (p1.mp4)

Download
213.75 MB

References

[1]

Altera SDK for OpenCL. http://www.altera.com/.

[2]

Cavium Networks OCTEON II processors. http://www.caviumnetworks.com.

[3]

Dell networking s6000 spec sheet.

[4]

Linux virtual server. http://www.linuxvirtualserver.org/.

[5]

Netronome Flow Processor NFP-6xxx. https://netronome.com/product/nfp-6xxx/.

[6]

SDAccel Development Environment. http://www.xilinx.com/.

[7]

Strongswan ipsec-based vpn. https://www.strongswan.org/.

[8]

The OpenCL Specifications ver 2.1. Khronos Group.

[9]

Vivado Design Suite. http://www.xilinx.com/.

[10]

Ethernet switch series, 2013. Broadcom Trident II.

[11]

Introducing EDR 100GB/s - Enabling the Use of Data, 2014. Mellanox White Paper.

[12]

M. Alizadeh, S. Yang, M. Sharif, S. Katti, N. McKeown, B. Prabhakar, and S. Shenker. pfabric: Minimal near-optimal datacenter transport. In Proc. ACM SIGCOMM, 2013.

Digital Library

[13]

J. Auerbach, D. F. Bacon, P. Cheng, and R. Rabbah. Lime: a java-compatible and synthesizable language for heterogeneous architectures. In ACM SIGPLAN Notices, volume 45, pages 89–108. ACM, 2010.

Digital Library

[14]

J. Bachrach, H. Vo, B. Richards, Y. Lee, A. Waterman, R. Avižienis, J. Wawrzynek, and K. Asanović. Chisel: constructing hardware in a scala embedded language. In Proc. ACM Annual Design Automation Conf., 2012.

Digital Library

[15]

D. F. Bacon, R. Rabbah, and S. Shukla. Fpga programming for the masses. Communications of the ACM, 56(4):56–63, 2013.

Digital Library

[16]

W. Bai, L. Chen, K. Chen, and H. Wu. Enabling ecn in multi-service multi-queue data centers. In Proc. USENIX NSDI, 2016.

Digital Library

[17]

T. Barbette, C. Soldani, and L. Mathy. Fast userspace packet processing. In Proc. ANCS, 2015.

Digital Library

[18]

A. Bernstein. Analysis of programs for parallel processing. IEEE Transactions on Electronic Computers, EC-15(5):757–763, Oct 1966.

[19]

B. Betkaoui, D. B. Thomas, and W. Luk. Comparing performance and energy efficiency of fpgas and gpus for high productivity computing. In 2010 International Conference on Field-Programmable Technology, 2010.

[20]

M. Dobrescu, N. Egi, K. Argyraki, B.-G. Chun, K. Fall, G. Iannaccone, A. Knies, M. Manesh, and S. Ratnasamy. Routebricks: Exploiting parallelism to scale software routers. In Proc. ACM SOSP, 2009.

Digital Library

[21]

N. Egi, A. Greenhalgh, M. Handley, M. Hoerdt, F. Huici, and L. Mathy. Towards high performance virtual routers on commodity hardware. In Proc. ACM CoNEXT, 2008.

Digital Library

[22]

R. Gandhi, H. H. Liu, Y. C. Hu, G. Lu, J. Padhye, L. Yuan, and M. Zhang. Duet: Cloud scale load balancing with hardware and software. In Proc. ACM SIGCOMM, 2014.

Digital Library

[23]

A. Greenberg. Windows Azure: Scaling SDN in Public Cloud, 2014. OpenNet Submit.

[24]

A. Greenberg. SDN for the Cloud, 2015. Keynote at SIGCOMM 2015 (https://azure.microsoft.com/en-us/blog/microsoft-showcases-software-defined-networking-innovation-at-sigcomm-v2/).

[25]

A. Greenhalgh, F. Huici, M. Hoerdt, P. Papadimitriou, M. Handley, and L. Mathy. Flow processing and the rise of commodity network hardware. ACM SIGCOMM CCR, 39(2):20–26, Mar. 2009.

Digital Library

[26]

S. Han, K. Jang, K. Park, and S. Moon. Packetshader: A gpu-accelerated software router. In Proc. ACM SIGCOMM, 2010.

Digital Library

[27]

W. Jiang. Scalable ternary content addressable memory implementation using fpgas. In Proc. ANCS, 2013.

Digital Library

[28]

S. Kestur, J. D. Davis, and O. Williams. Blas comparison on fpga, cpu and gpu. In IEEE Computer Society Symposium on VLSI, July 2010.

Digital Library

[29]

E. Kohler, R. Morris, B. Chen, J. Jannotti, and M. F. Kaashoek. The click modular router. ACM Transactions on Computer Systems (TOCS), 18(3):263–297, 2000.

Digital Library

[30]

T. Koponen, K. Amidon, P. Balland, M. Casado, A. Chanda, B. Fulton, I. Ganichev, J. Gross, N. Gude, P. Ingram, et al. Network virtualization in multi-tenant datacenters. In Proc. USENIX NSDI, Berkeley, CA, USA, 2014.

Digital Library

[31]

M. Lavasani, L. Dennison, and D. Chiou. Compiling high throughput network processors. In Proc. FPGA, 2012.

Digital Library

[32]

J. Lee, S. Lee, J. Lee, Y. Yi, and K. Park. Flosis: a highly scalable network flow capture system for fast retrieval and storage efficiency. In Proc. USENIX ATC, 2015.

Digital Library

[33]

J. Martins, M. Ahmed, C. Raiciu, V. Olteanu, M. Honda, R. Bifulco, and F. Huici. Clickos and the art of network function virtualization. In Proc. USENIX NSDI, 2014.

Digital Library

[34]

N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson, J. Rexford, S. Shenker, and J. Turner. Openflow: enabling innovation in campus networks. ACM SIGCOMM CCR, 38(2):69–74, 2008.

Digital Library

[35]

S.-W. Moon, J. Rexford, and K. G. Shin. Scalable hardware priority queue architectures for high-speed packet switches. IEEE Transactions on Computers, 2000.

Digital Library

[36]

J. Naous, G. Gibb, S. Bolouki, and N. McKeown. Netfpga: Reusable router architecture for experimental research. In Proc. PRESTO, 2008.

Digital Library

[37]

R. S. Nikhil and Arvind. What is bluespec? ACM SIGDA Newsletter, 39(1):1–1, Jan. 2009.

Digital Library

[38]

R. Pagh and F. F. Rodler. Cuckoo hashing. Algorithms - ESA 2001. Lecture Notes in Computer Science 2161, 2001.

Digital Library

[39]

P. Patel, D. Bansal, L. Yuan, A. Murthy, A. Greenberg, D. A. Maltz, R. Kern, H. Kumar, M. Zikos, H. Wu, C. Kim, and N. Karri. Ananta: Cloud scale load balancing. In Proc. ACM SIGCOMM, 2013.

Digital Library

[40]

A. Putnam, A. M. Caulfield, E. S. Chung, D. Chiou, K. Constantinides, J. Demme, H. Esmaeilzadeh, J. Fowers, G. P. Gopal, J. Gray, et al. A reconfigurable fabric for accelerating large-scale datacenter services. In Proc. Intl. Symp. on Computer Architecture (ISCA), 2014.

Digital Library

[41]

T. Rinta-aho, M. Karlstedt, and M. P. Desai. The click2netfpga toolchain. In Proc. USENIX ATC, 2012.

Digital Library

[42]

E. Rubow, R. McGeer, J. Mogul, and A. Vahdat. Chimpp: A click-based programming and simulation environment for reconfigurable networking hardware. In Proc. ANCS, 2010.

Digital Library

[43]

V. Sekar, N. Egi, S. Ratnasamy, M. K. Reiter, and G. Shi. Design and implementation of a consolidated middlebox architecture. In Proc. USENIX NSDI, 2012.

Digital Library

[44]

J. Sherry, P. Gao, S. Basu, A. Panda, A. Krishnamurthy, C. Macciocco, M. Manesh, J. Martins, S. Ratnasamy, L. Rizzo, and S. Shenker. Rollback recovery for middleboxes. In Proc. ACM SIGCOMM, 2015.

Digital Library

[45]

D. Singh. Implementing fpga design with the opencl standard. Altera whitepaper, 2011.

[46]

R. Wester. A transformation-based approach to hardware design using higher-order functions. 2015.

Cited By

Kumar ATanksale AChowfin AAjjampudi MMishra AKhan ASaha VNaik PVutukuru M(2024)Pyramis: Domain Specific Language for Developing Multi-tier SystemsProceedings of the 8th Asia-Pacific Workshop on Networking10.1145/3663408.3663431(156-162)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.1145/3663408.3663431
Lin WShan YKosta RKrishnamurthy AZhang YZhang ZPutnam A(2024)SuperNIC: An FPGA-Based, Cloud-Oriented SmartNICProceedings of the 2024 ACM/SIGDA International Symposium on Field Programmable Gate Arrays10.1145/3626202.3637564(130-141)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1145/3626202.3637564
Ma ZLi TXu ZFonseca NZhu Z(2024)SFCache: Hybrid NF Synthesization in Runtime With Rule-Caching in Programmable SwitchesIEEE Transactions on Network and Service Management10.1109/TNSM.2024.339014021:4(4613-4624)Online publication date: Aug-2024
https://doi.org/10.1109/TNSM.2024.3390140
Show More Cited By

Index Terms

ClickNP: Highly Flexible and High Performance Network Processing with Reconfigurable Hardware
1. Hardware
  1. Electronic design automation
    1. High-level and register-transfer level synthesis
      1. Hardware-software codesign
2. Networks
  1. Network components
    1. Middle boxes / network appliances
  2. Network types
    1. Data center networks

Recommendations

Acceleration of Image Processing Algorithms Using Minimal Resources of Custom Reconfigurable Hardware
PCI '12: Proceedings of the 2012 16th Panhellenic Conference on Informatics

The hardware/software implementation of a custom vision board using minimal resources out of a reconfigurable platform is described. Demanding robotic vision applications in most cases require dedicated hardware for reliable operation. The designed ...
Implementing an OFDM Receiver on the RaPiD Reconfigurable Architecture

Field-programmable gate arrays (FPGAs) have become an extremely popular implementation technology for custom hardware because they offer a combination of low cost and very fast turnaround. Because of their in-system reconfigurability, FPGAs have also ...
Self-Reconfigurable Embedded Systems on Low-Cost FPGAs

Hardware acceleration significantly increases the performance of embedded systems built on programmable logic. Allowing a FPGA-based MicroBlaze processor to self-select the coprocessors it uses can help reduce area requirements and increase a system's ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGCOMM '16: Proceedings of the 2016 ACM SIGCOMM Conference

August 2016

645 pages

ISBN:9781450341936

DOI:10.1145/2934872

General Chairs:
Marinho Barcellos
UFRGS
,
Jon Crowcroft
University of Cambridge
,
Program Chairs:
Amin Vahdat
Google
,
Sachin Katti
Stanford University

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 August 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGCOMM '16

Sponsor:

SIGCOMM

SIGCOMM '16: ACM SIGCOMM 2016 Conference

August 22 - 26, 2016

Florianopolis, Brazil

Acceptance Rates

SIGCOMM '16 Paper Acceptance Rate 39 of 231 submissions, 17%;

Overall Acceptance Rate 462 of 3,389 submissions, 14%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

201
Total Citations
View Citations
5,133
Total Downloads

Downloads (Last 12 months)402
Downloads (Last 6 weeks)52

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kumar ATanksale AChowfin AAjjampudi MMishra AKhan ASaha VNaik PVutukuru M(2024)Pyramis: Domain Specific Language for Developing Multi-tier SystemsProceedings of the 8th Asia-Pacific Workshop on Networking10.1145/3663408.3663431(156-162)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.1145/3663408.3663431
Lin WShan YKosta RKrishnamurthy AZhang YZhang ZPutnam A(2024)SuperNIC: An FPGA-Based, Cloud-Oriented SmartNICProceedings of the 2024 ACM/SIGDA International Symposium on Field Programmable Gate Arrays10.1145/3626202.3637564(130-141)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1145/3626202.3637564
Ma ZLi TXu ZFonseca NZhu Z(2024)SFCache: Hybrid NF Synthesization in Runtime With Rule-Caching in Programmable SwitchesIEEE Transactions on Network and Service Management10.1109/TNSM.2024.339014021:4(4613-4624)Online publication date: Aug-2024
https://doi.org/10.1109/TNSM.2024.3390140
Huang HTian JMin GYin HZeng CZhao YWu D(2024)Parallel Placement of Virtualized Network Functions via Federated Deep Reinforcement LearningIEEE/ACM Transactions on Networking10.1109/TNET.2024.336695032:4(2936-2949)Online publication date: Aug-2024
https://doi.org/10.1109/TNET.2024.3366950
Xiao JZuo XLi QZhao DZhao HJiang YSun JChen BLiang YLi J(2024)FlexNF: Flexible Network Function Orchestration for Scalable On-Path Service Chain ServingIEEE/ACM Transactions on Networking10.1109/TNET.2023.333423732:3(2026-2041)Online publication date: Jun-2024
https://doi.org/10.1109/TNET.2023.3334237
Pacífico RDuarte LVieira LRaghavan BNacif JVieira M(2024)eBPFlow: A Hardware/Software Platform to Seamlessly Offload Network Functions Leveraging eBPFIEEE/ACM Transactions on Networking10.1109/TNET.2023.331825132:2(1319-1332)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1109/TNET.2023.3318251
Carvalho FFerreira RCunha ÍVieira MRamanathan M(2024)State Disaggregation for Dynamic Scaling of Network FunctionsIEEE/ACM Transactions on Networking10.1109/TNET.2023.328256232:1(81-95)Online publication date: Feb-2024
https://doi.org/10.1109/TNET.2023.3282562
Liu JDragojević AFleming SKatsarakis AKorolija DZablotchi INg HKalia ACastro M(2024)Honeycomb: Ordered Key-Value Store Acceleration on an FPGA-Based SmartNICIEEE Transactions on Computers10.1109/TC.2023.334517373:3(857-871)Online publication date: Mar-2024
https://doi.org/10.1109/TC.2023.3345173
Choudhury SDas SPaul SMaddala PSeskar IRaychaudhuri D(2024)MEC-Intelligent Agent Support for Low-Latency Data Plane in Private NextG CoreICC 2024 - IEEE International Conference on Communications10.1109/ICC51166.2024.10622989(4942-4947)Online publication date: 9-Jun-2024
https://doi.org/10.1109/ICC51166.2024.10622989
Natori KOtani IHarasawa HSaito SFujimoto K(2024)Power Saving for Hardware Accelerated Applications With Dynamical Processor SwitchingIEEE Access10.1109/ACCESS.2024.344843212(118109-118121)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3448432
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents