tutorial

Fast Packet Processing with eBPF and XDP: Concepts, Code, Challenges, and Applications

Authors:

Marcos A. M. Vieira,

Matheus S. Castanho,

Racyus D. G. Pacífico,

Elerson R. S. Santos,

Eduardo P. M. Câmara Júnior,

Luiz F. M. VieiraAuthors Info & Claims

ACM Computing Surveys (CSUR), Volume 53, Issue 1

Article No.: 16, Pages 1 - 36

https://doi.org/10.1145/3371038

Published: 06 February 2020 Publication History

Abstract

Extended Berkeley Packet Filter (eBPF) is an instruction set and an execution environment inside the Linux kernel. It enables modification, interaction, and kernel programmability at runtime. eBPF can be used to program the eXpress Data Path (XDP), a kernel network layer that processes packets closer to the NIC for fast packet processing. Developers can write programs in C or P4 languages and then compile to eBPF instructions, which can be processed by the kernel or by programmable devices (e.g., SmartNICs). Since its introduction in 2014, eBPF has been rapidly adopted by major companies such as Facebook, Cloudflare, and Netronome. Use cases include network monitoring, network traffic manipulation, load balancing, and system profiling. This work aims to present eBPF to an inexpert audience, covering the main theoretical and fundamental aspects of eBPF and XDP, as well as introducing the reader to simple examples to give insight into the general operation and use of both technologies.

References

[1]

2019. bpf: Increase Complexity Limit and Maximum Program Size. Retrieved from https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git/commit/?id=c04

[2]

2019. libbpf Unification and Golang Bindings. Discussion Summary, Linux Kernel Developers’ bpfconf 2019. Retrieved from http://vger.kernel.org/bpfconf2019.html#session-4.

[3]

2019. XDP Project Repository. Retrieved from https://github.com/xdp-project/xdp-project.

[4]

Ahmed Abdelsalam, Francois Clad, Clarence Filsfils, Stefano Salsano, Giuseppe Siracusano, and Luca Veltri. 2017. Implementation of virtual network function chaining through segment routing in a linux-based NFV infrastructure. In Proceedings of the 2017 IEEE Conference on Network Softwarization: Softwarization Sustaining a Hyper-Connected World: en Route to 5G (NetSoft’17). IEEE, Los Alamitos, CA, 1--5. arXiv:1702.05157

[5]

Zaafar Ahmed, Muhammad Hamad Alizai, and Affan A. Syed. 2018. InKeV: In-kernel distributed network virtualization for DCN. SIGCOMM Comput. Commun. Rev. 46, 3, Article 4 (Jul. 2018), 6 pages.

Digital Library

[6]

S. Baidya, Y. Chen, and M. Levorato. 2018. eBPF-based content and computation-aware communication for real-time edge computing. In Proceedings of the INFOCOM IEEE Conference on Computer Communications Workshops (INFOCOM WORKSHOPS’18). IEEE, Los Alamitos, CA, 865--870.

[7]

BCC. 2019. BPF Compiler Collection. Retrieved from https://github.com/iovisor/bcc.

[8]

BCC. 2019. BPF Program Types. Retrieved from https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md#program-types.

[9]

BCC. 2019. XDP Compatible Drivers. Retrieved from https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md#xdp.

[10]

David Beckett, Jaco Joubert, and Simon Horman. 2018. Host dataplane acceleration (HDA). In ACM SIGCOMM 2018 Tutorials (SIGCOMM'18). ACM, New York, NY.

[11]

Gilberto Bertin. 2017. XDP in practice: Integrating XDP into our DDoS mitigation pipeline. In Proceedings of the Netdev 2.1 Technical Conference on Linux Networking. 1--5.

[12]

Matteo Bertrone, Sebastiano Miano, Fulvio Risso, and Massimo Tumolo. 2018. Accelerating linux security with eBPF iptables. In Proceedings of the ACM SIGCOMM 2018 Conference on Posters and Demos (SIGCOMM’18). ACM, New York, NY, 108--110.

Digital Library

[13]

Daniel Borkmann. 2019. bpf, Libbpf: Support Global Data/bss/rodata Sections. Retrieved from https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d859900c4c56.

[14]

Pat Bosshart, Dan Daly, Glen Gibb, Martin Izzard, Nick McKeown, Jennifer Rexford, Cole Schlesinger, Dan Talayco, Amin Vahdat, George Varghese, and David Walker. 2014. P4: Programming protocol-independent packet processors. SIGCOMM Comput. Commun. Rev. 44, 3 (Jul. 2014), 87--95.

Digital Library

[15]

Autores bpftool. 2018. Manual Bpftool. Retrieved from https://elixir.bootlin.com/linux/v4.18-rc1/ source/tools/bpf/bpftool/Documentation/bpftool.rst.

[16]

bpftrace. 2019. High-level Tracing Language for Linux eBPF. Retrieved from https://github.com/iovisor/bpftrace.

[17]

Mihai Budiu. 2015. Compiling p4 to ebpf. Retrieved from https://github.com/iovisor/bcc/tree/master/src/cc/frontends/p4.

[18]

Cilium. 2018. Cilium 1.0: Bringing the BPF Revolution to Kubernetes Networking and Security. Retrieved from https://cilium.io/blog/2018/04/24/cilium-10/.

[19]

Cilium. 2019. BPF and XDP Reference Guide. Retrieved September 9, 2019 from https://cilium.readthedocs.io/en/latest/bpf/.

[20]

Cilium. 2019. Cilium: API-aware Networking and Security. Retrieved September 9, 2019 from https://cilium.io/.

[21]

Jonathan Corbet. 2014. BPF: The Universal In-kernel Virtual Machine. Retrieved from https://lwn.net/Articles/599755/.

[22]

DPDK. 2019. AF_XDP Poll Mode Driver. Retrieved from https://doc.dpdk.org/guides/nics/af_xdp.html.

[23]

DPDK. 2019. Berkeley Packet Filter Library. Retrieved from https://doc.dpdk.org/guides/prog_guide/bpf_lib.html.

[24]

Fabien Duchene, Mathieu Jadin, and Olivier Bonaventure. 2018. Exploring various use cases for IPv6 segment routing. In Proceedings of the ACM SIGCOMM 2018 Conference on Posters and Demos (SIGCOMM’18). ACM, New York, NY, 129--131.

Digital Library

[25]

Eric Dumazet. 2011. A JIT for Packet Filters. Retrieved from https://lwn.net/Articles/437981/.

[26]

Facebook. 2018. Katran Source Code Repository. Retrieved from https://github.com/facebookincubator/katran.

[27]

John Fastabend. 2018. [RFC PATCH 00/16] bpf, Bounded Loop Support Work in Progress. Retrieved from https://lwn.net/ml/netdev/20180601092646.15353.28269.stgit@john-Precision-Tower-5810/.

[28]

Nick Feamster, Jennifer Rexford, and Ellen Zegura. 2014. The road to SDN: An intellectual history of programmable networks. ACM SIGCOMM Comput. Commun. Rev. 44, 2 (2014), 87--98.

Digital Library

[29]

C. Filsfils, S. Previdi, L. Ginsberg, B. Decraene, S. Litkowski, and R. Shakir. 2018. Segment Routing Architecture. RFC 8402. RFC Editor.

[30]

gobpf. 2019. Go Bindings for Creating BPF Programs. Retrieved from https://github.com/iovisor/gobpf.

[31]

Toke Høiland-Jørgensen, Jesper Dangaard Brouer, Daniel Borkmann, John Fastabend, Tom Herbert, David Ahern, and David Miller. 2018. The eXpress data path: Fast programmable packet processing in the operating system kernel. In Proceedings of the 14th International Conference on Emerging Networking EXperiments and Technologies (CoNEXT’18). ACM, New York, NY, 54--66.

Digital Library

[32]

IOvisor. 2019. Iovisor Project. Retrieved March 29, 2019 from www.iovisor.org.

[33]

S. Jouet, R. Cziva, and D. P. Pezaros. 2015. Arbitrary packet matching in OpenFlow. In Proceedings of the 16th International Conference on High Performance Switching and Routing (HPSR’15). IEEE, Los Alamitos, CA, 1--6.

[34]

Simon Jouet and Dimitrios P. Pezaros. 2017. BPFabric: Data plane programmability for software defined networks. In Proceedings of the Symposium on Architectures for Networking and Communications Systems (ANCS’17). IEEE Press, Piscataway, NJ, 38--48.

[35]

Michael Kerrisk. 2013. BFPC 8 Linux Manual Page. Retrieved June 8, 2019 from http://man7.org/linux/man-pages/man8/bpfc.8.html.

[36]

Jason Koch, Martin Spier, Brendan Gregg, and Ed Hunter. 2019. Extending Vector with eBPF to Inspect Host and Container Performance. Retrieved from https://medium.com/netflix-techblog/extending-vector-with-ebpf-to-inspect-host-and-container-performance-5da3af4c584b.

[37]

Eddie Kohler, Robert Morris, Benjie Chen, John Jannotti, and M. Frans Kaashoek. 2000. The click modular router. ACM Trans. Comput. Syst. 18, 3 (2000), 263--297.

Digital Library

[38]

Bojie Li, Kun Tan, Layong (Larry) Luo, Yanqing Peng, Renqian Luo, Ningyi Xu, Yongqiang Xiong, Peng Cheng, and Enhong Chen. 2016. ClickNP: Highly flexible and high performance network processing with reconfigurable hardware. In Proceedings of the 2016 ACM SIGCOMM Conference (SIGCOMM’16). ACM, New York, NY, 1--14.

Digital Library

[39]

libbpf. 2018. Libbpf Source Code. Retrieved from https://elixir.bootlin.com/linux/v4. 18-rc1/source/tools/lib/bpf.

[40]

libbpf. 2019. Stand-alone Libbpf. Retrieved from https://github.com/libbpf/libbpf.

[41]

Linux. 2017. Net: Xdp: Support Xdp Generic on Virtual Devices. Retrieved from https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d445516966dcb2924741b13b27738b54df2af01a.

[42]

Linux Foundation. 2015. Data Plane Development Kit (DPDK). Retrieved from http://www.dpdk.org.

[43]

D. F. Macedo, D. Guedes, L. F. M. Vieira, M. A. M. Vieira, and M. Nogueira. 2015. Programmable networks: From software-defined radio to software-defined networking. IEEE Commun. Surv. Tutor. 17, 2 (2015), 1102--1125.

Digital Library

[44]

Alan Maguire. 2019. Notes on BPF (1)—A Tour of Program Types. Retrieved from https://blogs.oracle.com/linux/notes-on-bpf-1.

[45]

Marek Majkowski. 2019. Cloudflare Architecture and How BPF Eats the World. Retrieved from https://blog.cloudflare.com/cloudflare-architecture-and-how-bpf-eats-the-world/.

[46]

Steven McCanne and Van Jacobson. 1993. The BSD packet filter: A new architecture for user-level packet capture. In Proceedings of the USENIX Winter 1993 Conference Proceedings on USENIX Winter 1993 Conference Proceedings (USENIX’93). USENIX Association, Berkeley, CA, 1--11.

Digital Library

[47]

Sebastiano Miano, Matteo Bertrone, Fulvio Risso, Massimo Tumolo, and Mauricio Vásquez Bernal. 2018. Creating complex network service with ebpf: Experience and lessons learned. In Proceedings of the High Performance Switching and Routing (HPSR’18). IEEE, Los Alamitos, CA, 1--8.

[48]

Rashid Mijumbi, Joan Serrat, Juan Luis Gorricho, Niels Bouten, Filip De Turck, and Raouf Boutaba. 2016. Network function virtualization: State-of-the-art and research challenges. IEEE Communi. Surv. Tutor. 18, 1 (2016), 236--262.

Digital Library

[49]

David Miller. 2017. BPF Verifier Overview. Retrieved April 9, 2019 from https://lwn.net/Articles/794934/.

[50]

J. Mogul, R. Rashid, and M. Accetta. 1987. The packer filter: An efficient mechanism for user-level network code. In Proceedings of the 11th ACM Symposium on Operating Systems Principles (SOSP’87). ACM, New York, NY, 39--51.

[51]

Quentin Monnet. 2019. All-Out Programmability in Linux: An Introduction to BPF as a Monitoring Tool. Retrieved April 9, 2019 from https://qmo.fr/docs/talk_20190516_allout_programmability_bpf.pdf.

[52]

Quentin Monnet. 2019. Rust Virtual Machine and JIT Compiler for eBPF Programs. Retrieved from https://github.com/qmonnet/rbpf.

[53]

Netronome. 2019. Sample BPF Offload Apps. Retrieved from https://github.com/Netronome/bpf-samples.

[54]

PLUMgrid. 2016. Linux Kernel Source Code. Retrieved June 7, 2019 from https://github.com/torvalds/linux/blob/v5.3/samples/bpf/xdp1_kern.c.

[55]

PLUMgrid. 2016. Linux Kernel Source Code. Retrieved June 7, 2019 from https://github.com/torvalds/linux/blob/v5.3/samples/bpf/xdp1_user.c.

[56]

ply. 2019. Dynamic Tracing in Linux. Retrieved from https://github.com/iovisor/ply.

[57]

Alex Pollitt. 2019. Tigera adds eBPF Support to Calico. Retrieved September 9, 2019 from https://www.projectcalico.org/tigera-adds-ebpf-support-to-calico/.

[58]

Luigi Rizzo. 2012. netmap: A novel framework for fast packet I/O. In Proceedings of the 2012 USENIX Annual Technical Conference (USENIX ATC’12). USENIX Association, Berkeley, CA, 101--112. https://www.usenix.org/conference/atc12/technical-sessions/presentation/rizzo

[59]

Marta Rybczyńska. 2019. Bounded Loops in BPF for the 5.3 Kernel. Retrieved April 9, 2019 from https://www.spinics.net/lists/xdp-newbies/msg00185.html.

[60]

Jay Schulist, Daniel Borkmann, and Alexei Starovoitov. 2019. Linux Socket Filtering aka Berkeley Packet Filter (BPF). Retrieved March 17, 2019 from www.kernel.org/doc/Documentation/networking/filter.txt.

[61]

Haoyu Song. 2013. Protocol-oblivious forwarding: Unleash the power of SDN through a future-proof forwarding plane. In Proceedings of the Second ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking (HotSDN’13). ACM, New York, NY, 127--132.

Digital Library

[62]

Alexei Starovoitov. 2015. bpf: Introduce bpf_tail_call() Helper. Retrieved from https://lwn.net/Articles/645169/.

[63]

Alexei Starovoitov. 2018. Lifetime of BPF Objects. Retrieved from https://facebookmicrosites.github.io/bpf/blog/2018/08/31/object-lifetime.html.

[64]

Cheng-Chun Tu, Joe Stringer, and Justin Pettit. 2017. Building an extensible open vSwitch datapath. SIGOPS Oper. Syst. Rev. 51, 1 (Sep. 2017), 72--77.

Digital Library

[65]

uBPF. 2019. Userspace eBPF VM. Retrieved from https://github.com/iovisor/ubpf.

[66]

Marcos A. M. Vieira, Matheus S. Castanho, Racyus D. G. Pacífico, Elerson R. S. Santos, Eduardo P. M. Câmara Júnior, and Luiz F. M. Vieira. 2019. Zenodo—eBPF Tutorial. Retrieved from https://zenodo.org/record/3519347#.XbMxR6zMNhE.

[67]

Marcos A. M. Vieira, Matheus S. Castanho, Racyus D. G. Pacífico, Elerson R. S. Santos, Eduardo P. M. Câmara Júnior, and Luiz F. M. Vieira. 2019. eBPF Tutorial. Retrieved from https://github.com/racyusdelanoo/bpf-tutorial.

[68]

VMWare. 2018. p4c-xdp. Retrieved from https://github.com/vmware/p4c-xdp.

[69]

WeaveWorks. 2017. Improving Performance and Reliability in Weave Scope with eBPF. Retrieved from https://www.weave.works/blog/improving-performance-reliability-weave-scope-ebpf/.

[70]

XDP-Project. 2019. AXDP Hands-On Tutorial. Retrieved August 20, 2019 from https://github.com/xdp-project/xdp-tutorial.

Cited By

Hadi HAdnan MCao YHussain FAhmad NAlshara MJaved Y(2024)iKern: Advanced Intrusion Detection and Prevention at the Kernel Level Using eBPFTechnologies10.3390/technologies1208012212:8(122)Online publication date: 30-Jul-2024
https://doi.org/10.3390/technologies12080122
Garcia JSundberg SBrunstrom A(2024)Fine-Grained Starlink Throughput Variation Examined With State-Transition Modeling2024 19th Wireless On-Demand Network Systems and Services Conference (WONS)10.23919/WONS60642.2024.10449629(69-76)Online publication date: 29-Jan-2024
https://doi.org/10.23919/WONS60642.2024.10449629
Mahadevan STakano YMiyaji A(2024)PRSafe: A Domain Specific Language Created with LLVMJournal of Information Processing10.2197/ipsjjip.32.20732(207-222)Online publication date: 2024
https://doi.org/10.2197/ipsjjip.32.207
Show More Cited By

Index Terms

Fast Packet Processing with eBPF and XDP: Concepts, Code, Challenges, and Applications
1. Networks
  1. Network architectures
    1. Programming interfaces
  2. Network components
    1. End nodes
    2. Middle boxes / network appliances

Recommendations

Fast In-kernel Traffic Sketching in eBPF

The extended Berkeley Packet Filter (eBPF) is an infrastructure that allows to dynamically load and run micro-programs directly in the Linux kernel without recompiling it.

In this work, we study how to develop high-performance network measurements in ...
Optimizing latency and CPU load in packet processing systems
Spects '15: Proceedings of the International Symposium on Performance Evaluation of Computer and Telecommunication Systems

High-speed network cards supporting 10 or 40GbE (Gigabit Ethernet) are available today. Software frameworks for high-speed packet reception and transmission were created to exhaust the performance of these cards. However, these frameworks are not ...
Low latency network traffic processing with commodity hardware
Spects '15: Proceedings of the International Symposium on Performance Evaluation of Computer and Telecommunication Systems

Packet processing on commodity hardware is a cost-efficient and flexible alternative to specialized networking hardware. In case of Linux, the classical QoS mechanisms (e.g DiffServ) assume that the outgoing link is the bottleneck. However, on commodity ...

Comments

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 53, Issue 1

January 2021

781 pages

ISSN:0360-0300

EISSN:1557-7341

DOI:10.1145/3382040

Editor:
Albert Zomaya
University of Sydney, Australia

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 February 2020

Accepted: 01 October 2019

Revised: 01 October 2019

Received: 01 June 2019

Published in CSUR Volume 53, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Tutorial
Survey
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

128
Total Citations
View Citations
5,116
Total Downloads

Downloads (Last 12 months)1,025
Downloads (Last 6 weeks)107

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hadi HAdnan MCao YHussain FAhmad NAlshara MJaved Y(2024)iKern: Advanced Intrusion Detection and Prevention at the Kernel Level Using eBPFTechnologies10.3390/technologies1208012212:8(122)Online publication date: 30-Jul-2024
https://doi.org/10.3390/technologies12080122
Garcia JSundberg SBrunstrom A(2024)Fine-Grained Starlink Throughput Variation Examined With State-Transition Modeling2024 19th Wireless On-Demand Network Systems and Services Conference (WONS)10.23919/WONS60642.2024.10449629(69-76)Online publication date: 29-Jan-2024
https://doi.org/10.23919/WONS60642.2024.10449629
Mahadevan STakano YMiyaji A(2024)PRSafe: A Domain Specific Language Created with LLVMJournal of Information Processing10.2197/ipsjjip.32.20732(207-222)Online publication date: 2024
https://doi.org/10.2197/ipsjjip.32.207
A A(2024)arkLB: High Performance eBPF-Based Load Balancer for MicroservicesSSRN Electronic Journal10.2139/ssrn.4680868Online publication date: 2024
https://doi.org/10.2139/ssrn.4680868
Parola FQi SNarappa ARamakrishnan KRisso F(2024)SURE: Secure Unikernels Make Serverless Computing Rapid and EfficientProceedings of the ACM Symposium on Cloud Computing10.1145/3698038.3698558(668-688)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3698038.3698558
Ribeiro GPedrosa LSignorello SRamos F(2024)Internet Architecture Evolution: Found in TranslationProceedings of the 23rd ACM Workshop on Hot Topics in Networks10.1145/3696348.3696876(300-307)Online publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1145/3696348.3696876
Cannatà RSun HDumitriu DHassanieh H(2024)Towards Seamless 5G Open-RAN Integration with WebAssemblyProceedings of the 23rd ACM Workshop on Hot Topics in Networks10.1145/3696348.3696864(121-131)Online publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1145/3696348.3696864
Gerhorst LHerzog HWägemann POtt MKapitza RHönig T(2024)VeriFence: Lightweight and Precise Spectre Defenses for Untrusted Linux Kernel ExtensionsProceedings of the 27th International Symposium on Research in Attacks, Intrusions and Defenses10.1145/3678890.3678907(644-659)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3678890.3678907
Osaki APoisson MMakino SShiiba RFukuda KOkoshi TNakazawa J(2024)Dynamic Fixed-point Values in eBPF: a Case for Fully In-kernel Anomaly DetectionProceedings of the Asian Internet Engineering Conference 202410.1145/3674213.3674219(46-54)Online publication date: 9-Aug-2024
https://dl.acm.org/doi/10.1145/3674213.3674219
Bardinelli JZhang YSu JHuang LParilla AJarvi RKulkarni SZhang W(2024)hyDNS: Acceleration of DNS Through Kernel Space ResolutionProceedings of the ACM SIGCOMM 2024 Workshop on eBPF and Kernel Extensions10.1145/3672197.3673439(58-64)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3672197.3673439
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents