Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.5555/2874988.2874994acmconferencesArticle/Chapter ViewAbstractPublication PagessummersimConference Proceedingsconference-collections
research-article

Optimizing latency and CPU load in packet processing systems

Published: 26 July 2015 Publication History

Abstract

High-speed network cards supporting 10 or 40GbE (Gigabit Ethernet) are available today. Software frameworks for high-speed packet reception and transmission were created to exhaust the performance of these cards. However, these frameworks are not applicable as general-purpose solution. Thus, it is necessary to revisit general purpose network IO software that was designed more than a decade ago. In standard Linux settings, connectivity between applications and physical networks happens via the New API (NAPI). This motivated us to investigate how underlying NIC drivers can be adapted to improve latency in combination with the Linux NAPI. Based on testbed measurements, we propose an optimized algorithm for the NIC driver to dynamically adapt the Interrupt Throttling Rate (ITR). We implemented the algorithm and evaluated it with latency and throughput measurements based on the Linux module of Open vSwitch that operates on top of the NAPI. Our measurements show that our new ITR algorithm improves the packet latency without affecting the CPU load as much as other solutions.

References

[1]
M. Dobrescu, N. Egi, K. Argyraki, B. Chun, K. Fall, G. Iannaccone, A. Knies, M. Manesh, and S. Ratnasamy, "RouteBricks: Exploiting Parallelism To Scale Software Routers," in 22nd ACM Symposium on Operating Systems Principles (SOSP), October 2009.
[2]
R. Bolla and R. Bruschi, "PC-based Software Routers: High Performance and Application Service Support," in ACM SIGCOMM Workshop on Programmable Routers for Extensible Services of Tomorrow (PRESTO), August 2008, pp. 27--32.
[3]
T. Meyer, F. Wohlfart, D. Raumer, B. E. Wolfinger, and G. Carle, "Validated Model-Based Performance Prediction of Multi-Core Software Routers," Praxis der Informationsverarbeitung und Kommunikation (PIK), vol. 37, no. 2, pp. 93--107, 2014.
[4]
"Intel Ethernet Controller XL710 Datasheet Rev. 2.1." Intel, December 2014.
[5]
L. Rizzo, "netmap: a novel framework for fast packet I/O," in USENIX Annual Technical Conference, April 2012.
[6]
F. Fusco and L. Deri, "High Speed Network Traffic Analysis with Commodity Multi-core Systems," in Internet Measurement Conference, November 2010, pp. 218--224.
[7]
"Data Plane Development Kit: Programmer's Guide, Revision 6." Intel Corporation, 2014.
[8]
J. H. Salim, R. Olsson, and A. Kuznetsov, "Beyond softnet," in Proceedings of the 5th annual Linux Showcase & Conference, vol. 5, 2001, pp. 18--18.
[9]
J. Mogul, D. Western, J. C. Mogul, and K. K. Ramakrishnan, "Eliminating receive livelock in an interrupt-driven kernel," ACM Transactions on Computer Systems, vol. 15, pp. 217--252, 1997.
[10]
R. Bolla and R. Bruschi, "Linux Software Router: Data Plane Optimization and Performance Evaluation," Journal of Networks, vol. 2, no. 3, pp. 6--17, June 2007.
[11]
A. Tedesco, G. Ventre, L. Angrisani, and L. Peluso, "Measurement of processing and queuing delays introduced by a software router in a single-hop network," in IEEE Instrumentation and Measurement Technology Conference 2005, 2005, pp. 1797--1802.
[12]
P. Emmerich, D. Raumer, F. Wohlfart, and G. Carle, "Performance Characteristics of Virtual Switching," in 2014 IEEE 3rd International Conference on Cloud Networking (CloudNet), Luxembourg, October 2014.
[13]
Intel, "Assigning Interrupts to Processor Cores using an Intel® 82575/82576 or 82598/82599 Ethernet Controller," http://www.intel.com/content/dam/doc/application-note/82575-82576-82598-82599-ethernet-controllers-interrupts-appl-note.pdf, 2009.
[14]
M. Wilcox, "I'll do it later: softirqs, tasklets, bottom halves, task queues, work queues and timers," in Proceedings of the 2003 Linux Conference Australia (LCA 2003), 2003.
[15]
"Intel 82599 10 GbE Controller Datasheet Rev. 2.76," Intel Corporation, 2012, Santa Clara, USA.
[16]
Intel, "Intel Server Adapters - Linux ixgbe Base Driver," http://www.intel.com/support/network/adapter/pro100/sb/CS-032530.htm, last visited 2015-01-27.
[17]
A. Beifuß, D. Raumer, P. Emmerich, T. M. Runge, F. Wohlfart, B. E. Wolfinger, and G. Carle, "A Study of Networking Software Induced Latency," in 2nd International Conference on Networked Systems (NetSys), March 2015.
[18]
P. Emmerich, S. Gallenmüller, D. Raumer, F. Wohlfart, and G. Carle, "MoonGen: A Scriptable High-Speed Packet Generator," ArXiv e-prints, Oct. 2014.
[19]
L. Erlacher, "Patch for ixgbe-3.14.5," https://github.com/duk3luk3/ixgbe-3.14.15/commit/c0a258a3d6fdd50f51e4231c946b13dc665eecf0.
[20]
L. Erlacher, "Patch for ixgbe-3.23.2," https://github.com/duk3luk3/ixgbe-3.23.2/commit/fcd21e9db103680d77bfd3a016eb3271b44d2a2e.
[21]
J. H. Salim, "When NAPI comes to Town," in UKUUG 2005 Linux Technical Conference, 2005.
[22]
K. Salah and A. Qahtan, "Implementation and experimental performance evaluation of a hybrid interrupt-handling scheme." Computer Communications, vol. 32, no. 1, pp. 179--188, 2009.
[23]
O. Maquelin, G. R. Gao, H. H. J. Humy, K. B. Theobald, and X. Tian, "Polling watchdog: Combining polling and interrupts for efficient message handling," in in Proceedings of the 23rd Annual International Symposium on Computer Architecture. ACM Press, 1995, pp. 179--188.
[24]
C. Dovrolis, B. Thayer, and P. Ramanathan, "Hip: Hybrid interrupt-polling for the network interface," SIGOPS Oper. Syst. Rev., vol. 35, no. 4, pp. 50--60, Oct. 2001.
[25]
X. Chang, J. K. Muppala, P. Zou, and X. Li, "A robust device hybrid scheme to improve system performance in gigabit ethernet networks." in LCN. IEEE Computer Society, 2007, pp. 444--454.
[26]
P. Emmerich, D. Raumer, F. Wohlfart, and G. Carle, "A Study of Network Stack Latency for Game Servers," in 13th Annual Workshop on Network and Systems Support for Games (NetGames'14), Nagoya, Japan, Dec. 2014.
[27]
J. Brandeburg, "A way towards Lower Latency and Jitter," Talk at the Linux Plumbers Conference, Slides available athttp://www.linuxplumbersconf.org/2012/wp-content/uploads/2012/09/2012-lpc-Low-Latency-Sockets-slides-brandeburg.pdf, 2012.
[28]
J. Corbet, "Low-latency Ethernet device polling," 2013.
[29]
L. Niccolini, G. Iannaccone, S. Ratnasamy, J. Chandrashekar, and L. Rizzo, "Building a power-proportional software router," in USENIX Annual Technical Conference (USENIX ATC 12). Boston, MA: USENIX, 2012, pp. 89--100.

Cited By

View all
  • (2021)Parallelizing packet processing in container overlay networksProceedings of the Sixteenth European Conference on Computer Systems10.1145/3447786.3456241(261-276)Online publication date: 21-Apr-2021
  • (2018)A Survey of End-System Optimizations for High-Speed NetworksACM Computing Surveys10.1145/318489951:3(1-36)Online publication date: 16-Jul-2018
  • (2016)Revisiting Benchmarking Methodology for Interconnect DevicesProceedings of the 2016 Applied Networking Research Workshop10.1145/2959424.2959430(55-61)Online publication date: 16-Jul-2016

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
Spects '15: Proceedings of the International Symposium on Performance Evaluation of Computer and Telecommunication Systems
July 2015
171 pages
ISBN:9781510810600

Sponsors

Publisher

Society for Computer Simulation International

San Diego, CA, United States

Publication History

Published: 26 July 2015

Check for updates

Author Tags

  1. ITR
  2. NAPI
  3. NIC driver
  4. commodity hardware
  5. linux
  6. packet latency
  7. packet processing

Qualifiers

  • Research-article

Conference

SummerSim '15
Sponsor:

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)8
  • Downloads (Last 6 weeks)2
Reflects downloads up to 24 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2021)Parallelizing packet processing in container overlay networksProceedings of the Sixteenth European Conference on Computer Systems10.1145/3447786.3456241(261-276)Online publication date: 21-Apr-2021
  • (2018)A Survey of End-System Optimizations for High-Speed NetworksACM Computing Surveys10.1145/318489951:3(1-36)Online publication date: 16-Jul-2018
  • (2016)Revisiting Benchmarking Methodology for Interconnect DevicesProceedings of the 2016 Applied Networking Research Workshop10.1145/2959424.2959430(55-61)Online publication date: 16-Jul-2016

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media