Article

Connection handoff policies for TCP offload network interfaces

Authors:

Hyong-youb Kim,

Scott RixnerAuthors Info & Claims

OSDI '06: Proceedings of the 7th symposium on Operating systems design and implementation

Pages 293 - 306

Published: 06 November 2006 Publication History

Abstract

This paper presents three policies for effectively utilizing TCP offload network interfaces that support connection handoff. These policies allow connection handoff to reduce the computation and memory bandwidth requirements for packet processing on the host processor without causing the resource constraints on the network interface to limit overall system performance. First, prioritizing packet processing on the network interface ensures that its TCP processing does not harm performance of the connections on the host operating system. Second, dynamically adapting the number of connections on the network interface to the current load avoids overloading the network interface. Third, the operating system can predict connection lifetimes to select long-lived connections for handoff to better utilize the network interface. The use of the first two policies improves web server throughput by 12--31% over the baseline throughput achieved without offload. The third policy helps improve performance when the network interface can only handle a small number of connections at a time. Furthermore, by using a faster offload processor, offloading can improve server throughput by 33--72%.

References

[1]

Alteon Networks. Tigon/PCI Ethernet Controller, Aug. 1997. Revision 1.04.

[2]

M. F. Arlitt and C. L. Williamson. Internet Web Servers: Workload Characterization and Performance Implications. IEEE/ACM Transactions on Networking, 5(5):631--645, Oct. 1997.

Digital Library

[3]

K. Banerjee, A. Bohra, S. Gopalakrishnan, M. Rangarajan, and L. Iftode. Split-OS: An Operating System Architecture for Clusters of Intelligent Devices. Work-in-Progress Session at the 18th Symposium on Operating Systems Principles, Oct. 2001.

[4]

G. Banga and P. Druschel. Measuring the Capacity of a Web Server. In Proceedings of the USENIX Symposium on Internet Technologies and Systems, Dec. 1997.

Digital Library

[5]

N. L. Binkert, E. G. Hallnor, and S. K. Reinhardt. Network-Oriented Full-System Simulation using M5. In Proceedings of the Sixth Workshop on Computer Architecture Evaluation using Commercial Workloads (CAECW), pages 36--43, Feb. 2003.

[6]

T. Brecht, G. J. Janakiraman, B. Lynn, V. Saletore, and Y. Turner. Evaluating Network Processing Efficiency with Processor Partitioning and Asynchronous I/O. In Proceedings of EuroSys 2006, pages 265--278, Apr. 2006.

Digital Library

[7]

L. Breslau, P. Cao, L. Fan, G. Phillips, and S. Schenker. Web Caching and Zipf-like Distributions: Evidence and Implications. In Proceedings of IEEE INFOCOM '99, volume 1, pages 126--134, Mar. 1999.

[8]

P. Cao and S. Irani. Cost-Aware WWW Proxy Caching Algorithms. In Proceedings of the 1997 USENIX Symposium on Internet Technology and Systems, pages 193--206, Dec. 1997.

Digital Library

[9]

H. K. J. Chu. Zero-Copy TCP in Solaris. In Proceedings of the 1996 Annual USENIX Technical Conference, pages 253--264, 1996.

Digital Library

[10]

D. D. Clark, V. Jacobson, J. Romkey, and H. Salwen. An Analysis of TCP Processing Overhead. IEEE Communications Magazine, pages 23--29, June 1989.

Digital Library

[11]

L. T. Clark, E. J. Hoffman, J. Miller, M. Biyani, Y. Liao, S. Strazdus, M. Morrow, K. E. Velarde, and M. A. Yarch. An Embedded 32-b Microprocessor Core for Low-Power and High-Performance Applications. IEEE Journal of Solid-State Circuits, 36(11):1599--1608, Nov. 2001.

[12]

P. Druschel and L. L. Peterson. Fbufs: A High-Bandwidth Cross-Domain Transfer Facility. In Proceedings of the 14th Symposium on Operating Systems Principles (SOSP-14), pages 189--202, Dec. 1993.

Digital Library

[13]

W. Feng, P. Balaji, C. Baron, L. N. Bhuyan, and D. K. Panda. Performance Characterization of a 10-Gigabit Ethernet TOE. In Proceedings of the 13th IEEE Symposium on High-Performance Interconnects, 2005.

Digital Library

[14]

D. Freimuth, E. Hu, J. LaVoie, R. Mraz, E. Nahum, P. Pradhan, and J. Tracey. Server Network Scalability and TCP Offload. In Proceedings of the 2005 Annual USENIX Technical Conference, pages 209--222, Apr. 2005.

Digital Library

[15]

Y. Hoskote, B. A. Bloechel, G. E. Dermer, V. Erraguntla, D. Finan, J. Howard, D. Klowden, S. Narendra, G. Ruhl, J. W. Tschanz, S. Vangal, V. Veeramachaneni, H. Wilson, J. Xu, and N. Borkar. A TCP Offload Accelerator for 10 Gb/s Ethernet in 90-nm CMOS. IEEE Journal of Solid-State Circuits, 38(11):1866--1875, Nov. 2003.

[16]

L. R. Hsu, A. G. Saidi, N. L. Binkert, and S. K. Reinhardt. Sampling and Stability in TCP/IP Workloads. In Proceedings of the First Annual Workshop on Modeling, Benchmarking, and Simulation (MoBS), pages 68--77, 2005.

[17]

H. Kim and S. Rixner. Performance Characterization of the FreeBSD Network Stack. Computer Science Department, Rice University, June 2005. Technical Report TR05--450.

[18]

H. Kim and S. Rixner. TCP Offload through Connection Handoff. In Proceedings of EuroSys 2006, pages 279--290, Apr. 2006.

Digital Library

[19]

K. Kleinpaste, P. Steenkiste, and B. Zill. Software Support for Outboard Buffering and Checksumming. In Proceedings of the ACM SIGCOMM '95 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, pages 87--98, Aug. 1995.

Digital Library

[20]

P. S. Magnusson, M. Christensson, J. Eskilson, D. Forsgren, G. Hállberg, J. Högberg, F. Larsson, A. Moestedt, and B. Werner. Simics: A Full System Simulation Platform. Computer, 35(2):50--58, 2002.

Digital Library

[21]

P. E. McKenney and K. F. Dove. Efficient Demultiplexing of Incoming TCP Packets. In Proceedings of the ACM SIGCOMM '92 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, pages 269--279, 1992.

Digital Library

[22]

Microsoft Corporation. Scalable Networking: Network Protocol Offload -- Introducing TCP Chimney, Apr. 2004. WinHEC Version.

[23]

J. Mogul, L. Brakmo, D. E. Lowell, D. Subhraveti, and J. Moore. Unveiling the Transport. ACM SIGCOMM Computer Communication Review, 34(1):99--106, 2004.

Digital Library

[24]

J. C. Mogul. TCP offload is a dumb idea whose time has come. In Proceedings of HotOS IX: The 9th Workshop on Hot Topics in Operating Systems, pages 25--30, 2003.

Digital Library

[25]

V. S. Pai, P. Druschel, and W. Zwaenepoel. Flash: An Efficient and Portable Web Server. In Proceedings of the USENIX 1999 Annual Technical Conference, pages 199--212, June 1999.

Digital Library

[26]

V. S. Pai, P. Druschel, and W. Zwaenepoel. I/O-Lite: A Unified I/O Buffering and Caching System. In Proceedings of the Third USENIX Symposium on Operating Systems Design and Implementation, pages 15--28, Feb. 1999.

Digital Library

[27]

M. Rangarajan, A. Bohra, K. Banerjee, E. V. Carrera, R. Bianchini, L. Iftode, and W. Zwaenepoel. TCP Servers: Offloading TCP/IP Processing in Internet Servers. Design, Implementation, and Performance. Computer Science Department, Rutgers University, Mar. 2002. Technical Report DCR-TR-481.

[28]

G. Regnier, S. Makineni, R. Illikkal, R. Iyer, D. Minturn, R. Huggahalli, D. Newell, L. Cline, and A. Foong. TCP Onloading for Data Center Servers. Computer, 37(11):48--58, Nov. 2004.

Digital Library

[29]

S. Rixner. Memory Controller Optimizations for Web Servers. In Proceedings of the 37th Annual International Symposium on Microarchitecture, pages 355--366, Dec. 2004.

Digital Library

[30]

P. Shivam and J. S. Chase. On the Elusive Benefits of Protocol Offload. In Proceedings of the ACM SIGCOMM Workshop on Network-I/O Convergence, pages 179--184, 2003.

Digital Library

[31]

R. Westrelin, N. Fugier, E. Nordmark, K. Kunze, and E. Lemoine. Studying Network Protocol Offload With Emulation: Approach And Preliminary Results. In Proceedings of the 12th Annual IEEE Symposium on High Performance Interconnects, pages 84--90, Aug. 2004.

Digital Library

Cited By

Yu YShin DShin WSong NChoi JKim HEom HYeom H(2014)Optimizing the Block I/O Subsystem for Fast Storage DevicesACM Transactions on Computer Systems10.1145/261909232:2(1-48)Online publication date: 1-Jun-2014
https://dl.acm.org/doi/10.1145/2619092
Solá-Sloan JCouvertier-Reyes IWainer GShaffer CMcGraw RChinni M(2009)A TCP offload engine emulator for estimating the impact of removing protocol processing from a host running Apache HTTP serverProceedings of the 2009 Spring Simulation Multiconference10.5555/1639809.1639889(1-8)Online publication date: 22-Mar-2009
https://dl.acm.org/doi/10.5555/1639809.1639889
Scheler FHofer WOechslein BPfister RSchröder-Preikschat WLohmann DHenkel JParameswaran S(2009)Parallel, hardware-supported interrupt handling in an event-triggered real-time operating systemProceedings of the 2009 international conference on Compilers, architecture, and synthesis for embedded systems10.1145/1629395.1629419(167-174)Online publication date: 11-Oct-2009
https://dl.acm.org/doi/10.1145/1629395.1629419
Show More Cited By

Connection handoff policies for TCP offload network interfaces
1. General and reference
  1. Cross-computing tools and techniques
2. Social and professional topics
  1. Professional topics
    1. Computing profession

Recommendations

Connection handoff policies for TCP offload network interfaces
OSDI '06: Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation - Volume 7

This paper presents three policies for effectively utilizing TCP offload network interfaces that support connection handoff. These policies allow connection handoff to reduce the computation and memory bandwidth requirements for packet processing on the ...
TCP offload through connection handoff
Proceedings of the 2006 EuroSys conference

This paper presents a connection handoff interface between the operating system and the network interface. Using this interface, the operating system can offload a subset of TCP connections in the system to the network interface, while the remaining ...
TCP offload through connection handoff
EuroSys '06: Proceedings of the 1st ACM SIGOPS/EuroSys European Conference on Computer Systems 2006

This paper presents a connection handoff interface between the operating system and the network interface. Using this interface, the operating system can offload a subset of TCP connections in the system to the network interface, while the remaining ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

OSDI '06: Proceedings of the 7th symposium on Operating systems design and implementation

November 2006

407 pages

ISBN:1931971471

Program Chairs:
Brian Bershad
University of Washington
,
Jeff Mogul
Hewlett-Packard Labs

Sponsors

VMware
NSF: National Science Foundation
Google Inc.
Infosys
SIGOPS: ACM Special Interest Group on Operating Systems
Sun Microsystems
Intel: Intel
Microsoft Research: Microsoft Research
DoCoMo USA Labs
USENIX Assoc: USENIX Assoc
HP invent
Ask.com
IBM: IBM

Publisher

USENIX Association

United States

Publication History

Published: 06 November 2006

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
185
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yu YShin DShin WSong NChoi JKim HEom HYeom H(2014)Optimizing the Block I/O Subsystem for Fast Storage DevicesACM Transactions on Computer Systems10.1145/261909232:2(1-48)Online publication date: 1-Jun-2014
https://dl.acm.org/doi/10.1145/2619092
Solá-Sloan JCouvertier-Reyes IWainer GShaffer CMcGraw RChinni M(2009)A TCP offload engine emulator for estimating the impact of removing protocol processing from a host running Apache HTTP serverProceedings of the 2009 Spring Simulation Multiconference10.5555/1639809.1639889(1-8)Online publication date: 22-Mar-2009
https://dl.acm.org/doi/10.5555/1639809.1639889
Scheler FHofer WOechslein BPfister RSchröder-Preikschat WLohmann DHenkel JParameswaran S(2009)Parallel, hardware-supported interrupt handling in an event-triggered real-time operating systemProceedings of the 2009 international conference on Compilers, architecture, and synthesis for embedded systems10.1145/1629395.1629419(167-174)Online publication date: 11-Oct-2009
https://dl.acm.org/doi/10.1145/1629395.1629419
Shafer JRixner S(2007)RiceNICProceedings of the 2007 workshop on Experimental computer science10.1145/1281700.1281721(21-es)Online publication date: 13-Jun-2007
https://dl.acm.org/doi/10.1145/1281700.1281721

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents