Article

Experiences in Design and Implementation of a High Performance Transport Protocol

Authors:

Robert L. GrossmanAuthors Info & Claims

SC '04: Proceedings of the 2004 ACM/IEEE conference on Supercomputing

Page 22

https://doi.org/10.1109/SC.2004.24

Published: 06 November 2004 Publication History

Abstract

This paper describes our experiences in the development of the UDP-based Data Transport (UDT) protocol, an application level transport protocol used in distributed data intensive applications. The new protocol is motivated by the emergence of wide area high-speed optical networks, in which TCP is often found to fail to utilize the abundant bandwidth. UDT demonstrates good efficiency and fairness (including RTT fairness and TCP friendliness) characteristics in high performance computing applications where a small number of bulk sources share the abundant bandwidth. It combines both rate and window control and uses bandwidth estimation to determine the control parameters automatically. This paper presents the rationale behind UDT: how UDT integrates these schemes to support high performance data transfer, why these schemes are used, and what the main issues are in the design and implementation of this high performance transport protocol.

References

[1]

{1} H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson. RTP: A Transport Protocol for Real-Time Applications. RFC 1889.

[2]

{2} Y. Gu and R. L. Grossman. SABUL: A Transport Protocol for Grid Computing. Journal of Grid Computing, to appear.

[3]

{3} UDT source release. http://sourceforge.net/projects/dataspace.

[4]

{4} D. Katabi, M. Hardley, and C. Rohrs. Internet Congestion Control for Future High Bandwidth-Delay Product Environments, ACM SIGCOMM '02, Pittsburgh, PA, Aug. 19-23, 2002.

Digital Library

[5]

{5} J. Padhye, V. Firoiu, D. Towsley, and J. Kurose. Modeling TCP throughput: a simple model and its empirical validation. ACM SIGCOMM '98, Vancouver, BC, Canada, Sep. 2-4, 1998.

Digital Library

[6]

{6} Y. Zhang, E. Yan, and S. K. Dao. A Measurement of TCP over Long-Delay Network. The 6th International Conference on Telecommunication Systems, Modeling and Analysis, Nashville, TN, March 1998.

[7]

{7} W. Feng and P. Tinnakornsrisuphap. The Failure of TCP in High-Performance Computational Grids. SC '00, Dallas, TX, Nov. 4-10, 2000.

Digital Library

[8]

{8} M. Mazzucco, A. Ananthanarayan, R. Grossman, J. Levera, and G. Bhagavantha Rao. Merging Multiple Data Streams on Common Keys over High Performance Networks. SC '02, Baltimore, MD, Nov. 16-22, 2002.

Digital Library

[9]

{9} NS-2, http://www.isi.edu/nsnam/ns/.

[10]

{10} D. Clark, M. Lambert, and L. Zhang. NETBLT: A high throughput transport protocol. ACM SIGCOMM '87, Stowe, VT, Aug. 1987.

Digital Library

[11]

{11} D. Cheriton. VMTP: A transport protocol for the next generation of communication systems. ACM SIGCOMM '87, Stowe, VT, Aug. 1987.

Digital Library

[12]

{12} T. Strayer, B. Dempsey, and A. Weaver. XTP - the Xpress Transfer Protocol. Addison-Wesley Publishing Company, 1992.

Digital Library

[13]

{13} S. Floyd, J. Mahdavi, M. Mathis, and M. Podolsky. An Extension to the Selective Acknowledgement (SACK) Option for TCP. RFC 2883, Proposed Standard, July 2000.

[14]

{14} M. Gerla, M. Y. Sanadidi, R. Wang, A. Zanella, C. Casetti, and S. Mascolo. TCP Westwood: Congestion Window Control Using Bandwidth Estimation. IEEE Globecom 2001, Volume: 3, pp 1698-1702.

[15]

{15} L. Brakmo and L. Peterson. TCP Vegas: End-to-End Congestion Avoidance on a Global Internet. IEEE Journal on Selected Areas in Communication, Vol 13, No. 8 (October 1995) pages 1465-1480.

Digital Library

[16]

{16} C. Jin, D. X. Wei, and S. H. Low. FAST TCP: motivation, architecture, algorithms, performance. IEEE Infocom '04, Hongkong, China, Mar. 2004.

[17]

{17} S. Floyd. HighSpeed TCP for Large Congestion Windows. RFC 3649, Experimental Standard, Dec. 2003.

Digital Library

[18]

{18} T. Kelly. Scalable TCP: Improving Performance in Highspeed Wide Area Networks. ACM Computer Communication Review, Apr. 2003.

Digital Library

[19]

{19} L. Xu, K. Harfoush, and I. Rhee. Binary Increase Congestion Control for Fast Long-Distance Networks. IEEE Infocom '04, Hongkong, China, Mar. 2004.

[20]

{20} H. Sivakumar, S. Bailey, R. L. Grossman. PSockets: The Case for Application-level Network Striping for Data Intensive Applications using High Speed Wide Area Networks, SC '00, Dallas, TX, Nov. 2000.

Digital Library

[21]

{21} D. Clark and D. Tennenhouse. Architectural Considerations for a New Generation of Protocols. ACM SIGCOMM '90, Philadelphia, PA, Sep. 24-27, 1990.

Digital Library

[22]

{22} T. Braun and C. Diot. Protocol Implementation Using Integrated Layer Processing. ACM SIGCOMM '95, Cambridge, MA, Aug. 28-Sep. 1, 1995.

Digital Library

[23]

{23} S. Leue and P. Oechslin. On parallelizing and optimizing the implementation of communication protocols. IEEE/ACM Transactions on Networking 4(1): 55-70 (1996).

Digital Library

[24]

{24} S. H. Rodrigues, T. E. Anderson, and D. E. Culler. High-Performance Local Area Communication with Fast Sockets. USENIX '97, Anaheim, California, January 6-10, 1997.

Digital Library

[25]

{25} J. Chu. Zero-copy TCP in Solaris. Usenix '96, San Diego, CA, Jan. 1996.

Digital Library

[26]

{26} A. Edwards and S. Muir. Experiences Implementing A High-Performance TCP In User-Space. ACM SIGCOMM '95, Cambridge, MA, Aug. 28-Sep. 1, 1995.

Digital Library

[27]

{27} A. Banerjea, D. Ferrari, B. Mah, M. Moran, D. C. Verma, and H. Zhang. The Tenet real-time protocol suite: Design, implementation, and experiences. IEEE/ACM Trans. Networking, vol. 4, pp. 1-11, Feb. 1996.

Digital Library

[28]

{28} Y. Gu and R. Grossman. UDT: A Transport Protocol for Data Intensive Applications. Internet Draft, work in progress.

[29]

{29} A. Aggarwal, S. Savage, and T. Anderson. Understanding the Performance of TCP Pacing. IEEE Infocom '00, Tel Aviv, Israel, Mar. 26-30, 2000.

[30]

{30} D. Chiu and R. Jain. Analysis of the Increase/Decrease Algorithms for Congestion Avoidance in Computer Networks. Journal of Computer Networks and ISDN, Vol. 17, No. 1, June 1989, pp. 1-14.

Digital Library

[31]

{31} M. Allman and V. Paxson. On Estimating End-to-End Network Path Properties. ACM SIGCOMM '99, Cambridge, MA, Aug. 30-Sep. 3, 1999.

Digital Library

[32]

{32} V. Paxson. End-to-End Internet Packet Dynamics. IEEE/ACM Transactions on Networking, Vol.7, No.3, pp. 277-292, June 1999.

Digital Library

[33]

{33} R. Jain. The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling. Wiley- Interscience, New York, NY, April 1991.

[34]

{34} S. Floyd and K. Fall. Promoting the use of end-to -end congestion control in the Internet. IEEE/ACM Transactions on Networking, 7(4): 458-472, 1999.

Digital Library

[35]

{35} N. Jain, M. Schawrtz, T. Bashkow. Transport protocol processing at GBPS rates. SIGCOMM '90, Philadelphia, PA, Sep. 24-27, 1990.

Digital Library

[36]

{36} D. Leith. Linux TCP Implementation Issues in High-Speed Networks. Technical report, http://www.hamilton.ie/net/LinuxHighSpeed.pdf.

[37]

{37} Intel 82093AA I/O Advanced Programmable Interrupt Controller (I/O APIC), http://www.intel.com/design/chipsets/datashts/290566.htm.

[38]

{38} M. Aron and P. Druschel. Soft timers: Efficient microsecond software timer support for network processing. ACM Transactions on Computer Systems 18, 3 (2000), 197-228.

Digital Library

[39]

{39} P. Joubert, R. King, R. Neves, M. Russinovich, and J. Tracey. High-Performance Memory-Based Web Servers: Kernel and User-Space Performance. USENIX '01, Boston, Massachusetts, June 2001.

Digital Library

[40]

{40} D. J. Leith and R. Shorten. H-TCP Protocol for High-Speed Long Distance Networks. PFLDnet '04, Feb. 2004. http://dsd.lbl.gov/DIDC/PFLDnet2004/talks/Leith-slides.pdf.

[41]

{41} K. Kumazoe, Y. Hori, M. Tsuru, and Y. Oie. Transport Protocol for Fast Long Distance Networks: Comparison of Their Performances in JGN. SAINT '04, Tokyo, Japan, 26- 30 Jan. 2004.

[42]

{42} R. Jain. A Delay Based Approach for Congestion Avoidance in Interconnected Heterogeneous Computer Networks. ACM SIGCOMM '89, Austin, TX, Sep. 19-22, 1989.

[43]

{43} A. Kuzmanovic and E. W. Knightly. TCP-LP: A Distributed Algorithm for Low Priority Data Transfer. IEEE Infocom '03, San Francisco, CA, Mar. 30-Apr. 3, 2003.

[44]

{44} M. Jain and C. Dovrolis. Pathload: A Measurement Tool for End-to-End Available Bandwidth. Passive and Active Measurements (PAM) 2002 workshop, pp 14-25, Fort Collins, CO.

[45]

{45} J. Martin, A. Nilsson, and I. Rhee. Delay-based congestion avoidance for TCP. ACM/IEEE Transactions on Networks, June 2003.

Digital Library

[46]

{46} Intel VTune Performance Analyzer, http://www.intel.com/software/products/vtune.

Cited By

Han ZFan WLi JXu M(2018)A Novel UDT-Based Transfer Speed-Up Protocol for Fog ComputingWireless Communications & Mobile Computing10.1155/2018/36812702018Online publication date: 13-May-2018
https://dl.acm.org/doi/10.1155/2018/3681270
Ahuja VBanerjee AFarrens MGhosal DSerazzi GMaccabe AThain D(2011)Introspective end-system modeling to optimize the transfer time of rate based protocolsProceedings of the 20th international symposium on High performance distributed computing10.1145/1996130.1996140(61-72)Online publication date: 8-Jun-2011
https://dl.acm.org/doi/10.1145/1996130.1996140
Banerjee AFeng WGhosal DMukherjee B(2008)Algorithms for Integrated Routing and Scheduling for Aggregating Data from Distributed Resources on a Lambda GridIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2007.111219:1(24-34)Online publication date: 1-Jan-2008
https://dl.acm.org/doi/10.1109/TPDS.2007.1112
Show More Cited By

Recommendations

Teraflows over Gigabit WANs with UDT
Special issue: High-speed networks and services for data-intensive grids: The DataTAG project

The TCP transport protocol is currently inefficient for high speed data transfers over long distance networks with high bandwidth delay products (BDP). The challenge is to develop a protocol which is fast over networks with high bandwidth delay products,...
RELD, RTT ECN Loss Differentiation to optimize the performance of transport protocols on wireless networks

One major yet unsolved problem in wired-cum-wireless networks is the classification of losses, which might result from wireless temporary interferences or from network congestion. The transport protocol response to losses should be different for these ...
TCP Westwood(+) protocol implementation in ns-3
SimuTools '13: Proceedings of the 6th International ICST Conference on Simulation Tools and Techniques

The poor performance of conventional TCP protocols in error prone channels is a well studied issue. Numerous optimizations to TCP to address this problem have been proposed. TCP Westwood is one such protocol engineered for use in wireless networks and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SC '04: Proceedings of the 2004 ACM/IEEE conference on Supercomputing

November 2004

724 pages

ISBN:0769521533

General Chair:
Jeff Huskamp

Sponsors

SIGARCH: ACM Special Interest Group on Computer Architecture
IEEE-CS: Computer Society

Publisher

IEEE Computer Society

United States

Publication History

Published: 06 November 2004

Check for updates

Author Tags

Qualifiers

Article

Conference

SC '04

Sponsor:

SIGARCH
IEEE-CS

SC '04: International Conference for High Performance Computing, Networking, Storage and Analysis

November 6 - 12, 2004

Acceptance Rates

SC '04 Paper Acceptance Rate 60 of 200 submissions, 30%;

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
22
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Han ZFan WLi JXu M(2018)A Novel UDT-Based Transfer Speed-Up Protocol for Fog ComputingWireless Communications & Mobile Computing10.1155/2018/36812702018Online publication date: 13-May-2018
https://dl.acm.org/doi/10.1155/2018/3681270
Ahuja VBanerjee AFarrens MGhosal DSerazzi GMaccabe AThain D(2011)Introspective end-system modeling to optimize the transfer time of rate based protocolsProceedings of the 20th international symposium on High performance distributed computing10.1145/1996130.1996140(61-72)Online publication date: 8-Jun-2011
https://dl.acm.org/doi/10.1145/1996130.1996140
Banerjee AFeng WGhosal DMukherjee B(2008)Algorithms for Integrated Routing and Scheduling for Aggregating Data from Distributed Resources on a Lambda GridIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2007.111219:1(24-34)Online publication date: 1-Jan-2008
https://dl.acm.org/doi/10.1109/TPDS.2007.1112
Datta PFeng WSharma SHorner-Miller B(2006)End-system aware, rate-adaptive protocol for network transport in LambdaGrid environmentsProceedings of the 2006 ACM/IEEE conference on Supercomputing10.1145/1188455.1188572(112-es)Online publication date: 11-Nov-2006
https://dl.acm.org/doi/10.1145/1188455.1188572

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten