research-article

On causes of GridFTP transfer throughput variance

Authors:

Malathi Veeraraghavan,

Yee-Ting LiAuthors Info & Claims

NDM '13: Proceedings of the Third International Workshop on Network-Aware Data Management

Article No.: 5, Pages 1 - 10

https://doi.org/10.1145/2534695.2534701

Published: 17 November 2013 Publication History

Abstract

In prior work, we analyzed the GridFTP usage logs collected by data transfer nodes (DTNs) located at national scientific computing centers, and found significant throughput variance even among transfers between the same two end hosts. The goal of this work is to quantify the impact of various factors on throughput variance. Our methodology consisted of executing experiments on a high-speed research testbed, running large-sized instrumented transfers between operational DTNs, and creating statistical models from collected measurements. A non-linear regression model for memory-to-memory transfer throughput as a function of CPU usage at the two DTNs and packet loss rate was created. The model is useful for determining concomitant resource allocations to use in scheduling requests. For example, if a whole NERSC DTN CPU core can be assigned to the GridFTP process executing a large memory-to-memory transfer to SLAC, then only 32% of a CPU core is required at the SLAC DTN for the corresponding GridFTP process due to a difference in the computing speeds of these two DTNs. With these CPU allocations, data can be moved at 6.3 Gbps, which sets the rate to request from the circuit scheduler.

References

[1]

GridFTP v2 Protocol Description. {Online}. Available: http://www.ggf.org/documents/GFD.47.pdf

[2]

S. Tuecke, "Enhancing and supporting GridFTP: An essential component of DOE high-speed networking," 2013, talk presented at DOE Next-generation Networks for Science Program PI Meeting. {Online}. Available: https://indico.bnl.gov/contributionDisplay.py?contribId=36\&confId=566

[3]

I. Foster, "Transforming research using software as a service," 2013, keynote speech at TIP 2013. {Online}. Available: {http://www.internet2.edu/presentations/tip2013/20130115-Foster-keynote.pdf}

[4]

Z. Liu, M. Veeraraghavan, Z. Yan, C. Tracy, J. Tie, I. Foster, J. Dennis, J. Hick, Y. Li, and W. Yang, "On using virtual circuits for GridFTP transfers," in Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, ser. SC '12. Los Alamitos, CA, USA: IEEE Computer Society Press, 2012, pp. 81:1--81:11.

Digital Library

[5]

On-Demand Secure Circuits and Advance Reservation System (OSCARS). {Online}. Available: http://www.es.net/services/virtual-circuits-oscars/

[6]

Science DMZ: A Scalable Network Design Model for Optimizing Science Data Transfers. {Online}. Available: http://fasterdata.es.net/science-dmz/

[7]

J. Gu, D. Katramatos, X. Liu, V. Natarajan, A. Shoshani, A. Sim, D. Yu, S. Bradley, and S. McKee, "Stornet: Co-scheduling of end-to-end bandwidth reservation on storage and network systems for high-performance data transfers," in Computer Communications Workshops (INFOCOM WKSHPS), 2011 IEEE Conference on, 2011, pp. 121--126.

[8]

TeraPaths: Configuring End-to-End Virtual Network Paths with QoS Guarantees. {Online}. Available: https://www.racf.bnl.gov/terapaths/

[9]

B. Allen, J. Bresnahan, L. Childers, I. Foster, G. Kandaswamy, R. Kettimuthu, J. Kordas, M. Link, S. Martin, K. Pickett, and S. Tuecke, "Software as a service for data scientists," Communications of the ACM, vol. 55, no. 2, Feb. 2012.

Digital Library

[10]

Grid File Transfer Service (FTS). {Online}. Available: http://www.scc.kit.edu/dienste/5726.php

[11]

Portable Batch System (PBS). {Online}. Available: http://www.pbsworks.com/

[12]

Q. Wu, M. Zhu, Y. Gu, P. Brown, X. Lu, W. Lin, and Y. Liu, "A distributed workow management system with case study of real-life scientific applications on Grids," Journal of Grid Computing, vol. 10, no. 3, pp. 367--393, 2012. {Online}. Available: http://dx.doi.org/10.1007/s10723-012-9222-7

Digital Library

[13]

L. Ramakrishnan, C. Guok, K. Jackson, E. Kissel, D. Swany, and D. Agarwal, "On-demand overlay networks for large scientific data transfers," in Cluster, Cloud and Grid Computing (CCGrid), 2010 10th IEEE/ACM International Conference on, 2010, pp. 359--367.

Digital Library

[14]

J. Bresnahan, M. Link, R. Kettimuthu, and I. Foster, "Managed GridFTP," in Parallel and Distributed Processing Workshops and Phd Forum (IPDPSW), 2011 IEEE International Symposium on, May 2011, pp. 907--913.

Digital Library

[15]

I. Foster, M. Fidler, A. Roy, V. Sander, and L. Winkler, "End-to-end quality of service for high-end applications," Computer Communications, vol. 27, no. 14, pp. 1375--1388, 2004.

Digital Library

[16]

H.-H. Chu and K. Nahrstedt, "CPU service classes for multimedia applications," in Multimedia Computing and Systems, 1999. IEEE International Conference on, vol. 1, 1999, pp. 296--301 vol. 1.

Digital Library

[17]

B. Tierney, J. Lee, L. T. Chen, H. Herzog, G. Hoo, G. Jin, and W. E. Johnston, "Distributed parallel data storage systems: a scalable approach to high speed image servers," in Proceedings of the second ACM international conference on Multimedia, ser. MULTIMEDIA '94. New York, NY, USA: ACM, 1994, pp. 399--405. {Online}. Available: http://doi.acm.org/10.1145/192593.192709

Digital Library

[18]

G. Shipman, S. Atchley, Y. Kim, G. Vallee, G. Bosilca, T. Herault, and S. Moreaud, "An adaptive end-to-end approach for terabit data movement optimization," Next-Generation networks for science program PI Meeting, Emeryville, CA, March 18--20. {Online}. Available: https://indico.bnl.gov/getFile.py/access?contribId=26\&resId=0\&materialId=slides\&confId=566

[19]

H. Abbasi, G. Eisenhauer, M. Wolf, K. Schwan, and S. Klasky, "Just in time: adding value to the IO pipelines of high performance applications with JITStaging," in Proceedings of the 20th international symposium on High performance distributed computing, ser. HPDC '11. New York, NY, USA: ACM, 2011, pp. 27--36. {Online}. Available: http://doi.acm.org/10.1145/1996130.1996137

Digital Library

[20]

J. Basney and M. Livny, "Managing network resources in Condor," in High-Performance Distributed Computing, 2000. Proceedings. The Ninth International Symposium on, 2000, pp. 298--299.

Digital Library

[21]

H. H. Mohamed and D. H. J. Epema, "The design and implementation of the KOALA co-allocating grid scheduler," in Proceedings of the 2005 European conference on Advances in Grid Computing, ser. EGC'05. Berlin, Heidelberg: Springer-Verlag, 2005, pp. 640--650. {Online}. Available: http://dx.doi.org/10.1007/11508380_65

Digital Library

[22]

S. Vazhkudai and J. M. Schopf, "Using regression techniques to predict large data transfers," International Journal of High Performance Computing Applications, vol. 17, no. 3, pp. 249--268, 2003.

Digital Library

[23]

R. Rahman, K. Barker, and R. Alhajj, "Predicting the performance of GridFTP transfers," in Parallel and Distributed Processing Symposium, 2004. Proceedings. 18th International, April 2004, p. 238.

[24]

S. Vazhkudai and J. Schopf, "Using disk throughput data in predictions of end-to-end grid data transfers," in Grid Computing - GRID 2002, ser. Lecture Notes in Computer Science, M. Parashar, Ed. Springer Berlin Heidelberg, 2002, vol. 2536, pp. 291--304.

Digital Library

[25]

Esnet perfSONAR dashboard. {Online}. Available: http://ps-dashboard.es.net

[26]

Linux tuning. {Online}. Available: http://fasterdata.es.net/host-tuning/linux/

[27]

ESnet 100G Testbed. {Online}. Available: http://www.es.net/RandD/100g-testbed/

[28]

D. Leith and R. Shorten, "H-TCP: TCP for high-speed and long-distance networks," Proc. of PFLDnet, Feb. 16--17, 2004.

[29]

byte-unixbench - A Unix benchmark suite. {Online}. Available: http://code.google.com/p/byte-unixbench/

[30]

"Emulab - network emulation testbed." {Online}. Available: https://www.emulab.net/

[31]

ESnet. {Online}. Available: http://www.es.net/

[32]

tcptrace. {Online}. Available: http://www.tcptrace.org/

[33]

M. Mathis, J. Semke, J. Mahdavi, and T. Ott, "The macroscopic behavior of the TCP congestion avoidance algorithm," SIGCOMM Comput. Commun. Rev., vol. 27, no. 3, pp. 67--82, Jul. 1997.

Digital Library

[34]

"fda: Functional Data Analysis R statistical package." {Online}. Available: http://cran.r-project.org/web/packages/fda/index.html

[35]

NERSC file systems. {Online}. Available: http://www.nersc.gov/users/data-and-networking/file-systems/

[36]

Z. Yan, M. Veeraraghavan, C. Tracy, and C. Guok, "On how to provision Quality of Service (QoS) for large dataset transfers," in Proceedings of the Sixth International Conference on Communication Theory, Reliability, and Quality of Service (CTRQ), 2013.

[37]

Y. Gu and R. L. Grossman, "UDT: UDP-based data transfer for high-speed wide area networks," Comput. Netw., vol. 51, no. 7, pp. 1777--1799, May 2007. {Online}. Available: http://dx.doi.org/10.1016/j.comnet.2006.11.009

Digital Library

Cited By

Syal ALazar AKim JSim AWu KKim JSim A(2019)Automatic Detection of Network Traffic Anomalies and ChangesProceedings of the ACM Workshop on Systems and Network Telemetry and Analytics10.1145/3322798.3329255(3-10)Online publication date: 17-Jun-2019
https://dl.acm.org/doi/10.1145/3322798.3329255
Lazar AWu KSim A(2018)Predicting Network Traffic Using TCP Anomalies2018 IEEE International Conference on Big Data (Big Data)10.1109/BigData.2018.8622522(5369-5371)Online publication date: Dec-2018
https://doi.org/10.1109/BigData.2018.8622522
Kettimuthu RAgrawal GSadayappan PFoster I(2016)Differentiated Scheduling of Response-Critical and Best-Effort Wide-Area Data Transfers2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS.2016.97(1113-1122)Online publication date: May-2016
https://doi.org/10.1109/IPDPS.2016.97

Recommendations

Communicating Security Assertions over the GridFTP Control Channel
ESCIENCE '08: Proceedings of the 2008 Fourth IEEE International Conference on eScience

The GridFTP [1] protocol defines a general-purpose mechanism for secure, reliable, high-performance data movement. GridFTP has been widely used for efficiently transferring large volumes of data. GSI is the commonly used security mechanism for GridFTP ...
Long-haul secure data transfer using hardware-assisted GridFTP

Extreme-scale scientific collaborations require high-performance wide-area end-to-end data transports to enable fast and secure transfer of high data volumes among collaborating institutions. GridFTP is the de facto protocol for large-scale data ...
Globus GridFTP: what's new in 2007
GridNets '07: Proceedings of the first international conference on Networks for grid applications

GridFTP is a high-performance, secure, reliable data transfer protocol optimized for high-bandwidth wide-area networks. It is based on the Internet FTP protocol, and it defines extensions for high-performance operation and security. The Globus ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

NDM '13: Proceedings of the Third International Workshop on Network-Aware Data Management

November 2013

84 pages

ISBN:9781450325226

DOI:10.1145/2534695

General Chairs:
Mehmet Balman
VMware R&D & Lawrence Berkeley National Laboratory
,
Surendra Byna
Lawrence Berkeley National Laboratory
,
Brian L. Tierney
Energy Sciences Network & Lawrence Berkeley National Laboratory

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 November 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

SC13

Sponsor:

SC13: International Conference for High Performance Computing, Networking, Storage and Analysis

November 17 - 21, 2013

Colorado, Denver

Acceptance Rates

NDM '13 Paper Acceptance Rate 9 of 14 submissions, 64%;

Overall Acceptance Rate 14 of 23 submissions, 61%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
91
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Syal ALazar AKim JSim AWu KKim JSim A(2019)Automatic Detection of Network Traffic Anomalies and ChangesProceedings of the ACM Workshop on Systems and Network Telemetry and Analytics10.1145/3322798.3329255(3-10)Online publication date: 17-Jun-2019
https://dl.acm.org/doi/10.1145/3322798.3329255
Lazar AWu KSim A(2018)Predicting Network Traffic Using TCP Anomalies2018 IEEE International Conference on Big Data (Big Data)10.1109/BigData.2018.8622522(5369-5371)Online publication date: Dec-2018
https://doi.org/10.1109/BigData.2018.8622522
Kettimuthu RAgrawal GSadayappan PFoster I(2016)Differentiated Scheduling of Response-Critical and Best-Effort Wide-Area Data Transfers2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS.2016.97(1113-1122)Online publication date: May-2016
https://doi.org/10.1109/IPDPS.2016.97

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten