research-article

Virtual Environment for Testing Software-Defined Networking Solutions for Scientific Workflows

Authors:

Nageswara S. V. Rao,

Satyabrata Sen,

Bradley W. Settlemyer,

Hsing-Bung Chen,

Joshua M. Boley,

Rajkumar Kettimuthu,

Dimitrios KatramatosAuthors Info & Claims

AI-Science'18: Proceedings of the 1st International Workshop on Autonomous Infrastructure for Science

Article No.: 3, Pages 1 - 8

https://doi.org/10.1145/3217197.3217202

Published: 11 June 2018 Publication History

Abstract

Recent developments in software-defined infrastructures promise that scientific workflows utilizing supercomputers, instruments, and storage systems will be dynamically composed and orchestrated using software at unprecedented speed and scale in the near future. Testing of the underlying networking software, particularly during initial exploratory stages, remains a challenge due to potential disruptions, and resource allocation and coordination needed over the multi-domain physical infrastructure. To overcome these challenges, we develop the Virtual Science Network Environment (VSNE) that emulates the multi-site host, storage, and network infrastructure using Virtual Machines (VMs), wherein the production and nascent software can be tested. Within each VM, which represents a site, the hosts and local-area networks are emulated using Mininet, and the Software-Defined Network (SDN) controllers and service daemon codes are natively run to support dynamic provisioning of network connections. Additionally, Lustre filesystem support at the sites and an emulation of the long-haul network using Mininet, are provided using separate VMs. As case studies, we describe Lustre file transfers using XDD, Red5 streaming service demonstration, and an emulated experiment with remote monitoring and steering modules, all supported over dynamically configured connections using SDN controllers.

References

[1]

Software defined networking for extreme-scale science: Data, compute, and instrument facilities. DOE ASCR Workshop, Bethesda, MD, Aug. 2014.

[2]

ASCR network requirements review. Esnet report, Germantown, MD, Apr. 2015.

[3]

S. Bemby, H. Lu, K. H. Zadeh, H. Bannazadeh, and A. Leon-Garcia. ViNO: SDN overlay to allow seamless migration across heterogeneous infrastructure. In IFIP/IEEE International Symposium on Integrated Network Management (IM), pages 782--785, Ottawa, Canada, May 2015.

[4]

Chameleon: A configurable experimental environment for large-scale cloud research. https://www.chameleoncloud.org.

[5]

R. F. da Silva, E. Deelman, R. Filgueira, K. Vahi, M. Rynge, R. Mayani, and B. Mayer. Automating environmental computing applications with scientific workflows. In 2016 IEEE 12th International Conference on e-Science (e-Science), pages 400--406, Oct. 2016.

[6]

Energy Sciences Network. http://www.es.net.

[7]

Floodlight. https://www.projectfloodlight.org.

[8]

Jetstream: A National Science and Engineering Cloud. https://jetstream-cloud.org.

[9]

D. Kreutz, F. M. V. Ramos, P. E. Verissimo, C. E. Rothenberg, S. Azodolmolky, and S. Uhlig. Software-defined networking: A comprehensive survey. Proceedings of the IEEE, 103(1):14--76, 2015.

[10]

N. M. Law et al. The Palomar Transient Factory: System overview, performance, and first results. Pub. Astronomical Soc. Pacific, 121(886):1395--1408, Dec. 2009.

[11]

Y. D. Lin, D. Pitt, D. Hausheer, E. Johnson, and Y. B. Lin. Software-defined networking: Standardization for cloud computing's second wave. Computer, 47(11):19--21, Nov. 2014.

Digital Library

[12]

Lustre Basics, https://www.olcf.ornl.gov/kb_articles/lustre-basics.

[13]

Mininet: An instant virtual network on your laptopr. https://mininet.org.

[14]

OpenDaylight. https://www.opendaylight.org.

[15]

ONOS. https://www.onosproject.org.

[16]

The OpenGENI rack project. http://www.opengeni.net.

[17]

On-demand Secure Circuits and Advance Reservation System. http://www.es.net/oscars.

[18]

N. S. V. Rao, Q. Liu, S. Sen, G. Hinkel, N. Imam, B. W. Settlemyer, I. T. Foster, et al. Experimental analysis of file transfer rates over wide-area dedicated connections. In 18th IEEE International Conference on High Performance Computing and Communications (HPCC), pages 198--205, Sydney, Australia, Dec. 2016.

[19]

N. S. V. Rao, Q. Liu, S. Sen, R. Kettimuthu, J. Boley, Bradley W. Settlemyer, H. B. Chen, and Katramatos D. Regression-based analytics for response dynamics of SDN solutions and components. In Proceedings of Workshop on Emerging Trends in Softwarized Networks, pages 53:1--53:10, Montreal, Canada, June 2018.

[20]

N. S. V. Rao, Q. Liu, S. Sen, R. Kettimuthu, J. Boley, Bradley W. Settlemyer, H. B. Chen, D. Katramatos, and D. Yu. Software-defined network solutions for science scenarios: Performance testing framework and measurements. In Proceedings of the 19th ACM International Conference on Distributed Computing and Networking (ICDCN '18), pages 53:1--53:10, Varanasi, India, Jan. 2018.

Digital Library

[21]

Red5 Media Server. https://red5.org.

[22]

A. R. Roy, M. F. Bari, M. F. Zhani, R. Ahmed, and R. Boutaba. Design and management of DOT: A distributed openflow testbed. In IEEE Network Operations and Management Symposium (NOMS), pages 1--9, Krakow, Poland, May 2014.

[23]

Web2py Web Framework. http://www.web2py.com/.

[24]

XDD - The eXtreme dd toolset, https://github.com/bws/xdd.

[25]

W. Xia, Y. Wen, C. H. Foh, D. Niyato, and H. Xie. A survey on software-defined networking. IEEE Communication Surveys & Tutorials, 17(1):27--51, First Quarter 2015.

Cited By

Al-Najjar ARao N(2023)Virtual Infrastructure Twin for Computing-Instrument Ecosystems: Software and MeasurementsIEEE Access10.1109/ACCESS.2023.324695411(20254-20266)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3246954
Rao NAl-Najjar AZandi HSankaran RHicks SRoccapriori KMukherjee D(2023)Virtual Infrastructure Twins: Software Testing Platforms for Computing-Instrument EcosystemsAccelerating Science and Engineering Discoveries Through Integrated Research Infrastructure for Experiment, Big Data, Modeling and Simulation10.1007/978-3-031-23606-8_10(155-172)Online publication date: 18-Jan-2023
https://doi.org/10.1007/978-3-031-23606-8_10
Al-Najjar ARao NSankaran RZiatdinov MMukherjee DOvchinnikova ORoccapriore KLupini AKalinin S(2022)Enabling Autonomous Electron Microscopy for Networked Computation and Steering2022 IEEE 18th International Conference on e-Science (e-Science)10.1109/eScience55777.2022.00040(267-277)Online publication date: Oct-2022
https://doi.org/10.1109/eScience55777.2022.00040
Show More Cited By

Index Terms

Virtual Environment for Testing Software-Defined Networking Solutions for Scientific Workflows
1. Networks
  1. Network performance evaluation
    1. Network experimentation
  2. Network services

Recommendations

Software-Defined Networking: On the Verge of a Breakthrough?

Many experts predict that software-defined networking, a technology that's been highly touted for several years, will soon finally begin gaining ground in the marketplace.
Security in Software-Defined-Networking: A Survey
Security, Privacy, and Anonymity in Computation, Communication, and Storage
Abstract
With the development of information and networking technologies, conventional network has been unable to meet the demands of practical applications and network users. A new network paradigm called Software-Defined Networking (SDN) was proposed and ...
Network Traffic Analysis in Software-Defined Networking Using RYU Controller
Abstract
Software-Defined Networking (SDN) has emerged as a promising paradigm to enhance network control and management by decoupling the planes. With SDN, the centralized controller plays a critical role in managing network resources and traffic flows. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

AI-Science'18: Proceedings of the 1st International Workshop on Autonomous Infrastructure for Science

June 2018

53 pages

ISBN:9781450358620

DOI:10.1145/3217197

Copyright © 2018 ACM.

© 2018 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the United States Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

University of Arizona: University of Arizona
SIGARCH: ACM Special Interest Group on Computer Architecture

In-Cooperation

SIGHPC: ACM Special Interest Group on High Performance Computing, Special Interest Group on High Performance Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 June 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

HPDC '18

Sponsor:

University of Arizona
SIGARCH

HPDC '18: The 27th International Symposium on High-Performance Parallel and Distributed Computing

June 11, 2018

AZ, Tempe, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
97
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)1

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Al-Najjar ARao N(2023)Virtual Infrastructure Twin for Computing-Instrument Ecosystems: Software and MeasurementsIEEE Access10.1109/ACCESS.2023.324695411(20254-20266)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3246954
Rao NAl-Najjar AZandi HSankaran RHicks SRoccapriori KMukherjee D(2023)Virtual Infrastructure Twins: Software Testing Platforms for Computing-Instrument EcosystemsAccelerating Science and Engineering Discoveries Through Integrated Research Infrastructure for Experiment, Big Data, Modeling and Simulation10.1007/978-3-031-23606-8_10(155-172)Online publication date: 18-Jan-2023
https://doi.org/10.1007/978-3-031-23606-8_10
Al-Najjar ARao NSankaran RZiatdinov MMukherjee DOvchinnikova ORoccapriore KLupini AKalinin S(2022)Enabling Autonomous Electron Microscopy for Networked Computation and Steering2022 IEEE 18th International Conference on e-Science (e-Science)10.1109/eScience55777.2022.00040(267-277)Online publication date: Oct-2022
https://doi.org/10.1109/eScience55777.2022.00040
Papadimitriou GLyons EWang CThareja KTanaka RRuth PRodero IDeelman EZink MMandal A(2021)Fair sharing of network resources among workflow ensemblesCluster Computing10.1007/s10586-021-03457-325:4(2873-2891)Online publication date: 22-Nov-2021
https://doi.org/10.1007/s10586-021-03457-3
Papadimitriou GLyons EWang CThareja KTanaka RRuth PVillalobos JRodero IDeelman EZink MMandal A(2020)Application Aware Software Defined Flows of Workflow Ensembles2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)10.1109/INDIS51933.2020.00007(10-21)Online publication date: Nov-2020
https://doi.org/10.1109/INDIS51933.2020.00007

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents