extended-abstract

Public Access

Using Containers and Tapis to Structure Portable, Composable and Reproducible Climate Science Workflows

Authors:

Michael Dodge II,

Sean Cleveland, and

Gwen A. JacobsAuthors Info & Claims

PEARC '22: Practice and Experience in Advanced Research Computing

July 2022

Article No.: 76, Pages 1 - 2

https://doi.org/10.1145/3491418.3535126

Published: 08 July 2022 Publication History

All formats PDF

Abstract

Provenance and Reproducibility have been growing needs in scientific computing workflows. This project seeks to split the traditionally monolithic code-base of a climate data computing workflow into small, functional, and semi-independent containers. Each container image is built from public code repositories, and allows a researcher to determine the exact process that was executed for both technical and scientific validation. These containers are composed into their workflows using the Tapis API’s Actor-Based Container (Abaco) system, which can be hosted on a variety of computing infrastructures. They may also be run as standalone containers on computers or virtual machines with Docker installed.

References

[1]

Joe Stubbs 2018. Rapid Development of Scalable, Distributed Computation with Abaco. Science Gateways Community Institute, 10th International Workshop on Science Gateways.

Google Scholar

[2]

Joe Stubbs, Richard Cardone, Mike Packard, Anagha Jamthe, Smruti Padhy, Steve Terry, Julia Looney, Joseph Meiring, Steve Black, Maytal Dahan, Sean Cleveland, and Gwen Jacobs. 2020. Tapis: An API Platform for Reproducible, Distributed Computational Research. (2020). submitted.

Google Scholar

Recommendations

Extending Tapis Workflow Management Framework with Elastic Google Cloud Distributed System using CloudyCluster by Omnibond
PEARC '22: Practice and Experience in Advanced Research Computing

The goal of a robust cyberinfrastructure (CI) ecosystem is to catalyse discovery and innovation. Tapis does this through offering a sustainable production-quality set of API services to support modern science and engineering research, which ...
Read More
Comparing FutureGrid, Amazon EC2, and Open Science Grid for Scientific Workflows

Scientists have many computing infrastructures available to conduct their research, including grids and public or private clouds. This article explores the use of these cyberinfrastructures to execute scientific workflows, an important class of ...
Read More
Novel Approaches Toward Scalable Composable Workflows in Hyper-Heterogeneous Computing Environments
SC-W '23: Proceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis

The annual Workshop on Workflows in Support of Large-Scale Science (WORKS) is a premier venue for the scientific workflow community to present the latest advances in research and development on the many facets of scientific workflows throughout their ...
Read More

Comments

Information & Contributors

Information

Published In

PEARC '22: Practice and Experience in Advanced Research Computing

July 2022

455 pages

ISBN:9781450391610

DOI:10.1145/3491418

Editors:
Julie Wernert
Indiana University
,
Alan Chalker
Ohio Supercomputer Center
,
Shava Smallen
San Diego Supercomputer Center, University of California, San Diego
,
Tabitha Samuel
University of Tennessee, Knoxville
,
JP Navarro
Argonne National Laboratory

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 July 2022

Check for updates

Author Tags

Qualifiers

Extended-abstract
Research
Refereed limited

Funding Sources

NSF (National Science Foundation)

Conference

PEARC '22

Sponsor:

PEARC '22: Practice and Experience in Advanced Research Computing

July 10 - 14, 2022

MA, Boston, USA

Acceptance Rates

Overall Acceptance Rate 133 of 202 submissions, 66%

Upcoming Conference

PEARC '24

Sponsor:
sighpc
sighpc

PEARC '24: Practice and Experience in Advanced Research Computing

July 21 - 25, 2024

Providence , RI , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
89
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)1

Other Metrics

View Author Metrics

Citations

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Recommendations

Extending Tapis Workflow Management Framework with Elastic Google Cloud Distributed System using CloudyCluster by Omnibond

Comparing FutureGrid, Amazon EC2, and Open Science Grid for Scientific Workflows

Novel Approaches Toward Scalable Composable Workflows in Hyper-Heterogeneous Computing Environments

Comments

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Acceptance Rates

Upcoming Conference

Other Metrics

Article Metrics

Other Metrics

PDF

eReader

HTML Format

Login options

Full Access

Abstract

References

Recommendations

Extending Tapis Workflow Management Framework with Elastic Google Cloud Distributed System using CloudyCluster by Omnibond

Comparing FutureGrid, Amazon EC2, and Open Science Grid for Scientific Workflows

Novel Approaches Toward Scalable Composable Workflows in Hyper-Heterogeneous Computing Environments

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

View options

PDF

eReader

HTML Format

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations