article

A taxonomy of scientific workflow systems for grid computing

Authors:

Rajkumar BuyyaAuthors Info & Claims

ACM SIGMOD Record, Volume 34, Issue 3

Pages 44 - 49

https://doi.org/10.1145/1084805.1084814

Published: 01 September 2005 Publication History

Abstract

With the advent of Grid and application technologies, scientists and engineers are building more and more complex applications to manage and process large data sets, and execute scientific experiments on distributed resources. Such application scenarios require means for composing and executing complex workflows. Therefore, many efforts have been made towards the development of workflow management systems for Grid computing. In this paper, we propose a taxonomy that characterizes and classifies various approaches for building and executing workflows on Grids. The taxonomy not only highlights the design and engineering similarities and differences of state-of-the-art in Grid workflow systems, but also identifies the areas that need further research.

References

[1]

J. Almond and D. Snelling. UNICORE: Secure and Uniform Access to Distributed Resources via the World Wide Web. White Paper, October 1998,]]

[2]

I. Altintas et al. A Framework for the Design and Reuse of Grid Workflows, International Workshop on Scientific Applications on Grid Computing (SAG'04), LNCS 3458, Springer, 2005]]

Digital Library

[3]

F. Berman et al. The GrADS Project: Software Support for High-Level Grid Application Development. International Journal of High Performance Computing Applications(JHPCA), 15(4):327--344, SAGE Publications Inc., London, UK, Winter 2001.]]

Digital Library

[4]

J. Cao et al. GridFlow:Workflow Management for Grid Computing. In 3rd International Symposium on Cluster Computing and the Grid (CCGrid), Tokyo, Japan, IEEE CS Press, Los Alamitos, CA, USA, May 12--15, 2003.]]

Digital Library

[5]

E. Deelman, J. Blythe, Y. Gil, and C. Kesselman. Workflow Management in GriPhyN. The Grid Resource Management, Kluwer, Netherlands, 2003.]]

[6]

E. Deelman et al. Mapping Abstract Complex Workflows onto Grid Environments. Journal of Grid Computing, 1:25--39, Kluwer Academic Publishers, Netherlands, 2003.]]

[7]

T. Fahringer et al. Truong. ASKALON: a tool set for cluster and Grid computing. Concurrency and Computation: Practice and Experience, 17:143--169, Wiley InterScience, 2005.]]

Digital Library

[8]

D. Fernández-Baca. Allocating Modules to Processors in a Distributed System. IEEE Transactions on Software Engineering, 15(11): 1427--1436, November 1989.]]

Digital Library

[9]

I. Foster and C. Kesselman (editors), The Grid: Blueprint for a Future Computing Infrastructure, Morgan Kaufmann Publishers, USA, 1999.]]

Digital Library

[10]

A. Geppert, M. Kradolfer, and D. Tombros. Market-based Workflow Management. International Journal of Cooperative Information Systems, World Scientific Publishing Co., NJ, USA, 1998.]]

[11]

V. Hamscher et al. Evaluation of Job-Scheduling Strategies for Grid Computing. In 1st IEEE/ACM International Workshop on Grid Computing (Grid 2000), Springer-Verlag, Heidelberg, Germany, 2000; 191--202.]]

Digital Library

[12]

S. Hwang and C. Kesselman. Grid Workflow: A Flexible Failure Handling Framework for the Grid. In 12th IEEE International Symposium on High Performance Distributed Computing (HPDC'03), Seattle, Washington, USA, IEEE CS Press, Los Alamitos, CA, USA, June 22--24, 2003.]]

Digital Library

[13]

G. von Laszewski. Java CoG Kit Workflow Concepts for Scientific Experiments. Technical Report, Argonne National Laboratory, Argonne, IL, USA, 2005.]]

[14]

B. Ludäscher et al. Scientific Workflow Management and the KEPLER System. Concurrency and Computation: Practice & Experience, Special Issue on Scientific Workflows, to appear, 2005]]

[15]

A. Mayer et al. Workflow Expression: Comparison of Spatial and Temporal Approaches. In Workflow in Grid Systems Workshop, GGF-10, Berlin, March 9, 2004.]]

[16]

S. McGough et al. Workflow Enactment in ICENI. In UK e-Science All Hands Meeting, Nottingham, UK, IOP Publishing Ltd, Bristol, UK, Sep. 2004; 894--900.]]

[17]

T. Oinn et al. Taverna: a tool for the composition and enactment of bioinformatics workflows. Bioinformatics, 20(17):3045--3054, Oxford University Press, London, UK, 2004.]]

Digital Library

[18]

S. S. Song, Y. K. Kwok, and K. Hwang. Trusted Job Scheduling in Open computational Grids: Security-Driven heuristics and A Fast Genetic Algorithm. In 19th IEEE International Parallel & Distributed Processing Symposium (IPDPS-2005), Denver, CO, USA., IEEE CS Press, Los Alamitos, CA, USA., April 4--8, 2005.]]

Digital Library

[19]

T. Tannenbaum, D. Wright, K. Miller, and M. Livny. Condor - A Distributed Job Scheduler. Beowulf Cluster Computing with Linux, The MIT Press, MA, USA, 2002.]]

Digital Library

[20]

I. Taylor, M. Shields, and I. Wang. Resource Management of Triana P2P Services. Grid Resource Management, Kluwer, Netherlands, June 2003.]]

[21]

W3C. Extensible Markup Language (XML) 1.0]]

[22]

J. Yu and R. Buyya. A Novel Architecture for Realizing Grid Workflow using Tuple Spaces. In 5th IEEE/ACM International Workshop on Grid Computing (GRID 2004), Pittsburgh, USA, IEEE CS Press, Los Alamitos, CA, USA, Nov. 8, 2004.]]

Digital Library

[23]

J. Yu and R. Buyya. A Taxonomy of Workflow Management Systems for Grid Computing. Technical Report, GRIDS-TR-2005-1, Grid Computing and Distributed Systems Laboratory, University of Melbourne, Australia, March 10, 2005.]]

Cited By

Sänger MDe Mecquenem NLewińska KBountris VLehmann FLeser UKosch T(2024)A qualitative assessment of using ChatGPT as large language model for scientific workflow developmentGigaScience10.1093/gigascience/giae03013Online publication date: 19-Jun-2024
https://doi.org/10.1093/gigascience/giae030
Xu QXiang LWu HTao CWang HYu LLiu QWang X(2024)Dynamic heatmap pyramid computation for massive high-parallel spatial streaming in urban environmentsInternational Journal of Digital Earth10.1080/17538947.2024.236809917:1Online publication date: 25-Jun-2024
https://doi.org/10.1080/17538947.2024.2368099
Pramanik PPal SChoudhury P(2024)Mobile crowd computing: potential, architecture, requirements, challenges, and applicationsThe Journal of Supercomputing10.1007/s11227-023-05545-080:2(2223-2318)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s11227-023-05545-0
Show More Cited By

Index Terms

A taxonomy of scientific workflow systems for grid computing

Recommendations

The Grid Resource Broker workflow engine
2nd International Workshop on Workflow Management and Applications in Grid Environments (WaGe2007)

Increasingly, complex scientific applications are structured in terms of workflows. These applications are usually computationally and-or data intensive and thus are well suited for execution in grid environments. Distributed, geographically spread ...
Comparing FutureGrid, Amazon EC2, and Open Science Grid for Scientific Workflows

Scientists have many computing infrastructures available to conduct their research, including grids and public or private clouds. This article explores the use of these cyberinfrastructures to execute scientific workflows, an important class of ...
Specification and runtime workflow support in the ASKALON Grid environment
Dynamic Computational Workflows: Discovery, Optimization and Scheduling

We describe techniques to support the runtime execution of scientific workflows in the ASKALON Grid environment. We present a formal model and three middleware services that support in combination the effective execution in heterogeneous and dynamic ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGMOD Record

ACM SIGMOD Record Volume 34, Issue 3

September 2005

115 pages

ISSN:0163-5808

DOI:10.1145/1084805

Issue’s Table of Contents

Copyright © 2005 Authors.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 September 2005

Published in SIGMOD Volume 34, Issue 3

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

398
Total Citations
View Citations
2,700
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)5

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sänger MDe Mecquenem NLewińska KBountris VLehmann FLeser UKosch T(2024)A qualitative assessment of using ChatGPT as large language model for scientific workflow developmentGigaScience10.1093/gigascience/giae03013Online publication date: 19-Jun-2024
https://doi.org/10.1093/gigascience/giae030
Xu QXiang LWu HTao CWang HYu LLiu QWang X(2024)Dynamic heatmap pyramid computation for massive high-parallel spatial streaming in urban environmentsInternational Journal of Digital Earth10.1080/17538947.2024.236809917:1Online publication date: 25-Jun-2024
https://doi.org/10.1080/17538947.2024.2368099
Pramanik PPal SChoudhury P(2024)Mobile crowd computing: potential, architecture, requirements, challenges, and applicationsThe Journal of Supercomputing10.1007/s11227-023-05545-080:2(2223-2318)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s11227-023-05545-0
Covington ESuresh KAnderson BBarker MDess KPrice JMoncion AVaccarelli MSantanam LXiao YMayo C(2024)Perceptions on and roadblocks to implementation of standardized nomenclature in radiation oncology: A survey from TG‐263U1Journal of Applied Clinical Medical Physics10.1002/acm2.14359Online publication date: 30-Apr-2024
https://doi.org/10.1002/acm2.14359
Tang WHearne HSlocum ZChen T(2023)GIS-Based Scientific Workflows for Automated Spatially Driven Sea Level Rise ModelingSustainability10.3390/su15171270415:17(12704)Online publication date: 22-Aug-2023
https://doi.org/10.3390/su151712704
Bai CLiu JTahabilder AImran MLu SChe D(2023)A Generic Efficient Scientific Workflow Engine for the Optimizations of Run-Time Execution2023 IEEE International Conference on Software Services Engineering (SSE)10.1109/SSE60056.2023.00023(98-103)Online publication date: Jul-2023
https://doi.org/10.1109/SSE60056.2023.00023
dos Santos MGrabher GKovaleski MGeyer CCavalheiro G(2023)WCSim: A Cloud Computing Simulator with Support for Bag of Tasks Workflows2023 IEEE 35th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)10.1109/SBAC-PAD59825.2023.00032(230-241)Online publication date: 17-Oct-2023
https://doi.org/10.1109/SBAC-PAD59825.2023.00032
Kejriwal MShen K(2023)A Lightweight Global Taxonomy Induction System for E-Commerce Concept LabelsCyber Security and Social Media Applications10.1007/978-3-031-33065-0_5(99-114)Online publication date: 28-Apr-2023
https://doi.org/10.1007/978-3-031-33065-0_5
Warade MSchneider JLee K(2022)Measuring the Energy and Performance of Scientific Workflows on Low-Power ClustersElectronics10.3390/electronics1111180111:11(1801)Online publication date: 6-Jun-2022
https://doi.org/10.3390/electronics11111801
Chowdhury FDi Natale FMoody AMohror KYu W(2022)DFMan: A Graph-based Optimization of Dataflow Scheduling on High-Performance Computing Systems2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS53621.2022.00043(368-378)Online publication date: May-2022
https://doi.org/10.1109/IPDPS53621.2022.00043
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents