research-article

A distributed evolutionary method to design scheduling policies for volunteer computing

Authors:

Trilce Estrada,

Michela TauferAuthors Info & Claims

CF '08: Proceedings of the 5th conference on Computing frontiers

Pages 313 - 322

https://doi.org/10.1145/1366230.1366282

Published: 05 May 2008 Publication History

Abstract

Volunteer Computing (VC) is a paradigm that uses idle cycles from computing resources donated by volunteers and connected through the Internet to compute large-scale, loosely-coupled simulations. A big challenge in VC projects is the scheduling of work-units across heterogeneous, volatile, and error-prone computers. The design of effective scheduling policies for VC projects involves subjective and time-demanding tuning that is driven by the knowledge of the project designer. VC projects are in need of a faster and project-independent method to automate the scheduling design.

To automatically generate a scheduling policy, we must explore the extremely large space of syntactically valid policies. Given the size of this search space, exhaustive search is not feasible. Thus in this paper we propose to solve the problem using an evolutionary method to automatically generate a set of scheduling policies that are project-independent, minimize errors, and maximize throughput in VC projects. Our method includes a genetic algorithm where the representation of individuals, the fitness function, and the genetic operators are specifically tailored to get effective policies in a short time. The effectiveness of our method is evaluated with SimBA, a Simulator of BOINC Applications. Contrary to manually-designed scheduling policies that often perform well only for the specific project they were designed for and require months of tuning, our resulting scheduling policies provide better overall throughput across the different VC projects considered in this work and were generated by our method in a time window of one week.

References

[1]

D. Anderson, E. Corpela, and R. Walton. High-performance task distribution for volunteer computing. In Proc. of the 1st IEEE Int. Conference on e-Science and Grid Technologies, 2005.

Digital Library

[2]

F. Bellosa. Locality-information-based scheduling in shared-memory multiprocessors. In Job Scheduling Strategies for Parallel Processing, volume 1162 of Lecture Notes in Computer Science, pages 271--289. 1996.

Digital Library

[3]

T. Blickle and L. Thiele. A mathematical analysis of tournament selection. In Proceedings of the 6th Int. Conference on Genetic Algorithms, 1995.

Digital Library

[4]

J. Cavazos and E. B. Moss. Inducing heuristics to decide whether to schedule. In Proc. of the ACM SIGPLAN 04 Conference on Programming Language Design and Implementation, 2004.

Digital Library

[5]

K. A. De-Jong and W. M. Spears. Learning Concept Classification Rules using Genetic Algorithms. In Proc. of the 12th Int. Conference on Artificial Intelligence IJCAI-91, 1991.

[6]

C. Dimopoulos and A. M. S. Zalzala. A genetic programming heuristic for the one-machine total tardiness problem. In Proc. of the Congress on Evolutionary Computation, 1999.

[7]

C. Dimopoulos and A. M. S. Zalzala. Investigating the use of genetic programming for a classic one-machine scheduling problem. Advances in Engineering Software, 32:489--498, 6 2001.

[8]

T. Estrada, D. Flores, M. Taufer, P. J. Teller, A. Kerstens, and D. Anderson. The effectiveness of threshold-based scheduling policies in BOINC projects. In Proc. of the 2nd IEEE Int. Conference on e-Science and Grid Technologies (eScience), 2006.

Digital Library

[9]

E. Hart, P. Ross, and D. Corne. Evolutionary scheduling: A review. Genetic Programming and Evolvable Machines, 6(2):191--220, 2005.

Digital Library

[10]

J. H. Holland. Genetic algorithms and classifier systems: foundations and future directions. In Proc. of the 2nd Int. Conference on Genetic Algorithms and their Application, 1987.

Digital Library

[11]

D. Jakobović and L. Budin. Dynamic scheduling with genetic programming. In Proc. of the 9th European Conference on Genetic Programming, 2006.

Digital Library

[12]

D. Jakobović, L. Jelenković, and L. Budin. Genetic programming heuristics for multiple machine scheduling. In Proc. of the 10th European Conference on Genetic Programming, 2007.

Digital Library

[13]

J. R. Koza. Concept formation and decision tree induction using the genetic programming paradigm. In Proc. of 1st Workshop on Self Adaptivity in Grid Computing, 1991.

Digital Library

[14]

A. D. MacKerell Jr., B. Brooks, C. L. Brooks III, L. N. B. Roux, Y. Won, and M. Karplus. CHARMM: The Energy Function and Its Parametrization with an Overview of the Program, volume 1. John Wiley & Sons, 1998.

[15]

M. J. Oudshoorn and L. Huang. Evolving toward an optimal scheduling solution through adaptivity. J. Parallel Distrib. Comput., 62(7):1203--1222, 2002.

Digital Library

[16]

V. S. Pande et al. Atomistic protein folding simulations on the submillisecond time scale using worldwide distributed computing. Biopolymers, 68(91), 2003.

[17]

D. Puppin. Adapting convergent scheduling using machine learning. In Proc. of the 9th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 2003.

Digital Library

[18]

F. Seredynski and A. Y. Zomaya. Coevolution and evolving parallel cellular automata - based scheduling algorithms. In Selected Papers from the 5th European Conference on Artificial Evolution, pages 362--374. Springer-Verlag, 2002.

Digital Library

[19]

J. Skolnick, A. Kolinski, and A. R. Ortiz. Monsster: a method for folding globular proteins with a small number of distance restraints. J. of Molecular Biology, 265(2):217--241, 1997.

[20]

M. Stephenson, S. Amarasinghe, M. Martin, and U.-M. O'Reilly. Meta optimization: improving compiler heuristics with machine learning. SIGPLAN Not., 38(5):77--90, 2003.

Digital Library

[21]

M. Taufer, C. An, A. Kerstens, and C. L. Brooks III. Predictor@home: A protein structure prediction supercomputer based on global computing. IEEE Transactions on Parallel and Distributed Systems, 17(8):786--796, 2006.

Digital Library

[22]

M. Taufer, D. P. Anderson, P. Cicotti, and C. L. Brooks. Homogeneous redundancy: a technique to ensure integrity of molecularsimulation results using public computing. In Proc. of the IEEE Int. Parallel and Distributed Processing Symposium (IPDPS'05), 2005.

Digital Library

[23]

M. Taufer, A. Kerstens, T. Estrada, D. A. Flores, and P. J. Teller. SimBA: a discrete event simulator for performance prediction of volunteer computing projects. In Proc. of the Int. Workshop on Principles of Advanced and Distributed Simulation (PADS'07), 2007.

Digital Library

[24]

C. Zhou, W. Xiao, T. M. Tirpak, and P. C. Nelson. Evolving accurate and compact classification rules with gene expression programming. IEEE Transactions on Evolutionary Computation, 7(6):519--531, 2003.

Digital Library

Cited By

Panadero Jde Armas JSerra XMarquès J(2018)Multi criteria biased randomized method for resource allocation in distributed systems: Application in a volunteer computing systemFuture Generation Computer Systems10.1016/j.future.2017.11.03982(29-40)Online publication date: May-2018
https://doi.org/10.1016/j.future.2017.11.039
Panadero JCalvet LMarquès JJuan A(2017)A simheuristic approach for resource allocation in volunteer computingProceedings of the 2017 Winter Simulation Conference10.5555/3242181.3242301(1-12)Online publication date: 3-Dec-2017
https://dl.acm.org/doi/10.5555/3242181.3242301
Panadero JCalvet LMarques JJuan A(2017)A simheuristic approach for resource allocation in volunteer computing2017 Winter Simulation Conference (WSC)10.1109/WSC.2017.8247890(1479-1490)Online publication date: Dec-2017
https://doi.org/10.1109/WSC.2017.8247890
Show More Cited By

Index Terms

A distributed evolutionary method to design scheduling policies for volunteer computing
1. Computing methodologies
  1. Artificial intelligence
    1. Planning and scheduling
    2. Search methodologies
      1. Heuristic function construction
  2. Modeling and simulation
2. Theory of computation
  1. Design and analysis of algorithms
    1. Algorithm design techniques
      1. Dynamic programming

Recommendations

A distributed evolutionary method to design scheduling policies for volunteer computing

Volunteer Computing (VC) is a paradigm that takes advantage of idle cycles from computing resources donated by volunteers and connected through the Internet to compute large-scale, loosely coupled simulations. A big challenge in VC projects is the ...
Optimisation of distributed manufacturing flexible job shop scheduling by using hybrid genetic algorithms

In contrast to traditional job-shop scheduling problems, various complex constraints must be considered in distributed manufacturing environments; therefore, developing a novel scheduling solution is necessary. This paper proposes a hybrid genetic ...
A new genetic algorithm for lot-streaming flow shop scheduling with limited capacity buffers

Lot-streaming is the process of splitting a job (lot) into a number of smaller sublots to allow the overlapping of operations between successive machines in a multi-stage production system. The use of sublots usually results in substantially shorter job ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CF '08: Proceedings of the 5th conference on Computing frontiers

May 2008

334 pages

ISBN:9781605580777

DOI:10.1145/1366230

General Chair:
Alex Ramirez
UPC, Spain
,
Program Chairs:
Gianfranco Biliardi
University of Padova, Italy
,
Michael Gschwind
IBM TJ Watson Research Center, USA

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 May 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CF '08

Sponsor:

CF '08: Computing Frontiers Conference

May 5 - 7, 2008

Ischia, Italy

Acceptance Rates

Overall Acceptance Rate 273 of 785 submissions, 35%

Upcoming Conference

CF '25

Sponsor:
sigmicro

22nd ACM International Conference on Computing Frontiers

May 28 - 30, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
234
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Panadero Jde Armas JSerra XMarquès J(2018)Multi criteria biased randomized method for resource allocation in distributed systems: Application in a volunteer computing systemFuture Generation Computer Systems10.1016/j.future.2017.11.03982(29-40)Online publication date: May-2018
https://doi.org/10.1016/j.future.2017.11.039
Panadero JCalvet LMarquès JJuan A(2017)A simheuristic approach for resource allocation in volunteer computingProceedings of the 2017 Winter Simulation Conference10.5555/3242181.3242301(1-12)Online publication date: 3-Dec-2017
https://dl.acm.org/doi/10.5555/3242181.3242301
Panadero JCalvet LMarques JJuan A(2017)A simheuristic approach for resource allocation in volunteer computing2017 Winter Simulation Conference (WSC)10.1109/WSC.2017.8247890(1479-1490)Online publication date: Dec-2017
https://doi.org/10.1109/WSC.2017.8247890
Estrada TWyatt MTaufer M(2015)A Genetic Programming Approach to Design Resource Allocation Policies for Heterogeneous Workflows in the CloudProceedings of the 2015 IEEE 21st International Conference on Parallel and Distributed Systems (ICPADS)10.1109/ICPADS.2015.54(372-379)Online publication date: 14-Dec-2015
https://dl.acm.org/doi/10.1109/ICPADS.2015.54
Taufer MArmen RJianhan Chen Teller PBrooks C(2009)Computational multiscale modeling in protein--ligand dockingIEEE Engineering in Medicine and Biology Magazine10.1109/MEMB.2009.93178928:2(58-69)Online publication date: Mar-2009
https://doi.org/10.1109/MEMB.2009.931789
Heien EAnderson DHagihara K(2009)Computing Low Latency Batches with Unreliable Workers in Volunteer Computing EnvironmentsJournal of Grid Computing10.1007/s10723-009-9131-67:4(501-518)Online publication date: 25-Aug-2009
https://doi.org/10.1007/s10723-009-9131-6

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents