Article

Automatic heuristic generation with genetic programming: evolving a jack-of-all-trades or a master of one

Authors:

Edmund K. Burke,

Matthew R. Hyde,

Graham Kendall,

John WoodwardAuthors Info & Claims

GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computation

Pages 1559 - 1565

https://doi.org/10.1145/1276958.1277273

Published: 07 July 2007 Publication History

GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computation

Automatic heuristic generation with genetic programming: evolving a jack-of-all-trades or a master of one

Pages 1559 - 1565

Abstract
References

Abstract

It is possible to argue that online bin packing heuristics should be evaluated by using metrics based on their performance over the set of all bin packing problems, such as the worst case or average case performance. However, this method of assessing a heuristic would only be relevant to a user who employs the heuristic over a set of problems which is actually representative of the set of all possible bin packing problems. On the other hand, a real world user will often only deal with packing problems that are representative of a particular sub-set. Their piece sizes will all belong to a particular distribution. The contribution of this paper is to show that a Genetic Programming system can automate the process of heuristic generation and produce heuristics that are human-competitive over a range of sets of problems, or which excel on a particular sub-set. We also show that the choice of training instances is vital in the area of automatic heuristic generation, due to the trade-off between the performance and generality of the heuristics generated and their applicability to new problems.

References

[1]

E. K. Burke, E. Hart, G. Kendall, J. Newall, P. Ross, and S. Schulenburg. Hyper-heuristics: An emerging direction in modern search technology. InF. Glover and G. Kochenberger, editors, Handbook of Meta-Heuristics pages 457--474. Kluwer, 2003.

[2]

E. K. Burke, M. Hyde, and G. Kendall. Evolving bin packing heuristics with genetic programming. In T. Runarsson, H.-G. Beyer, E. Burke, J. J. Merelo-Guervos, D. Whitley, and X. Yao, editors, LNCS 4193, Proceedings of the 9th International Conference on Parallel Problem Solving from Nature (PPSN 2006) pages 860--869, Reykjavik, Iceland, September 2006.

Digital Library

[3]

E. k. Burke, M. R. Hyde, and G. Kendall. Evolving bin packing heuristics with genetic programming. In T. Runarsson, H.-G. Beyer, E. Burke, J. J. Merelo-Guervos, D. Whitley, and X. Yao, editors, LNCS 4193, Proceedings of the 9th International Conference on Parallel Problem Solving from Nature(PPSN 2006) pages 860--869, Reykjavik, Iceland, September 2006.

Digital Library

[4]

E. K. Burke, G. Kendall, and E. Soubeiga. A tabu-search hyper-heuristic for timetabling and rostering. Journal of Heuristics 9(6) :451--470, 2003.

Digital Library

[5]

E. K. Burke, B. McCollum, A. M. S. Petrovic, and R. Qu. A graph-based hyper heuristic for timetabling problems. European Journal of Operational Research 176:177--192, 2007.

[6]

E. K. Burke, S. Petrovic, and R. Qu. Case-based heuristic selection for timetabling problems. Journal of Scheduling 9(2) :115--132, 2006.

Digital Library

[7]

E. G. Co. man Jr, G. Galambos, S. Martello, and D. Vigo. Bin packing approximation algorithms: Combinatorial analysis. In D. Z. Du and P. M. Pardalos, editors, Handbook of Combinatorial Optimization Kluwer, 1998.

[8]

P. Cowling, G. Kendall, and E. Soubeiga. A hyperheuristic approach to scheduling a sales summit. In E. K. Burke and W. Erben, editors, Proceedings of the 3rd International Conference on the Practice and Theory of Automated Timetabling (PATAT 2000) pages 176--190, Konstanz, Germany, August 2000.

Digital Library

[9]

K. Dowsland, E. Soubeiga, and E. K. Burke. A simulated annealing hyper-heuristic for determining shipper sizes. European Journal of Operational Research 179(3) :759--774, 2007.

[10]

E. Falkenauer and A. Delchambre. A genetic algorithm for bin packing and line balancing. In Proceedings of the IEEE 1992 Int. Conference on Robotics and Automation Nice, France, May 1992.

[11]

A. Fukunaga. Automated discovery of composite sat variable-selection heuristics. In Eighteenth national conference on Artificial intelligence pages 641--648, Menlo Park, CA, USA, 2002. American Association for Artificial Intelligence.

Digital Library

[12]

A. S. Fukunaga. Evolving local search heuristics for SAT using genetic programming. In K. Deb, R. Poli, W. Banzhaf, H.-G. Beyer, E. Burke, P. Darwen, D. Dasgupta, D. Floreano, J. Foster, M. Harman, O. Holland, P. L. Lanzi, L. Spector, A. Tettamanzi, D. Thierens, and A. Tyrrell, editors, Proceedings of the Genetic and Evolutionary Computation Conference 2004 (GECCO '04) volume 3103, pages 483--494, Seattle, WA, USA, 2004. Springer-Verlag.

[13]

M. R. Garey and D. S. Johnson. Computers and Intractability: A Guide to the Theory of NP-Completeness W. H. Freeman and Company, San Fransisco, 1979.

Digital Library

[14]

D. Johnson, A. Demers, J. Ullman, M. Garey, and R. Graham. Worst-case performance bounds for simple one-dimensional packaging algorithms. SIAM Journal on Computing 3(4):299--325, December 1974.

[15]

Kenyon. Best-fit bin-packing with random order. In Proceedings of the Seventh Annual ACM-SIAM Symposium on Discrete Algorithms 1996.

Digital Library

[16]

J. R. Koza. Genetic Programming: on the Programming of Computers by Means of Natural Selection The MIT Press, Boston, Massachusetts, 1992.

Digital Library

[17]

J. R. Koza and R. Poli. Genetic programming. In E. K. Burke and G. Kendall, editors, Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques pages 127--164. Springer, Boston, 2005.

[18]

S. Martello and P. Toth. Knapsack Problems: Algorithms and Computer Implementations John Wiley and Sons, Chichester, 1990.

Digital Library

[19]

M. O'Neill, R. Cleary, and N. Nikolov. Solving knapsack problems with attribute grammars. In Proceedings of the Third Grammatical Evolution Workshop (GEWS'04) Seattle, WA, USA., 2004.

[20]

E. Ozcan, B. Bilgin, and E. E. Korkmaz. Hill climbers and mutational heuristics in hyperheuristics. In T. Runarsson, H.-G. Beyer, E. Burke, J. J. Merelo-Guervos, D. Whitley, and X. Yao, editors, LNCS 4193, Proceedings of the 9th International Conference on Parallel Problem Solving from Nature (PPSN 2006) pages 202--211, Reykjavik, Iceland, September 2006.

Digital Library

[21]

R. Poli. A simple but theoretically-motivated method to control bloat in genetic programming. In C. Ryan, T. Soule, M. Keijzer, E. Tsang, R. Poli, and E. Costa, editors, Genetic Programming, Proceedings of the 6th European Conference, EuroGP 2003 pages 211--223, Essex, April 2003. Springer-Verlag.

[22]

P. Rattadilok, A. Gaw, and R. Kwan. Distributed choice function hyper-heuristics for timetabling and scheduling. In E. Burke and M. Trick, editors, Practice and Theory of Automated Timetabling V, Springer Lecture notes in Computer Science volume 3616, pages 51--67, 2005.

Digital Library

[23]

W. T. Rhee and M. Talagrand. On line bin packing with items of random size. Math. Oper. Res. 18:438--445, 1993.

[24]

P. Ross. Hyper-heuristics. In E. K. Burke and G. Kendall, editors, Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques pages 529--556. Springer, Boston, 2005.

[25]

P. Ross, J. G. Marin-Blazquez, S. Schulenburg, and E. Hart. Learning a procedure that can solve hard bin-packing problems:A new ga-based approach to hyperheurstics. In Proceedings of the Genetic and Evolutionary Computation Conference 2003 (GECCO'03) pages 1295--1306, Chicago, Illinois, 2003.

[26]

P. Ross, S. Schulenburg, J. G. Marin-Blazquez, and E. Hart. Hyper heuristics:Learning to combine simple heuristics in bin packing problems. In Proceedings of the Genetic and Evolutionary Computation Conference 2002 (GECCO '02) 2002.

[27]

E. Soubeiga. Development and Application of Hyperheuristics to Personnel Scheduling PhDthesis, Univesity of Nottingham, School of Computer Science, 2003.

[28]

D. Whitley and J. P. Watson. Complexity theory and the no free lunch theorem. In E. K. Burke and G. Kendall, editors, Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques Kluwer, Boston, 2005.

[29]

D. H. Wolpert and W. G. Macready. No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation 4:67--82, 1997.

Digital Library

Cited By

Biemer CCooper S(2024)Solution Path Heuristics for Predicting Difficulty and Enjoyment Ratings of Roguelike Level SegmentsProceedings of the 19th International Conference on the Foundations of Digital Games10.1145/3649921.3659846(1-8)Online publication date: 21-May-2024
https://dl.acm.org/doi/10.1145/3649921.3659846
Tauritz DWoodward JLi XHandl J(2024)Generative Hyper-heuristicsProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3648417(1069-1095)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3648417
Tauritz DWoodward JSilva SPaquete L(2023)Generative Hyper-heuristicsProceedings of the Companion Conference on Genetic and Evolutionary Computation10.1145/3583133.3595033(1069-1098)Online publication date: 15-Jul-2023
https://dl.acm.org/doi/10.1145/3583133.3595033
Show More Cited By

Automatic heuristic generation with genetic programming: evolving a jack-of-all-trades or a master of one
1. Computing methodologies
  1. Artificial intelligence
    1. Search methodologies

Recommendations

Automatic Discovery of Optimisation Search Heuristics for Two Dimensional Strip Packing Using Genetic Programming
Simulated Evolution and Learning
Abstract
This paper presents a genetic programming based hyper-heuristic (GPHH) for automatic discovery of optimisation heuristics for the two dimensional strip packing problem (2D-SPP). The novelty of this method is to integrate both the construction and ...
Solving the one-dimensional bin packing problem with a weight annealing heuristic

In this paper, we develop a new procedure that uses the concept of weight annealing to solve the one-dimensional bin packing problem. Our procedure is straightforward and easy to follow. We apply it to 1587 instances taken from benchmark problem sets ...
Genetically designed heuristics for the bin packing problem
GECCO '07: Proceedings of the 9th annual conference companion on Genetic and evolutionary computation

The bin packing problem (BPP) is a real-world problem that arises in different industrial applications related to minimization of space or time. The aim of this research is to automatically design an algorithm that places a collection of objects into ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computation

July 2007

2313 pages

ISBN:9781595936974

DOI:10.1145/1276958

General Chair:
Hod Lipson
Cornell University, USA

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

GECCO07

Sponsor:

GECCO07: Genetic and Evolutionary Computation Conference

July 7 - 11, 2007

London, England

Acceptance Rates

GECCO '07 Paper Acceptance Rate 266 of 577 submissions, 46%;

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

89
Total Citations
View Citations
528
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)1

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Biemer CCooper S(2024)Solution Path Heuristics for Predicting Difficulty and Enjoyment Ratings of Roguelike Level SegmentsProceedings of the 19th International Conference on the Foundations of Digital Games10.1145/3649921.3659846(1-8)Online publication date: 21-May-2024
https://dl.acm.org/doi/10.1145/3649921.3659846
Tauritz DWoodward JLi XHandl J(2024)Generative Hyper-heuristicsProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3648417(1069-1095)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3648417
Tauritz DWoodward JSilva SPaquete L(2023)Generative Hyper-heuristicsProceedings of the Companion Conference on Genetic and Evolutionary Computation10.1145/3583133.3595033(1069-1098)Online publication date: 15-Jul-2023
https://dl.acm.org/doi/10.1145/3583133.3595033
Romera-Paredes BBarekatain MNovikov ABalog MKumar MDupont ERuiz FEllenberg JWang PFawzi OKohli PFawzi A(2023)Mathematical discoveries from program search with large language modelsNature10.1038/s41586-023-06924-6625:7995(468-475)Online publication date: 14-Dec-2023
https://doi.org/10.1038/s41586-023-06924-6
Stanovov VAkhmedova SSemenkin E(2022)Neuroevolution for Parameter Adaptation in Differential EvolutionAlgorithms10.3390/a1504012215:4(122)Online publication date: 7-Apr-2022
https://doi.org/10.3390/a15040122
Tauritz DWoodward JWagner M(2022)Generative hyper-heuristicsProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3520304.3533646(1111-1140)Online publication date: 9-Jul-2022
https://dl.acm.org/doi/10.1145/3520304.3533646
Singh EPillay N(2022)A Parameter-Based Analysis of Ant-Based Generation Hyper-Heuristics2022 IEEE Symposium Series on Computational Intelligence (SSCI)10.1109/SSCI51031.2022.10022101(812-819)Online publication date: 4-Dec-2022
https://doi.org/10.1109/SSCI51031.2022.10022101
Hiruta YNishihara KKoguma YFujii MNakata M(2022)Automatic Construction of Loading Algorithms With Interactive Genetic ProgrammingIEEE Access10.1109/ACCESS.2022.322554310(125167-125180)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3225543
Stanovov VAkhmedova SSemenkin E(2022)The automatic design of parameter adaptation techniques for differential evolution with genetic programmingKnowledge-Based Systems10.1016/j.knosys.2021.108070239:COnline publication date: 5-Mar-2022
https://dl.acm.org/doi/10.1016/j.knosys.2021.108070
Mellouli OHafidi IMetrane A(2021)A modified choice function hyper-heuristic with Boltzmann functionMathematical Modeling and Computing10.23939/mmc2021.04.7368:4(736-746)Online publication date: 2021
https://doi.org/10.23939/mmc2021.04.736
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten