Article

On the Parallel Speed-Up of Estimation of Multivariate Normal Algorithm and Evolution Strategies

Authors:

Fabien Teytaud,

Olivier TeytaudAuthors Info & Claims

EvoWorkshops '09: Proceedings of the EvoWorkshops 2009 on Applications of Evolutionary Computing: EvoCOMNET, EvoENVIRONMENT, EvoFIN, EvoGAMES, EvoHOT, EvoIASP, EvoINTERACTION, EvoMUSART, EvoNUM, EvoSTOC, EvoTRANSLOG

Pages 655 - 664

Published: 11 April 2009 Publication History

Abstract

Motivated by parallel optimization, we experiment EDA-like adaptation-rules in the case of <em>***</em> large. The rule we use, essentially based on estimation of multivariate normal algorithm, is (i) compliant with all families of distributions for which a density estimation algorithm exists (ii) simple (iii) parameter-free (iv) better than current rules in this framework of <em>***</em> large. The speed-up as a function of <em>***</em> is consistent with theoretical bounds.

References

[1]

Baluja, S.: Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning, Technical Report CMU-CS-94-163, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA (1994)

[2]

Beyer, H.-G.: The Theory of Evolution Strategies. Natural Computing Series. Springer, Heidelberg (2001)

[3]

Beyer, H.-G.: The Theory of Evolutions Strategies. Springer, Heidelberg (2001)

[4]

Beyer, H.-G., Sendhoff, B.: Covariance matrix adaptation revisited - the CMSA evolution strategy. In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 123-132. Springer, Heidelberg (2008)

[5]

Cai, Y., Sun, X., Xu, H., Jia, P.: Cross entropy and adaptive variance scaling in continuous eda. In: GECCO 2007: Proceedings of the 9th annual conference on Genetic and evolutionary computation, pp. 609-616. ACM Press, New York (2007)

[6]

Donga, W., Yao, X.: Unified eigen analysis on multivariate gaussian based estimation of distribution algorithms. Information Sciences 178(15), 3000-3023 (2008)

[7]

Grahl, J., Bosman, P.A., Rothlauf, F.: The correlation-triggered adaptive variance scaling idea. In: GECCO 2006: Proceedings of the 8th annual conference on Genetic and evolutionary computation, pp. 397-404. ACM, New York (2006)

[8]

Hansen, N.: Verallgemeinerte individuelle Schrittweitenregelung in der Evolutionsstrategie. Eine Untersuchung zur entstochastisierten, koordinatensystemunabhängigen Adaptation der Mutationsverteilung. Mensch und Buch Verlag, Berlin (1998) ISBN 3-933346-29-0

[9]

Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation 9(2), 159-195 (2001)

[10]

Harik, G.R., Lobo, F.G., Goldberg, D.E.: The compact genetic algorithm. IEEE Trans. on Evolutionary Computation 3(4), 287 (1999)

[11]

Larranaga, P., Lozano, J.A.: Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation. Kluwer Academic Publishers, Dordrecht (2001)

[12]

Liu, J., Teng, H.-F.: Model learning and variance control in continuous edas using pca. In: ICICIC 2008: Proceedings of the 2008 3rd International Conference on Innovative Computing Information and Control, Washington, DC, USA, p. 555. IEEE Computer Society, Los Alamitos (2008)

[13]

Mühlenbein, H., Höns, R.: The estimation of distributions and the minimum relative entropy principle. Evolutionary Computation 13(1), 1-27 (2005)

[14]

Mühlenbein, H., Mahnig, T.: Evolutionary computation and Wright's equation. Theoretical Computer Science 287(1), 145-165 (2002)

[15]

Posík, P.: Preventing premature convergence in a simple eda via global step size setting. In: Rudolph, et al. (ed.) {18}, pp. 549-558

[16]

Rechenberg, I.: Evolutionstrategie: Optimierung Technischer Systeme nach Prinzipien des Biologischen Evolution. Fromman-Holzboog Verlag, Stuttgart (1973)

[17]

Ros, R., Hansen, N.: A simple modification in cma-es achieving linear time and space complexity. In: Rudolph, et al. (ed.) {18}, pp. 296-305

[18]

Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.): PPSN 2008. LNCS, vol. 5199. Springer, Heidelberg (2008)

[19]

Schwefel, H.-P.: Adaptive Mechanismen in der biologischen Evolution und ihr Einfluss auf die Evolutionsgeschwindigkeit. Interner Bericht der Arbeitsgruppe Bionik und Evolutionstechnik am Institut für Mess- und Regelungstechnik Re 215/3, Technische Universität Berlin (July 1974)

[20]

Shapiro, J.L.: Drift and scaling in estimation of distribution algorithms. Evolutionary Computation 13(1) (2005)

[21]

Teytaud, O., Fournier, H.: Lower bounds for evolution strategies using vcdimension. In: Rudolph, et al. (ed.) {18}, pp. 102-111.

Cited By

Belkhir NDréo JSavéant PSchoenauer M(2015)Parameter Setting for Multicore CMA-ES withźLarge PopulationsRevised Selected Papers of the 12th International Conference on Artificial Evolution - Volume 955410.1007/978-3-319-31471-6_9(109-122)Online publication date: 26-Oct-2015
https://dl.acm.org/doi/10.1007/978-3-319-31471-6_9
Teytaud FTeytaud O(2010)Log(λ) modifications for optimal parallelismProceedings of the 11th international conference on Parallel problem solving from nature: Part I10.5555/1885031.1885059(254-263)Online publication date: 11-Sep-2010
https://dl.acm.org/doi/10.5555/1885031.1885059
Jebalia MAuger A(2010)Log-linear convergence of the scale-invariant (µ/µw, λ)-ES and optimal µ for intermediate recombination for large population sizesProceedings of the 11th international conference on Parallel problem solving from nature: Part I10.5555/1885031.1885038(52-62)Online publication date: 11-Sep-2010
https://dl.acm.org/doi/10.5555/1885031.1885038
Show More Cited By

Recommendations

Maximum likelihood estimation for multivariate skew normal mixture models

This paper provides a flexible mixture modeling framework using the multivariate skew normal distribution. A feasible EM algorithm is developed for finding the maximum likelihood estimates of parameters in this context. A general information-based ...
Inference for multivariate normal mixtures

Multivariate normal mixtures provide a flexible model for high-dimensional data. They are widely used in statistical genetics, statistical finance, and other disciplines. Due to the unboundedness of the likelihood function, classical likelihood-based ...
Estimation of the mean vector of a multivariate normal distribution: subspace hypothesis

This paper considers the estimation of the mean vector @q of a p-variate normal distribution with unknown covariance matrix @S when it is suspected that for a pxr known matrix B the hypothesis @q=B@h, @h@?R^r may hold. We consider empirical Bayes ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

EvoWorkshops '09: Proceedings of the EvoWorkshops 2009 on Applications of Evolutionary Computing: EvoCOMNET, EvoENVIRONMENT, EvoFIN, EvoGAMES, EvoHOT, EvoIASP, EvoINTERACTION, EvoMUSART, EvoNUM, EvoSTOC, EvoTRANSLOG

April 2009

827 pages

ISBN:9783642011283

Editors:
Mario Giacobini
Department of Animal Production Epidemiology and Ecology, University of Torino, Grugliasco, Italy 10095
,
Anthony Brabazon
Quinn School of Business, Belfield, University College Dublin, Dublin 4, Ireland
,
Stefano Cagnoni
Department of Computer Engineering, University of Parma, Parma, Italy 43100
,
Gianni A. Caro
Dalla Molle Institute for Artificial Intelligence (IDSIA), Manno-Lugano, Switzerland 6928
,
Anikó Ekárt
School of Engineering and Applied Science,Aston Triangle, Aston University, Birmingham, UK B4 7ET
,
Anna Isabel Esparcia-Alcázar
Complex Adaptive Systems Group, Instituto Tecnológico de Informática, Ciudad PolitŽcnica de la Innovacin, Edif. 8G Acceso B, Valencia, Spain 46022
,
Muddassar Farooq
Next Generation Itelligent Networks Research Center (nexGIN RC), National University of Computer and Emerging Sciences (NUCES), Islamabad, Pakistan
,
Andreas Fink
Faculty of Economics and Social Sciences, Helmut-Schmidt-University, Hamburg, Germany 22043
,
Penousal Machado
Department of Informatics Engineering, Polo II - Pinhal de Marrocos, University of Coimbra, Coimbra, Portugal 3030 - 290

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 11 April 2009

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Belkhir NDréo JSavéant PSchoenauer M(2015)Parameter Setting for Multicore CMA-ES withźLarge PopulationsRevised Selected Papers of the 12th International Conference on Artificial Evolution - Volume 955410.1007/978-3-319-31471-6_9(109-122)Online publication date: 26-Oct-2015
https://dl.acm.org/doi/10.1007/978-3-319-31471-6_9
Teytaud FTeytaud O(2010)Log(λ) modifications for optimal parallelismProceedings of the 11th international conference on Parallel problem solving from nature: Part I10.5555/1885031.1885059(254-263)Online publication date: 11-Sep-2010
https://dl.acm.org/doi/10.5555/1885031.1885059
Jebalia MAuger A(2010)Log-linear convergence of the scale-invariant (µ/µw, λ)-ES and optimal µ for intermediate recombination for large population sizesProceedings of the 11th international conference on Parallel problem solving from nature: Part I10.5555/1885031.1885038(52-62)Online publication date: 11-Sep-2010
https://dl.acm.org/doi/10.5555/1885031.1885038
Teytaud F(2010)A new selection ratio for large population sizesProceedings of the 2010 international conference on Applications of Evolutionary Computation - Volume Part I10.1007/978-3-642-12239-2_47(452-460)Online publication date: 7-Apr-2010
https://dl.acm.org/doi/10.1007/978-3-642-12239-2_47

View Options

View options

Media

Figures

Other

Tables

View Table of Contents