poster

A view of estimation of distribution algorithms through the lens of expectation-maximization

Authors:

Jennifer ListgartenAuthors Info & Claims

GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion

Pages 189 - 190

https://doi.org/10.1145/3377929.3389938

Published: 08 July 2020 Publication History

Get Access

Abstract

We show that a large class of Estimation of Distribution Algorithms, including, but not limited to, Covariance Matrix Adaption, can be written as a Monte Carlo Expectation-Maximization algorithm, and as exact EM in the limit of infinite samples. Because EM sits on a rigorous statistical foundation and has been thoroughly analyzed, this connection provides a new coherent framework with which to reason about EDAs.

References

[1]

Sivaraman Balakrishnan, Martin J. Wainwright, and Bin Yu. 2017. Statistical guarantees for the EM algorithm: From population to sample-based analysis. Ann. Statist. 45, 1 (02 2017), 77--120.

Crossref

Google Scholar

[2]

David H. Brookes, Akosua Busia, Clara Fannjiang, Kevin Murphy, and Jennifer Listgarten. 2019. A view of Estimation of Distribution Algorithms through the lens of Expectation-Maximization. (2019). arXiv:cs.LG/1905.10474

Google Scholar

[3]

A. P. Dempster, N. M. Laird, and D. B. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B 39, 1 (1977), 1--38.

Crossref

Google Scholar

[4]

G.R. Grimmett and D.R. Stirzaker. 2001. Probability and random processes. Vol. 80. Oxford university press.

Google Scholar

[5]

Nikolaus Hansen, Sibylle D. Müller, and Petros Koumoutsakos. 2003. Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES). Evol. Comput. 11, 1 (March 2003), 1--18.

Digital Library

Google Scholar

[6]

Nikolaus Hansen and Andreas Ostermeier. 2001. Completely Derandomized Self-Adaptation in Evolution Strategies. Evol. Comput. 9, 2 (June 2001), 159--195.

Digital Library

Google Scholar

[7]

Radford M. Neal and Geoffrey E. Hinton. 1999. Learning in Graphical Models. MIT Press, Cambridge, MA, USA, Chapter A View of the EM Algorithm That Justifies Incremental, Sparse, and Other Variants, 355--368. http://dl.acm.org/citation.cfm?id=308574.308679

Google Scholar

[8]

Daan Wierstra, Tom Schaul, Tobias Glasmachers, Yi Sun, Jan Peters, and Jürgen Schmidhuber. 2014. Natural Evolution Strategies. Journal of Machine Learning Research 15 (2014), 949--980. http://jmlr.org/papers/v15/wierstra14a.html

Digital Library

Google Scholar

[9]

D. Wierstra, T. Schaul, J. Peters, and J. Schmidhuber. 2008. Natural Evolution Strategies. In 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence). 3381--3387.

Crossref

Google Scholar

[10]

Sun Yi, Daan Wierstra, Tom Schaul, and Jürgen Schmidhuber. 2009. Stochastic Search Using the Natural Gradient. In Proceedings of the 26th Annual International Conference on Machine Learning (ICML '09). ACM, New York, NY, USA, 1161--1168.

Digital Library

Google Scholar

Cited By

View all

Fannjiang CListgarten J(2023)Is Novelty Predictable?Cold Spring Harbor Perspectives in Biology10.1101/cshperspect.a04146916:2(a041469)Online publication date: 5-Dec-2023
https://doi.org/10.1101/cshperspect.a041469
Fontaine MNikolaidis S(2022)Evaluating Human–Robot Interaction Algorithms in Shared Autonomy via Quality Diversity Scenario GenerationACM Transactions on Human-Robot Interaction10.1145/347641211:3(1-30)Online publication date: 2-Sep-2022
https://dl.acm.org/doi/10.1145/3476412
Matabuena MRodríguez-Mier PGarcía-Meixide CLeborán V(2021)COVID-19Computer Methods and Programs in Biomedicine10.1016/j.cmpb.2021.106399211:COnline publication date: 1-Nov-2021
https://dl.acm.org/doi/10.1016/j.cmpb.2021.106399
Show More Cited By

Index Terms

A view of estimation of distribution algorithms through the lens of expectation-maximization
1. Theory of computation
  1. Design and analysis of algorithms
    1. Mathematical optimization
      1. Continuous optimization
        Nonconvex optimization

Recommendations

An Expectation-Maximization Algorithm for Blind Separation of Noisy Mixtures Using Gaussian Mixture Model

In this paper, we propose a new expectation-maximization (EM) algorithm, named GMM-EM, to blind separation of noisy instantaneous mixtures, in which the non-Gaussianity of independent sources is exploited by modeling their distribution using the ...
PARAFAC-Based Blind Identification of Underdetermined Mixtures Using Gaussian Mixture Model

This paper presents a novel algorithm, named GMM-PARAFAC, for blind identification of underdetermined instantaneous linear mixtures. The GMM-PARAFAC algorithm uses Gaussian mixture model (GMM) to model non-Gaussianity of the independent sources. We show ...
Attraction-repulsion expectation-maximization algorithm for image reconstruction and sensor field estimation

In this paper, we propose an attraction-repulsion expectation-maximization (AREM) algorithm for image reconstruction and sensor field estimation. We rely on a new method for density estimation to address the problems of image reconstruction from limited ...

Comments

Information & Contributors

Information

Published In

GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion

July 2020

1982 pages

ISBN:9781450371278

DOI:10.1145/3377929

General Chair:
Carlos Artemio Coello Coello
CINVESTAV-IPN

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 July 2020

Check for updates

Author Tags

Qualifiers

Poster

Conference

GECCO '20

Sponsor:

SIGEVO

GECCO '20: Genetic and Evolutionary Computation Conference

July 8 - 12, 2020

Cancún, Mexico

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
160
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)0

Reflects downloads up to

Other Metrics

View Author Metrics

Citations

Cited By

View all

Fannjiang CListgarten J(2023)Is Novelty Predictable?Cold Spring Harbor Perspectives in Biology10.1101/cshperspect.a04146916:2(a041469)Online publication date: 5-Dec-2023
https://doi.org/10.1101/cshperspect.a041469
Fontaine MNikolaidis S(2022)Evaluating Human–Robot Interaction Algorithms in Shared Autonomy via Quality Diversity Scenario GenerationACM Transactions on Human-Robot Interaction10.1145/347641211:3(1-30)Online publication date: 2-Sep-2022
https://dl.acm.org/doi/10.1145/3476412
Matabuena MRodríguez-Mier PGarcía-Meixide CLeborán V(2021)COVID-19Computer Methods and Programs in Biomedicine10.1016/j.cmpb.2021.106399211:COnline publication date: 1-Nov-2021
https://dl.acm.org/doi/10.1016/j.cmpb.2021.106399
Schneider PWalters WPlowright ASieroka NListgarten JGoodnow RFisher JJansen JDuca JRush TZentgraf MHill JKrutoholow EKohler MBlaney JFunatsu KLuebkemann CSchneider G(2019)Rethinking drug design in the artificial intelligence eraNature Reviews Drug Discovery10.1038/s41573-019-0050-319:5(353-364)Online publication date: 4-Dec-2019
https://doi.org/10.1038/s41573-019-0050-3

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

An Expectation-Maximization Algorithm for Blind Separation of Noisy Mixtures Using Gaussian Mixture Model

PARAFAC-Based Blind Identification of Underdetermined Mixtures Using Gaussian Mixture Model

Attraction-repulsion expectation-maximization algorithm for image reconstruction and sensor field estimation

Comments

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

An Expectation-Maximization Algorithm for Blind Separation of Noisy Mixtures Using Gaussian Mixture Model

PARAFAC-Based Blind Identification of Underdetermined Mixtures Using Gaussian Mixture Model

Attraction-repulsion expectation-maximization algorithm for image reconstruction and sensor field estimation

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations