research-article

Open access

Is there anisotropy in structural bias?

Authors:

Diederick Vermetten,

Anna V. Kononova,

Fabio Caraffini,

Thomas BäckAuthors Info & Claims

GECCO '21: Proceedings of the Genetic and Evolutionary Computation Conference Companion

Pages 1243 - 1250

https://doi.org/10.1145/3449726.3463218

Published: 08 July 2021 Publication History

Abstract

Structural Bias (SB) is an important type of algorithmic deficiency within iterative optimisation heuristics. However, methods for detecting structural bias have not yet fully matured, and recent studies have uncovered many interesting questions. One of these is the question of how structural bias can be related to anisotropy. Intuitively, an algorithm that is not isotropic would be considered structurally biased. However, there have been cases where algorithms appear to only show SB in some dimensions. As such, we investigate whether these algorithms actually exhibit anisotropy, and how this impacts the detection of SB. We find that anisotropy is very rare, and even in cases where it is present, there are clear tests for SB which do not rely on any assumptions of isotropy, so we can safely expand the suite of SB tests to encompass these kinds of deficiencies not found by the original tests.

We propose several additional testing procedures for SB detection and aim to motivate further research into the creation of a robust portfolio of tests. This is crucial since no single test will be able to work effectively with all types of SB we identify.

References

[1]

Yoav Benjamini and Daniel Yekutieli. 2001. The Control of the False Discovery Rate in Multiple Testing under Dependency. The Annals of Statistics 29, 4 (2001), 1165--1188.

[2]

Fabio Caraffini and Giovanni Iacca. 2020. The SOS Platform: Designing, Tuning and Statistically Benchmarking Optimisation Algorithms. Mathematics 8, 5 (May 2020), 785.

[3]

Fabio Caraffini, Anna V. Kononova, and David W. Corne. 2019. Infeasibility and structural bias in differential evolution. Information Sciences 496 (2019), 161--179.

Digital Library

[4]

Sandor Csorgo and Julian J. Faraway. 1996. The Exact and Asymptotic Distributions of Cramer-von Mises Statistics. Journal of the Royal Statistical Society. Series B (Methodological) 58, 1 (1996), 221--234.

[5]

Swagatam Das, Sankha Subhra Mullick, and P.N. Suganthan. 2016. Recent advances in differential evolution - An updated survey. Swarm and Evolutionary Computation 27 (2016), 1 -- 30.

[6]

Julian Faraway, George Marsaglia, John Marsaglia, and Adrian Baddeley. [n.d.]. goftest: Classical Goodness-of-Fit Tests for Univariate Distributions. https://CRAN.R-project.org/package=goftest

[7]

David E Goldberg and John Henry Holland. 1988. Genetic algorithms and machine learning. (1988).

[8]

Nikolaus Hansen and Anne Auger. 2014. Principled Design of Continuous Stochastic Search: From Theory to Practice. In Theory and Principled Methods for the Design of Metaheuristics, Yossi Borenstein and Alberto Moraglio (Eds.). Springer, 145--180.

[9]

Nikolaus Hansen and Andreas Ostermeier. 2001. Completely Derandomized Self-Adaptation in Evolution Strategies. Evolutionary Computation 9, 2 (2001), 159--195.

Digital Library

[10]

Charles R. Harris, K. Jarrod Millman, St'efan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fern'andez del R'io, Mark Wiebe, Pearu Peterson, Pierre G'erard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. 2020. Array programming with NumPy. Nature 585, 7825 (Sept. 2020), 357--362.

[11]

Giovanni Iacca and Fabio Caraffini. 2020. Re-sampled inheritance compact optimization. Knowledge-Based Systems 208 (2020), 106416.

[12]

James Kennedy and Russell Eberhart. 1995. Particle swarm optimization. In Proceedings of ICNN'95-international conference on neural networks, Vol. 4. IEEE, 1942--1948.

[13]

Anna V. Kononova, Fabio Caraffini, and Thomas Bäck. 2020. Differential evolution outside the box results. https://arxiv.org/abs/2004.10489.

[14]

Anna V. Kononova, Fabio Caraffini, Hao Wang, and Thomas Bäck. 2020. Can Compact Optimisation Algorithms Be Structurally Biased?. In Parallel Problem Solving from Nature - PPSN XVI, T. Bäck, M. Preuss, A. Deutz, H. Wang, C. Doerr, M. Emmerich, and H. Trautmann (Eds.). Springer International Publishing, Cham, 229--242.

Digital Library

[15]

Anna V. Kononova, Fabio Caraffini, Hao Wang, and Thomas Bäck. 2020. Can Single Solution Optimisation Methods Be Structurally Biased?. In 2020 IEEE Congress on Evolutionary Computation (CEC). IEEE, Glasgow, 1--9.

Digital Library

[16]

Anna V. Kononova, David W. Corne, Philippe De Wilde, Vsevolod Shneer, and Fabio Caraffini. 2015. Structural bias in population-based algorithms. Information Sciences 298 (2015), 468--490.

Digital Library

[17]

Frank J Massey Jr. 1951. The Kolmogorov-Smirnov test for goodness of fit. Journal of the American statistical Association 46, 253 (1951), 68--78.

[18]

John A Nelder and Roger Mead. 1965. A simplex method for function minimization. The computer journal 7, 4 (1965), 308--313.

[19]

Michael JD Powell. 1978. A fast algorithm for nonlinearly constrained optimization calculations. In Numerical analysis. Springer, 144--157.

[20]

Kenneth Price, Rainer M Storn, and Jouni A Lampinen. 2006. Differential evolution: a practical approach to global optimization. Springer Science & Business Media.

[21]

Ronald Pyke. 1965. Spacings. Journal of the Royal Statistical Society: Series B (Methodological) 27, 3 (1965), 395--436.

[22]

R Core Team. 2021. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org

[23]

J.C. Spall. 1987. A Stochastic Approximation Technique for Generating Maximum Likelihood Parameter Estimates. In American Control Conference, 1987. 1161--1167.

[24]

Michael A Stephens. 1974. EDF statistics for goodness of fit and some comparisons. Journal of the American statistical Association 69, 347 (1974), 730--737.

[25]

Bas van Stein, Fabio Caraffini, and Anna V. Kononova. 2021. Emergence of Structural Bias in Differential Evolution. In Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion (Lille, France) (GECCO '21 Companion). Association for Computing Machinery, New York, NY, USA.

Digital Library

[26]

Diederick Vermetten, Anna V. Kononova, Fabio Carafini, Hao Wang, and Thomas Bäck. 2021. Algorithm implementations used in: Is there anisotropy in structural bias? https://github.com/facaraff/SOS/releases/tag/V1.0.2

[27]

Diederick Vermetten, Anna V. Kononova, Fabio Carafini, Hao Wang, and Thomas Bäck. 2021. Data and Code from: Is there Anisotropy in Structural Bias?

Cited By

Kononova AVermetten DCaraffini FMitran MZaharie D(2024)The Importance of Being Constrained: Dealing with Infeasible Solutions in Differential Evolution and BeyondEvolutionary Computation10.1162/evco_a_0033332:1(3-48)Online publication date: 1-Mar-2024
https://doi.org/10.1162/evco_a_00333
Vermetten DCaraffini Fvan Stein BKononova AWagner M(2022)Using structural bias to analyse the behaviour of modular CMA-ESProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3520304.3534035(1674-1682)Online publication date: 9-Jul-2022
https://dl.acm.org/doi/10.1145/3520304.3534035
Vermetten Dvan Stein BCaraffini FMinku LKononova A(2022)BIAS: A Toolbox for Benchmarking Structural Bias in the Continuous DomainIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.318984826:6(1380-1393)Online publication date: Dec-2022
https://doi.org/10.1109/TEVC.2022.3189848
Show More Cited By

Index Terms

Is there anisotropy in structural bias?
1. General and reference
  1. Cross-computing tools and techniques
    1. Empirical studies
2. Theory of computation
  1. Design and analysis of algorithms
    1. Mathematical optimization
      1. Mixed discrete-continuous optimization
        Bio-inspired optimization
  2. Theory and algorithms for application domains
    1. Theory of randomized search heuristics

Recommendations

Using structural bias to analyse the behaviour of modular CMA-ES
GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference Companion

The Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is a commonly used iterative optimisation heuristic for optimising black-box functions. CMA-ES comes in many flavours with different configuration settings. In this work, we investigate whether ...
Emergence of structural bias in differential evolution
GECCO '21: Proceedings of the Genetic and Evolutionary Computation Conference Companion

Heuristic optimisation algorithms are in high demand due to the overwhelming amount of complex optimisation problems that need to be solved. The complexity of these problems is well beyond the boundaries of applicability of exact optimisation algorithms ...
Uncovering structural bias in population-based optimization algorithms: A theoretical and simulation-based analysis of the Generalized Signature Test
Abstract
Population-based optimization algorithms are popular tools for optimization, influenced by two interactive factors: landscape bias, which guides the population toward better objective function values, and structural bias, which directs the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '21: Proceedings of the Genetic and Evolutionary Computation Conference Companion

July 2021

2047 pages

ISBN:9781450383516

DOI:10.1145/3449726

Editor:
Francisco Chicano
University of Malaga
,
General Chair:
Krzysztof Krawiec
Poznan University of Technology

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO '21

Sponsor:

SIGEVO

GECCO '21: Genetic and Evolutionary Computation Conference

July 10 - 14, 2021

Lille, France

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
184
Total Downloads

Downloads (Last 12 months)74
Downloads (Last 6 weeks)17

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kononova AVermetten DCaraffini FMitran MZaharie D(2024)The Importance of Being Constrained: Dealing with Infeasible Solutions in Differential Evolution and BeyondEvolutionary Computation10.1162/evco_a_0033332:1(3-48)Online publication date: 1-Mar-2024
https://doi.org/10.1162/evco_a_00333
Vermetten DCaraffini Fvan Stein BKononova AWagner M(2022)Using structural bias to analyse the behaviour of modular CMA-ESProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3520304.3534035(1674-1682)Online publication date: 9-Jul-2022
https://dl.acm.org/doi/10.1145/3520304.3534035
Vermetten Dvan Stein BCaraffini FMinku LKononova A(2022)BIAS: A Toolbox for Benchmarking Structural Bias in the Continuous DomainIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.318984826:6(1380-1393)Online publication date: Dec-2022
https://doi.org/10.1109/TEVC.2022.3189848
Vermetten Dvan Stein BKononova ACaraffini F(2022)Analysis of Structural Bias in Differential Evolution ConfigurationsDifferential Evolution: From Theory to Practice10.1007/978-981-16-8082-3_1(1-22)Online publication date: 25-Jan-2022
https://doi.org/10.1007/978-981-16-8082-3_1

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten