research-article

An Efficient Adaptive Differential Grouping Algorithm for Large-Scale Black-Box Optimization

Authors:

Yongsheng Liang,

Zuren FengAuthors Info & Claims

IEEE Transactions on Evolutionary Computation, Volume 27, Issue 3

Pages 475 - 489

https://doi.org/10.1109/TEVC.2022.3170793

Published: 01 June 2023 Publication History

Abstract

Decomposition plays a significant role in cooperative coevolution (CC), which shows great potential in large-scale black-box optimization (LSBO). However, current learning-based decomposition algorithms require many fitness evaluations (FEs) to detect variable interdependencies and encounter the difficulty of threshold setting. To address these issues, this study proposes an efficient adaptive differential grouping (EADG) algorithm. Instead of homogeneously tackling different types of LSBO instances, EADG first identifies the instance type by detecting the interdependencies of a few pairs of variable subsets. Only if the instance is partially separable dose EADG further engages with it by converting its decomposition process into a search process in a binary tree. This facilitates the systematic reutilization of evaluated solutions so that half the interdependencies can be directly deduced without extra FEs. To promote the decomposition accuracy, EADG specially designs a normalized interdependency indicator that can adaptively generate a decomposition threshold according to its ordinal distribution. Theoretical analysis and experimental results show that EADG outperforms current popular decomposition algorithms. Further tests indicate that it can help CC achieve highly competitive optimization performance.

References

[1]

S. Mahdavi, M. E. Shiri, and S. Rahnamayan, “Metaheuristics in large-scale global continues optimization: A survey,” Inf. Sci., vol. 295, pp. 407–428, Feb. 2015.

Digital Library

[2]

M. Bhattacharya, R. Islam, and J. Abawajy, “Evolutionary optimization: A big data perspective,” J. Netw. Comput. Appl., vol. 59, pp. 416–426, Jan. 2016.

Digital Library

[3]

J. Jian, Z. Zhan, and J. Zhang, “Large-scale evolutionary optimization: A survey and experimental comparative study,” Int. J. Mach. Learn. Cyber., vol. 11, pp. 729–745, Mar. 2020.

[4]

T. Chai, Y. Jin, and B. Sendhoff, “Evolutionary complex engineering optimization: Opportunities and challenges,” IEEE Comput. Intell. Mag., vol. 8, no. 3, pp. 12–15, Aug. 2013.

Digital Library

[5]

F. Caraffini, F. Neri, and G. Iacca, “Large scale problems in practice: The effect of dimensionality on the interaction among variables,” in Proc. 20th Eur. Conf. Appl. Evol. Comput., 2017, pp. 636–652.

[6]

D. Molina, M. Lozano, and F. Herrera, “MA-SW-Chains: Memetic algorithm based on local search chains for large scale continuous global optimization,” in Proc. IEEE Congr. Evol. Comput., 2010, pp. 1–8.

[7]

A. LaTorre, S. Muelas, and J. M. Peña, “Large scale global optimization: Experimental results with MOS-based hybrid algorithms,” in Proc. IEEE Congr. Evol. Comput., 2013, pp. 2742–2749.

[8]

D. Molina, A. LaTorre, and F. Herrera, “SHADE with iterative local search for large-scale global optimization,” in Proc. IEEE Congr. Evol. Comput., 2018, pp. 1252–1259.

[9]

A. Kabán, J. Bootkrajang, and R. J. Durrant, “Toward large-scale continuous EDA: A random matrix theory perspective,” Evol. Comput., vol. 24, no. 2, pp. 255–291, Jun. 2016.

Digital Library

[10]

D. Guo, Z. Ren, Y. Liang, and A. Chen, “Scaling up radial basis function for high-dimensional expensive optimization using random projection,” in Proc. IEEE Congr. Evol. Comput., 2020, pp. 1–8.

[11]

M. A. Potter and K. A. De Jong, “A cooperative coevolutionary approach to function optimization,” in Proc. Int. Conf. Parallel Problem Solving Nat., 1994, pp. 249–257.

[12]

Z. Yang, K. Tang, and X. Yao, “Large scale evolutionary optimization using cooperative coevolution,” Inf. Sci., vol. 178, no. 15, pp. 2986–2999, Aug. 2008.

[13]

X. Maet al., “A survey on cooperative co-evolutionary algorithms,” IEEE Trans. Evol. Comput., vol. 23, no. 3, pp. 421–441, Jun. 2019.

[14]

Z. Ren, Y. Liang, A. Zhang, Y. Yang, Z. Feng, and L. Wang, “Boosting cooperative coevolution for large scale optimization with a fine-grained computation resource allocation strategy,” IEEE Trans. Cybern., vol. 49, no. 12, pp. 4180–4193, Dec. 2019.

[15]

L. Panait, “Theoretical convergence guarantees for cooperative coevolutionary algorithms,” Evol. Comput., vol. 18, no. 4, pp. 581–615, Dec. 2010.

Digital Library

[16]

Z. Renet al., “Surrogate model assisted cooperative coevolution for large scale optimization,” Appl. Intell., vol. 49, pp. 513–531, Feb. 2019.

Digital Library

[17]

F. van den Bergh and A. P. Engelbrecht, “A cooperative approach to particle swarm optimization,” IEEE Trans. Evol. Comput., vol. 8, no. 3, pp. 225–239, Jun. 2004.

Digital Library

[18]

X. Li and X. Yao, “Cooperatively coevolving particle swarms for large scale optimization,” IEEE Trans. Evol. Comput., vol. 16, no. 2, pp. 210–224, Apr. 2012.

Digital Library

[19]

W. Chen, T. Weise, Z. Yang, and K. Tang, “Large-scale global optimization using cooperative coevolution with variable interaction learning,” in Proc. Conf. Parallel Problem Solving Nat., 2010, pp. 300–309.

[20]

M. N. Omidvar, X. Li, Y. Mei, and X. Yao, “Cooperative co-evolution with differential grouping for large scale optimization,” IEEE Trans. Evol. Comput., vol. 18, no. 3, pp. 378–393, Jun. 2014.

[21]

Y. Sun, M. Kirley, and S. K. Halgamuge, “Quantifying variable interactions in continuous optimization problems,” IEEE Trans. Evol. Comput., vol. 21, no. 2, pp. 249–264, Apr. 2017.

Digital Library

[22]

Y. Mei, M. N. Omidvar, X. Li, and X. Yao, “A competitive divide-and-conquer algorithm for unconstrained large-scale black-box optimization,” ACM Trans. Math. Soft., vol. 42, no. 2, pp. 1–24, Jun. 2016.

Digital Library

[23]

M. N. Omidvar, M. Yang, Y. Mei, X. Li, and X. Yao, “DG2: A faster and more accurate differential grouping for large-scale black-box optimization,” IEEE Trans. Evol. Comput., vol. 21, no. 6, pp. 929–942, Dec. 2017.

Digital Library

[24]

Y. Sun, M. Kirley, and S. K. Halgamuge, “A recursive decomposition method for large scale continuous optimization,” IEEE Trans. Evol. Comput., vol. 22, no. 5, pp. 647–661, Oct. 2018.

Digital Library

[25]

M. Yang, A. Zhou, C. Li, and X. Yao, “An efficient recursive differential grouping for large-scale continuous problems,” IEEE Trans. Evol. Comput., vol. 25, no. 1, pp. 159–171, Feb. 2021.

[26]

Y. Sun, M. N. Omidvar, M. Kirley, and X. Li, “Adaptive threshold parameter estimation with recursive differential grouping for problem decomposition,” in Proc. Genet. Evol. Comput. Conf. Compan., 2018, pp. 889–896.

[27]

A. Chen, Y. Zhang, Z. Ren, Y. Liang, and B. Pang, “A global information based adaptive threshold for grouping large scale global optimization problems,” in Proc. Genet. Evol. Comput. Conf. Compan., 2018, pp. 838–840.

[28]

T. Weise, R. Chiong, and K. Tang, “Evolutionary optimization: Pitfalls and booby traps,” J. Comput. Sci. Technol., vol. 27, pp. 907–936, Nov. 2012.

[29]

C. C. Vu, H. H. Nguyen, and L. T. Bui, “A parallel cooperative coevolution evolutionary algorithm,” in Proc. Int. Conf. Knowl. Syst. Eng., 2011, pp. 48–53.

[30]

Y. Jiaet al., “Distributed cooperative co-evolution with adaptive computing resource allocation for large scale optimization,” IEEE Trans. Evol. Comput., vol. 23, no. 2, pp. 188–202, Apr. 2019.

[31]

N. R. Sabar, J. Abawajy, and J. Yearwood, “Heterogeneous cooperative co-evolution memetic differential evolution algorithm for big data optimization problems,” IEEE Trans. Evol. Comput., vol. 21, no. 2, pp. 315–327, Apr. 2017.

Digital Library

[32]

Y. Sun, M. Kirley, and X. Li, “Cooperative co-evolution with online optimizer selection for large-scale optimization,” in Proc. Genet. Evol. Comput. Conf. Compan., 2018, pp. 1079–1086.

[33]

T. Ray and X. Yao, “A cooperative coevolutionary algorithm with correlation based adaptive variable partitioning,” in Proc. IEEE Congr. Evol. Comput., 2009, pp. 983–989.

[34]

Q. Xu, M. L. Sanyang, and A. Kaban, “Large scale continuous EDA using mutual information,” in Proc. IEEE Congr. Evol. Comput., 2016, pp. 3718–3725.

[35]

M. N. Omidvar, X. Li, and X. Yao, “Cooperative co-evolution with delta grouping for large scale non-separable function optimization,” in Proc. IEEE Congr. Evol. Comput., 2010, pp. 1762–1769.

[36]

Q. Yang, W. Chen, and J. Zhang, “Evolution consistency based decomposition for cooperative coevolution,” IEEE Access, vol. 6, pp. 51084–51097, 2018.

[37]

L. Sun, S. Yoshida, X. Cheng, and Y. Liang, “A cooperative particle swarm optimizer with statistical variable interdependence learning,” Inf. Sci., vol. 186, no. 1, pp. 20–39, Mar. 2012.

Digital Library

[38]

H. Ge, L. Sun, X. Yang, S. Yoshida, and Y. Liang, “Cooperative differential evolution with fast variable interdependence learning and cross-cluster mutation,” Appl. Soft Comput., vol. 36, pp. 300–314, Nov. 2015.

Digital Library

[39]

H. Ge, L. Sun, G. Tan, Z. Chen, and C. L. P. Chen, “Cooperative hierarchical PSO with two stage variable interaction reconstruction for large scale optimization,” IEEE Trans. Cybern., vol. 47, no. 9, pp. 2809–2823, Sep. 2017.

[40]

L. Li, L. Jiao, R. Stolkin, and F. Liu, “Mixed second order partial derivatives decomposition method for large scale optimization,” Appl. Soft Comput., vol. 61, pp. 1013–1021, Dec. 2017.

[41]

X. Hu, F. He, W. Chen, and J. Zhang, “Cooperation coevolution with fast interdependency identification for large scale optimization,” Inf. Sci., vol. 381, pp. 142–160, Mar. 2017.

Digital Library

[42]

Z. Ren, A. Chen, L. Wang, Y. Liang, and B. Pang, “An efficient vector-growth decomposition algorithm for cooperative coevolution in solving large scale problems,” in Proc. Genet. Evol. Comput. Conf. Compan., 2017, pp. 41–42.

[43]

K. S. Kim and Y. S. Choi, “An efficient variable interdependency-identification and decomposition by minimizing redundant computations for large-scale global optimization,” Inf. Sci., vol. 513, pp. 289–323, Mar. 2020.

Digital Library

[44]

X. Xueet al., “A topology-based single-pool decomposition framework for largescale global optimization,” Appl. Soft Comput., vol. 92, Jul. 2020, Art. no.

[45]

X. Li, K. Tang, M. N. Omidvar, Z. Yang, and K. Qin, “Benchmark functions for the CEC’2013 special session and competition on large scale global optimization,” Evol. Comput. Mach. Learn. Group, RMIT Univ., Melbourne, VIC, Australia, Rep., 2013.

[46]

K. Tang, X. Li, P. N. Suganthan, Z. Yang, and T. Weise, “Benchmark functions for the CEC’2010 special session and competition on large scale global optimization,” Nat. Inspired Comput. Appl. Lab., Univ. Sci. Technol. China, Hefei, China, Rep., 2010.

[47]

L. Danon, A. D. Guilera, J. Duch, and A. Arenas, “Comparing community structure identification,” J. Stat. Mech. Theory Exp., vol. 2005, Sep. 2005, Art. no.

[48]

Z. Yang, K. Tang, and X. Yao, “Self-adaptive differential evolution with neighborhood search,” in Proc. IEEE Congr. Evol. Comput., 2008, pp. 1110–1116.

[49]

M. Yang, A. Zhou, C. Li, J. Guan, and X. Yan, “CCFR2: A more efficient cooperative co-evolutionary framework for large-scale global optimization,” Inf. Sci., vol. 512, pp. 64–79, Feb. 2020.

Digital Library

[50]

N. Hansen, “The CMA evolution strategy: A tutorial,” 2016, arXiv:1604.00772.

[51]

J. Cohen, Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, NJ, USA: Lawrence Erlbaum, 1988.

Cited By

Liu XZhan ZZhang J(2024)Transfer-Based Particle Swarm Optimization for Large-Scale Dynamic Optimization With Changing Variable InteractionsIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.332632728:6(1633-1643)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TEVC.2023.3326327
Yang WLiu JTan SZhang WLiu Y(2024)Evolutionary dynamic grouping based cooperative co-evolution algorithm for large-scale optimizationApplied Intelligence10.1007/s10489-024-05390-554:6(4585-4601)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s10489-024-05390-5
Chen MDu WTang YJin YYen G(2023)A Decomposition Method for Both Additively and Nonadditively Separable ProblemsIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.321837527:6(1720-1734)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TEVC.2022.3218375
Show More Cited By

Recommendations

Large-scale optimization using immune algorithm
GEC '09: Proceedings of the first ACM/SIGEVO Summit on Genetic and Evolutionary Computation

Immune-inspired optimization algorithms encoded the parameters into individuals where each individual represents a search point in the space of potential solutions. A large number of parameters would result in a large search space. Nowadays, there is ...
Decentralizing and coevolving differential evolution for large-scale global optimization problems

This paper presents a novel decentralizing and coevolving differential evolution (DCDE) algorithm to address the issue of scaling up differential evolution (DE) algorithms to solve large-scale global optimization (LSGO) problems. As most evolutionary ...
A Competitive Divide-and-Conquer Algorithm for Unconstrained Large-Scale Black-Box Optimization

This article proposes a competitive divide-and-conquer algorithm for solving large-scale black-box optimization problems for which there are thousands of decision variables and the algebraic models of the problems are unavailable. We focus on problems ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Evolutionary Computation

IEEE Transactions on Evolutionary Computation Volume 27, Issue 3

June 2023

345 pages

ISSN:1089-778X

Issue’s Table of Contents

1089-778X © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Press

Publication History

Published: 01 June 2023

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu XZhan ZZhang J(2024)Transfer-Based Particle Swarm Optimization for Large-Scale Dynamic Optimization With Changing Variable InteractionsIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.332632728:6(1633-1643)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TEVC.2023.3326327
Yang WLiu JTan SZhang WLiu Y(2024)Evolutionary dynamic grouping based cooperative co-evolution algorithm for large-scale optimizationApplied Intelligence10.1007/s10489-024-05390-554:6(4585-4601)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s10489-024-05390-5
Chen MDu WTang YJin YYen G(2023)A Decomposition Method for Both Additively and Nonadditively Separable ProblemsIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.321837527:6(1720-1734)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TEVC.2022.3218375
Chen ARen ZWang MChen HLeng HLiu S(2023)A surrogate-assisted highly cooperative coevolutionary algorithm for hyperparameter optimization in deep convolutional neural networksApplied Soft Computing10.1016/j.asoc.2023.110794147:COnline publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1016/j.asoc.2023.110794

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents