research-article

An efficient variable interdependency-identification and decomposition by minimizing redundant computations for large-scale global optimization

Authors:

Yong Suk ChoiAuthors Info & Claims

Volume 513, Issue C

Pages 289 - 323

https://doi.org/10.1016/j.ins.2019.10.049

Published: 01 March 2020 Publication History

Abstract

Although many variable decomposition methods for cooperative co-evolution (CC) have been proposed, researches on scalable and efficient decomposition have rarely been done, particularly, for the large-scale global optimization (LSGO) problems. In this paper, we propose an efficient variable interdependency identification and decomposition method, called EVIID. Different from existing studies focusing on only limited scale efficient or accurate variable decomposition, our purpose is to develop a scalable variable decomposition method with high efficiency and accuracy even on very high-dimensional problems. EVIID utilizes three core strategies: a binary variable space search, a dynamic perturbation caching, and a pre-variable sorting. Their synergy effect enables scalable and efficient variable decomposition without sacrificing decomposition accuracy by pruning many redundant computations required to identify interdependencies among decision variables. In comprehensive experiments, EVIID showed highly scalable decomposition ability on 1000 to 10,000 dimensional benchmark problems compared against the state-of-the-art variable decomposition methods. Moreover, when EVIID was embedded into practical CC frameworks, it showed good optimization performance and also fast convergence.

References

[1]

M. Amine Bouhlel, N. Bartoli, R.G. Regis, A. Otsmane, J. Morlier, Efficient global optimization for high-dimensional constrained problems by using the Kriging models combined with the partial least squares method, Eng. Optim. (2018) 1–16.

[2]

T. Bhowmik, H. Liu, Z. Ye, S. Oraintara, Dimensionality Reduction Based Optimization Algorithm for Sparse 3-D Image Reconstruction in Diffuse Optical Tomography, Sci. Rep. 6 (2016) 22242.

[3]

L. Bottou, F.E. Curtis, J. Nocedal, Optimization methods for large-scale machine learning, SIAM Rev. 60 (2018) 223–311.

[4]

H. Cao, X. Qian, Y. Zhou, Large-scale structural optimization using metaheuristic algorithms with elitism and a filter strategy, Struct. Multidisc. Optim. 57 (2018) 799–814.

[5]

W. Chen, T. Weise, Z. Yang, K. Tang, Large-scale global optimization using cooperative coevolution with variable interaction learning, in: International Conference on Parallel Problem Solving from Nature, Springer, 2010, pp. 300–309.

[6]

R. Cheng, Y. Jin, A social learning particle swarm optimization algorithm for scalable optimization, Inf. Sci. 291 (2015) 43–60.

Digital Library

[7]

Y.S. Choi, Tree pattern expression for extracting information from syntactically parsed text corpora, Data Min. Knowl. Discov. 22 (2010) 211–231.

[8]

Y.S. Choi, TPEMatcher: a tool for searching in parsed text corpora, Knowl. Based Syst. 24 (2011) 1139–1150.

[9]

H. Deng, L. Peng, H. Zhang, B. Yang, Z. Chen, Ranking-based biased learning swarm optimizer for large-scale optimization, Inf. Sci. 493 (2019) 120–137.

[10]

W. Dong, Y. Wang, M. Zhou, A latent space-based estimation of distribution algorithm for large-scale global optimization, Soft Comput. 23 (2019) 4593–4615.

[11]

L.C. Echevarría, O.L. Santiago, H.F. de Campos Velho, A.J. da Silva Neto, Metaheuristics for Optimization Problems, Fault Diagnosis Inverse Problems: Solution with Metaheuristics, Springer, 2019, pp. 43–83.

[12]

X.-M. Hu, F.-L. He, W.-N. Chen, J. Zhang, Cooperation coevolution with fast interdependency identification for large scale optimization, Inf. Sci. 381 (2017) 142–160.

Digital Library

[13]

K. Hussain, M.N.M. Salleh, S. Cheng, Y. Shi, Metaheuristic research: a comprehensive survey, Artificial Intelligence Review, 2018, pp. 1–43.

[14]

A. LaTorre, S. Muelas, J.-M. Peña, A comprehensive comparison of large scale global optimizers, Inf. Sci 316 (2015) 517–549.

[15]

L. Li, L. Jiao, R. Stolkin, F. Liu, Mixed second order partial derivatives decomposition method for large scale optimization, Appl. Soft Comput. 61 (2017) 1013–1021.

[16]

Z. Li, Q. Zhang, A Simple Yet Efficient Evolution Strategy for Large-Scale Black-Box Optimization, IEEE Trans. Evol. Comput. 22 (2018) 637–646.

[17]

H. Liu, Y. Wang, X. Liu, S. Guan, Empirical study of effect of grouping strategies for large scale optimization, in: Proceedings of the International Joint Conference on Neural Networks (IJCNN), 2016, pp. 3433–3439.

[18]

J.K. Liu, Y.M. Feng, L.M. Zou, A spectral conjugate gradient method for solving large-scale unconstrained optimization, Comput. Math. Appl. 77 (2019) 731–739.

[19]

W. Long, T. Wu, X. Liang, S. Xu, Solving high-dimensional global optimization problems using an improved sine cosine algorithm, Expert Syst. Appl. 123 (2019) 108–126.

Digital Library

[20]

X. Ma, X. Li, Q. Zhang, K. Tang, Z. Liang, W. Xie, Z. Zhu, A survey on cooperative co-evolutionary algorithms, IEEE Trans. Evol. Comput. 23 (2019) 421–441.

[21]

S. Mahdavi, S. Rahnamayan, M.E. Shiri, Multilevel framework for large-scale global optimization, Soft Comput. 21 (2017) 4111–4140.

[22]

S. Mahdavi, M.E. Shiri, S. Rahnamayan, Metaheuristics in large-scale global continues optimization: A survey, Inf. Sci. 295 (2015) 407–428.

[23]

Y. Mei, M.N. Omidvar, X. Li, X. Yao, A Competitive Divide-and-conquer algorithm for unconstrained large-scale black-box optimization, ACM Trans. Math. Softw. 42 (2016) 1–24.

[24]

A. Nakib, L. Souquet, E.G. Talbi, Parallel fractal decomposition based algorithm for big continuous optimization problems, J. Parallel Distrib. Comput. (2018).

[25]

M.N. Omidvar, X. Li, Y. Mei, X. Yao, Cooperative co-evolution with differential grouping for large scale optimization, IEEE Trans. Evol. Comput. 18 (2014) 378–393.

[26]

M.N. Omidvar, X. Li, X. Yao, Cooperative Co-evolution with delta grouping for large scale non-separable function optimization, in: Proceedings of the IEEE Congress on Evolutionary Computation, 2010, pp. 1–8.

[27]

M.N. Omidvar, M. Yang, Y. Mei, X. Li, X. Yao, DG2: A Faster and More Accurate Differential Grouping for Large-Scale Black-Box Optimization, IEEE Trans. Evol. Comput. 21 (2017) 929–942.

[28]

P. Palittapongarnpim, P. Wittek, E. Zahedinejad, S. Vedaie, B.C. Sanders, Learning in quantum control: High-dimensional global optimization for noisy quantum dynamics, Neurocomputing 268 (2017) 116–126.

[29]

A.P. Piotrowski, J.J. Napiorkowski, Some metaheuristics should be simplified, Inf. Sci. 427 (2018) 32–62.

Digital Library

[30]

M.A. Potter, K.A. De Jong, A cooperative coevolutionary approach to function optimization, in: Proceedings of the International Conference on Parallel Problem Solving from Nature, Springer, 1994, pp. 249–257.

[31]

E. Sayed, D. Essam, R. Sarker, S. Elsayed, Decomposition-based evolutionary algorithm for large scale constrained problems, Inf. Sci. 316 (2015) 457–486.

[32]

Y.D. Sergeyev, D.E. Kvasov, M.S. Mukhametzhanov, On the efficiency of nature-inspired metaheuristics in expensive global optimization with limited budget, Sci. Rep. 8 (2018) 453.

[33]

D.J. Sheskin, Handbook of Parametric and Nonparametric Statistical Procedures, third ed., Chapman and Hall/CRC Press, 2004.

[34]

Y. Sun, M. Kirley, S.K. Halgamuge, Extended differential grouping for large scale global optimization with direct and indirect variable interactions, in: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, Madrid, Spain, ACM, 2015, pp. 313–320.

[35]

Y. Sun, M. Kirley, S.K. Halgamuge, A Recursive Decomposition Method for Large Scale Continuous Optimization, IEEE Trans. Evol. Comput. 22 (2018) 647–661.

[36]

Y. Sun, M. Kirley, X. Li, Cooperative co-evolution with online optimizer selection for large-scale optimization, in: Proceedings of the Genetic and Evolutionary Computation Conference, Kyoto, Japan, ACM, 2018, pp. 1079–1086.

[37]

Y. Sun, X. Wang, Y. Chen, Z. Liu, A modified whale optimization algorithm for large-scale global optimization problems, Expert Syst. Appl. 114 (2018) 563–577.

[38]

K. Tang, X. Li, P.N. Suganthan, Z. Yang, T. Weise, Benchmark Functions for the CEC’2010 Special Session and Competition on Large-Scale Global Optimization, Nature Inspired Computation and Applications Laboratory (NICAL), USTC, China (2010) Technical report.

[39]

G.A. Trunfio, P. Topa, J. Wąs, A new algorithm for adapting the configuration of subcomponents in large-scale optimization with cooperative coevolution, Inf. Sci. 372 (2016) 773–795.

[40]

Q. Wei, D. Liu, Q. Lin, R. Song, Adaptive dynamic programming for discrete-time zero-sum games, IEEE Trans. Neural Netw. Learn. Syst. 29 (2018) 957–969.

[41]

Y. Xue, J. Jiang, B. Zhao, T. Ma, A self-adaptive artificial bee colony algorithm based on global best for global optimization, Soft Comput. 22 (2017) 2935–2952.

[42]

Z. Yan, J. Fan, J. Wang, A Collective Neurodynamic Approach to Constrained Global Optimization, IEEE Trans. Neural Netw. Learn. Syst. 28 (2017) 1206–1215.

[43]

M. Yang, M.N. Omidvar, C. Li, X. Li, Z. Cai, B. Kazimipour, X. Yao, Efficient resource allocation in cooperative co-evolution for large-scale global optimization, IEEE Trans. Evol. Comput. 21 (2017) 493–505.

[44]

P. Yang, K. Tang, X. Yao, Turning high-dimensional optimization into computationally expensive optimization, IEEE Trans. Evol. Comput. 22 (2018) 143–156.

[45]

T. Yang, A.A. Asanjan, M. Faridzad, N. Hayatbini, X. Gao, S. Sorooshian, An enhanced artificial neural network with a shuffled complex evolutionary global optimization with principal component analysis, Inf. Sci. 418-419 (2017) 302–316.

[46]

Z. Yang, S. Ping, A. Aijaz, A.-H. Aghvami, A global optimization-based routing protocol for cognitive-radio-enabled smart grid AMI networks, IEEE Syst. J. 12 (2018) 1015–1023.

[47]

Z. Yang, K. Tang, X. Yao, Large scale evolutionary optimization using cooperative coevolution, Inf. Sci. 178 (2008) 2985–2999.

Digital Library

[48]

Y.E. Yildiz, A.O. Topal, Large scale continuous global optimization based on micro differential evolution with local directional search, Inf. Sci. 477 (2019) 533–544.

[49]

Y. Zhenyu, T. Ke, Y. Xin, Self-adaptive differential evolution with neighborhood search, in: Proceedings of the IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence), 2008, pp. 1110–1116.

[50]

IEEE Standard for Floating-Point Arithmetic, IEEE Std 754-2008, (2008) 1-70.

Cited By

Fonseca TNassar Sde Oliveira AAgard B(2023)Low-Dimensional Space Modeling-Based Differential Evolution for Large-Scale Global Optimization ProblemsIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.322744027:5(1529-1543)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1109/TEVC.2022.3227440
Chen MDu WTang YJin YYen G(2023)A Decomposition Method for Both Additively and Nonadditively Separable ProblemsIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.321837527:6(1720-1734)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TEVC.2022.3218375
Chen ARen ZGuo WLiang YFeng Z(2023)An Efficient Adaptive Differential Grouping Algorithm for Large-Scale Black-Box OptimizationIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.317079327:3(475-489)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1109/TEVC.2022.3170793
Show More Cited By

Index Terms

An efficient variable interdependency-identification and decomposition by minimizing redundant computations for large-scale global optimization

Index terms have been assigned to the content through auto-classification.

Recommendations

Multilevel framework for large-scale global optimization

Large-scale global optimization (LSGO) algorithms are crucially important to handle real-world problems. Recently, cooperative co-evolution (CC) algorithms have successfully been applied for solving many large-scale practical problems. Many applications ...
Incremental cooperative coevolution for large-scale global optimization

Cooperative coevolution (CC) is an efficient framework for solving large-scale global optimization (LSGO) problems. It uses a decomposition method to divide the LSGO problems into several low-dimensional subcomponents; then, subcomponents are optimized. ...
Metaheuristics in large-scale global continues optimization

Metaheuristic algorithms are extensively recognized as effective approaches for solving high-dimensional optimization problems. These algorithms provide effective tools with important applications in business, engineering, economics, and science. This ...

Comments

Information & Contributors

Information

Published In

cover image Information Sciences: an International Journal

Information Sciences: an International Journal Volume 513, Issue C

Mar 2020

627 pages

ISSN:0020-0255

Issue’s Table of Contents

Copyright © 2019.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 March 2020

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Fonseca TNassar Sde Oliveira AAgard B(2023)Low-Dimensional Space Modeling-Based Differential Evolution for Large-Scale Global Optimization ProblemsIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.322744027:5(1529-1543)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1109/TEVC.2022.3227440
Chen MDu WTang YJin YYen G(2023)A Decomposition Method for Both Additively and Nonadditively Separable ProblemsIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.321837527:6(1720-1734)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TEVC.2022.3218375
Chen ARen ZGuo WLiang YFeng Z(2023)An Efficient Adaptive Differential Grouping Algorithm for Large-Scale Black-Box OptimizationIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.317079327:3(475-489)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1109/TEVC.2022.3170793
Chen ARen ZWang MLiang YLiu HDu W(2023)A surrogate-assisted variable grouping algorithm for general large-scale global optimization problemsInformation Sciences: an International Journal10.1016/j.ins.2022.11.117622:C(437-455)Online publication date: 9-Mar-2023
https://dl.acm.org/doi/10.1016/j.ins.2022.11.117
Omidvar MLi XYao X(2022)A Review of Population-Based Metaheuristics for Large-Scale Black-Box Global Optimization—Part IIEEE Transactions on Evolutionary Computation10.1109/TEVC.2021.313083826:5(802-822)Online publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1109/TEVC.2021.3130838
Chen ARen ZLiang YGuo D(2021)Enhancing Cooperative Coevolution for Large Scale Optimization by Exploiting Decomposition Solutions2021 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC45853.2021.9504977(1047-1053)Online publication date: 28-Jun-2021
https://dl.acm.org/doi/10.1109/CEC45853.2021.9504977
Zhang ZXiao TQin X(2021)Fly visual evolutionary neural network solving large‐scale global optimizationInternational Journal of Intelligent Systems10.1002/int.2256436:11(6680-6712)Online publication date: 26-Jul-2021
https://dl.acm.org/doi/10.1002/int.22564

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents