Abstract
The optimization of massively multi-modal functions is a challenging task, particularly for problems where the search space can lead the optimization process to local optima. While evolutionary algorithms have been extensively investigated for these optimization problems, Bayesian Optimization algorithms have not been explored to the same extent. In this paper, we study the behavior of Bayesian Optimization as part of a hybrid approach for solving several massively multi-modal functions. We use well-known benchmarks and metrics to evaluate how different variants of Bayesian Optimization deal with multi-modality.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Brochu, E., Cora, V.M., de Freitas, N.: A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning, December 2010. arXiv:1012.2599
Fieldsend, J.E.: Running up those hills: multi-modal search with the niching migratory multi-swarm optimiser. In: 2014 IEEE Congress on Evolutionary Computation (CEC), pp. 2593–2600. IEEE (2014)
Ginsbourger, D., Riche, R.L., Carraro, L.: A multi-points criterion for deterministic parallel global optimization based on Gaussian processes. Technical report, CCSD, March 2008. https://hal.archives-ouvertes.fr/hal-00260579/document
Goldberg, D.E., Richardson, J.: Genetic algorithms with sharing for multi-modal function optimisation. In: Proceedings of the of Second International Conference on Genetic Algorithms and Their Applications, pp. 41–49 (1987)
González, J., Dai, Z., Hennig, P., Lawrence, N.D.: Batch Bayesian optimization via local penalization (2015). arXiv:1505.08052
Guhaniyogi, R., Li, C., Savitsky, T.D., Srivastava, S.: A divide-and-conquer Bayesian approach to large-scale kriging, December 2017. arXiv:1712.09767
Jones, D.R., Perttunen, C.D., Stuckman, B.E.: Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl. 79(1), 157–181 (1993). https://doi.org/10.1007/BF00941892
Jones, D.R.: A taxonomy of global optimization methods based on response surfaces. J. Global Optim. 21(4), 345–383 (2001). https://doi.org/10.1023/A:1012771025575
Kvasov, D.E., Sergeyev, Y.D.: Deterministic approaches for solving practical black-box global optimization problems. Adv. Eng. Softw. 80, 58–66 (2015). https://doi.org/10.1016/j.advengsoft.2014.09.014. http://www.sciencedirect.com/science/article/pii/S096599781400163X
Li, X., Engelbrecht, A., Epitropakis, M.G.: Benchmark functions for CEC 2013 special session and competition on niching methods for multimodal function optimization. Technical report, Royal Melbourne Institute of Technology, March 2013. http://goanna.cs.rmit.edu.au/xiaodong/cec13-niching/
Mockus, J.: The Bayesian approach to local optimization. In: Mockus, J. (ed.) Bayesian Approach to Global Optimization. Mathematics and Its Applications, vol. 37, pp. 125–156. Springer, Dordrecht (1989). https://doi.org/10.1007/978-94-009-0909-0_7
Petrowski, A.: A clearing procedure as a niching method for genetic algorithms. In: Proceedings of IEEE International Conference on Evolutionary Computation, pp. 798–803 (1996). https://doi.org/10.1109/ICEC.1996.542703
Powell, M.J.D.: An efficient method for finding the minimum of a function of several variables without calculating derivatives. Comput. J. 7(2), 155–162 (1964). https://doi.org/10.1093/comjnl/7.2.155. http://comjnl.oxfordjournals.org/content/7/2/155
Preuss, M.: Niching the CMA-ES via nearest-better clustering. In: Proceedings of the 12th Annual Conference Companion on Genetic and Evolutionary Computation, GECCO 2010, pp. 1711–1718. ACM, New York (2010). https://doi.org/10.1145/1830761.1830793
Preuss, M.: Multimodal Optimization by Means of Evolutionary Algorithms. Natural Computing Series. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-07407-8
Quiñonero-Candela, J., Rasmussen, C.E.: A unifying view of sparse approximate Gaussian process regression. J. Mach. Learn. Res. 6(Dec), 1939–1959 (2005). http://www.jmlr.org/papers/v6/quinonero-candela05a
Rasmussen, C.E., Williams, C.K.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)
Roman, I., Santana, R., Mendiburu, A., Lozano, J.A.: Dynamic kernel selection criteria for Bayesian optimization. In: BayesOpt 2014: NIPS Workshop on Bayesian Optimization, Montreal (2014). http://bayesopt.github.io/papers/paper13.pdf
Sareni, B., Krahenbuhl, L.: Fitness sharing and niching methods revisited. IEEE Trans. Evol. Comput. 2(3), 97–106 (1998)
Silverman, B.W.: Some aspects of the spline smoothing approach to non-parametric regression curve fitting. J. Roy. Stat. Soc. Series B (Method.) 47(1), 1–52 (1985)
Singh, G., Deb, K.: Comparison of multi-modal optimization algorithms based on evolutionary algorithms. In: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, GECCO 2006, pp. 1305–1312. ACM, New York (2006). https://doi.org/10.1145/1143997.1144200
Srinivas, N., Krause, A., Kakade, S., Seeger, M.: Gaussian process optimization in the bandit setting: no regret and experimental design. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010), Haifa, 21–24 June 2010, pp. 1015–1022 (2010). http://www.icml2010.org/papers/422.pdf
Wang, H., van Stein, B., Emmerich, M., Bäck, T.: Time complexity reduction in efficient global optimization using cluster kriging. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2017, pp. 889–896. ACM, New York (2017). https://doi.org/10.1145/3071178.3071321
Wong, K.C., Leung, K.S., Wong, M.H.: Protein structure prediction on a lattice model via multimodal optimization techniques. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, GECCO 2010, pp. 155–162. ACM, New York (2010). https://doi.org/10.1145/1830483.1830513
Yin, X., Germay, N.: Investigations on solving the load flow problem by genetic algorithms. Electr. Power Syst. Res. 22(3), 151–163 (1991). https://doi.org/10.1016/0378-7796(91)90001-4. http://www.sciencedirect.com/science/article/pii/0378779691900014
Zhigljavsky, A., Žilinskas, A.: Methods based on statistical models of multimodal functions. In: Zhigljavsky, A., Žilinskas, A. (eds.) Stochastic Global Optimization. Springer Optimization and Its Applications, vol. 9, pp. 149–244. Springer, Boston (2008). https://doi.org/10.1007/978-0-387-74740-8_4
Acknowledgments
This research has been partially supported by the Basque Government (ELKARTEK programs), and Spanish Ministry of Economy and Competitiveness MINECO (project TIN2016-78365-R). Jose A. Lozano is also supported by BERC 2018-2021 (Basque Government), and Severo Ochoa Program SEV-2017-0718 (Spanish Ministry of Economy, Industry and Competitiveness).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Roman, I., Mendiburu, A., Santana, R., Lozano, J.A. (2020). Bayesian Optimization Approaches for Massively Multi-modal Problems. In: Matsatsinis, N., Marinakis, Y., Pardalos, P. (eds) Learning and Intelligent Optimization. LION 2019. Lecture Notes in Computer Science(), vol 11968. Springer, Cham. https://doi.org/10.1007/978-3-030-38629-0_31
Download citation
DOI: https://doi.org/10.1007/978-3-030-38629-0_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-38628-3
Online ISBN: 978-3-030-38629-0
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)