Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Open Issues in Surrogate-Assisted Optimization

  • Chapter
  • First Online:
High-Performance Simulation-Based Optimization

Part of the book series: Studies in Computational Intelligence ((SCI,volume 833))

Abstract

Surrogate-assisted optimization was developed for handling complex and costly problems, which arise from real-world applications. The main idea behind surrogate-assisted optimization is to optimally exhaust the available information to lower the amount of required expensive function evaluations thus saving time, resources and the related costs. This chapter outlines the existing challenges in this field that include benchmarking, constraint handling, constructing ensembles of surrogates and solving discrete and/or multi-objective optimization problems. We discuss shortcomings of existing techniques, propose suggestions for improvements and give an outlook on promising research directions. This is valuable for practitioners and researchers alike, since the increased availability of computational resources on the one hand and the continuous development of new approaches on the other hand raise many intricate new problems in this field.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    See https://www.lorentzcenter.nl/lc/web/2016/764/info.php3?wsid=764 for SAMCO’s website and http://samco.gforge.inria.fr/doku.php for the list of libraries (both accessed on 30. 11. 2017).

References

  1. Agrawal, R., Gehrke, J., Gunopulos, D., Raghavan, P.: Automatic subspace clustering of high dimensional data for data mining applications. In: Proceedings of the 1998 ACM SIGMOD International Conference on Management of Data, pp. 94–105 (1998)

    Google Scholar 

  2. Asadi, M., Goldak, J.: Combinatorial optimization of weld sequence by using a surrogate model to mitigate a weld distortion. Int. J. Mech. Mater. Des. 7(2), 123–139 (2011)

    Article  Google Scholar 

  3. Bandler, J.W., Cheng, Q.S., Dakroury, S.A., Mohamed, A.S., Bakr, M.H., Madsen, K., Sondergaard, J.: Space mapping: the state of the art. IEEE Trans. Microw. Theory Tech. 52(1), 337–361 (2004)

    Article  Google Scholar 

  4. Bartz-Beielstein, T.: How to create generalizable results. In: Springer Handbook of Computational Intelligence, pp. 1127–1142. Springer, Berlin (2015)

    Chapter  Google Scholar 

  5. Bartz-Beielstein, T.: Stacked generalization of surrogate models - a practical approach. Technical Report 5/2016, TH Köln (2016)

    Google Scholar 

  6. Bartz-Beielstein, T., Blum, D., Branke, J.: Particle swarm optimization and sequential sampling in noisy environments. In: Metaheuristics, pp. 261–273 (2007)

    Google Scholar 

  7. Bartz-Beielstein, T., Friese, M., Zaefferer, M., Naujoks, B., Flasch, O., Konen, W., Koch, P.: Noisy optimization with sequential parameter optimization and optimal computational budget allocation. In: Companion Material Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2011, pp. 119–120 (2011)

    Google Scholar 

  8. Bartz-Beielstein, T., Zaefferer, M.: Model-based methods for continuous and discrete global optimization. Appl. Soft Comput. 55, 154–167 (2017)

    Article  Google Scholar 

  9. Beyer, H.G., Sendhoff, B.: Robust optimization-a comprehensive survey. Comput. Methods Appl. Mech. Eng. 196(33), 3190–3218 (2007)

    Article  MathSciNet  Google Scholar 

  10. Bird, S., Li, X.: Improving local convergence in particle swarms by fitness approximation using regression. In: Computational Intelligence in Expensive Optimization Problems, pp. 265–293 (2010)

    Chapter  Google Scholar 

  11. Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, COLT92, pp. 144–152 (1992)

    Google Scholar 

  12. Branke, J., Kaußler, T., Smidt, C., Schmeck, H.: A multi-population approach to dynamic optimization problems. In: Evolutionary Design and Manufacture, pp. 299–307. Springer, Berlin (2000)

    Chapter  Google Scholar 

  13. Breiman, L., Friedman, J., Stone, C.J., Olshen, R.A.: Classification and Regression Trees. CRC Press (1984)

    Google Scholar 

  14. Brownlee, A.E., Wright, J.A.: Constrained, mixed-integer and multi-objective optimisation of building designs by NSGA-II with fitness approximation. Appl. Soft Comput. 33, 114–126 (2015)

    Article  Google Scholar 

  15. Chen, C.h., Lee, L.H.: Stochastic Simulation Optimization: An Optimal Computing Budget Allocation, vol. 1, World Scientific (2011)

    Google Scholar 

  16. Chen, X., Diez, M., Kandasamy, M., Zhang, Z., Campana, E.F., Stern, F.: High-fidelity global optimization of shape design by dimensionality reduction, metamodels and deterministic particle swarm. Eng. Optim. 47(4), 473–494 (2015)

    Article  Google Scholar 

  17. Coello Coello, C.A.: Constraint-handling techniques used with evolutionary algorithms. In: Companion Material Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2017, pp. 675–701 (2017)

    Google Scholar 

  18. Coello Coello, C.A., Lamont, G.B.: Applications of Multi-Objective Evolutionary Algorithms. World Scientific, New Jersey (2004)

    Chapter  Google Scholar 

  19. Coello Coello, C.A., Lamont, G.B., Van Veldhuizen, D.A.: Evolutionary Algorithms for Solving Multi-Objective Problems, 2nd edn, Springer, Berlin (2007)

    Google Scholar 

  20. Couckuyt, I., De Turck, F., Dhaene, T., Gorissen, D.: Automatic surrogate model type selection during the optimization of expensive black-box problems. In: Proceedings of the 2011 Winter Simulation Conference, WSC 2011, pp. 4269–4279 (2011)

    Google Scholar 

  21. Cressie, N.: Spatial prediction and ordinary kriging. Math. Geol. 20(4), 405–421 (1988)

    Article  MathSciNet  Google Scholar 

  22. Custódio, F.L., Barbosa, H.J., Dardenne, L.E.: Full-atom ab initio protein structure prediction with a genetic algorithm using a similarity-based surrogate model. Proceedings of the IEEE Congress on Evolutionary Computation, CEC 2010, 1–8 (2010)

    Google Scholar 

  23. Deb, K.: Multi-Objective Optimization Using Evolutionary Algorithms. Wiley (2001)

    Google Scholar 

  24. Emmerich, M.T.M., Giannakoglou, K.C., Naujoks, B.: Single- and multiobjective evolutionary optimization assisted by gaussian random field metamodels. IEEE Trans. Evol. Comput. 10(4), 421–439 (2006)

    Article  Google Scholar 

  25. Filipiak, P., Michalak, K., Lipinski, P.: Infeasibility driven evolutionary algorithm with arima-based prediction mechanism. Intell. Data Eng. Autom. Learn.-IDEAL 2011, 345–352 (2011)

    Google Scholar 

  26. Fischbach, A., Zaefferer, M., Stork, J., Friese, M., Bartz-Beielstein, T.: From real world data to test functions. In: Proceedings of the 26. Workshop Computational Intelligence, pp. 159–177 (2016)

    Google Scholar 

  27. Forrester, A., Keane, A.: Recent advances in surrogate-based optimization. Prog. Aerosp. Sci. 45, 50–79 (2009)

    Article  Google Scholar 

  28. Forrester, A., Sobester, A., Keane, A.: Engineering Design Via Surrogate Modelling: A Practical Guide. Wiley (2008)

    Google Scholar 

  29. Friese, M., Bartz-Beielstein, T., Emmerich, M.: Building ensembles of surrogates by optimal convex combination. Bioinspired Optim. Methods Their Appl. BIOMA 2016, 131–143 (2016)

    Google Scholar 

  30. Goel, T., Haftka, R.T., Shyy, W., Queipo, N.V.: Ensemble of surrogates. Struct. Multidiscip. Optim. 33(3), 199–216 (2006)

    Article  Google Scholar 

  31. Gómez-Bombarelli, R., Duvenaud, D.K., Hernández-Lobato, J.M., Aguilera-Iparraguirre, J., Hirzel, T.D., Adams, R.P., Aspuru-Guzik, A.: Automatic chemical design using a data-driven continuous representation of molecules. arXiv:1610.02415 (2016)

  32. Gramacy, R.B.: TGP: an R package for Bayesian nonstationary, semiparametric nonlinear regression and design by treed Gaussian process models. J. Stat. Softw. 19(9), 1–46 (2007)

    Article  Google Scholar 

  33. Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)

    Google Scholar 

  34. Haftka, R.T., Villanueva, D., Chaudhuri, A.: Parallel surrogate-assisted global optimization with expensive functions–a survey. Struct. Multidiscip. Optim. 54(1), 3–13 (2016)

    Article  MathSciNet  Google Scholar 

  35. Hansen, N., Auger, A., Mersmann, O., Tušar, T., Brockhoff, D.: COCO: a platform for comparing continuous optimizers in a black-box setting. arXiv:1603.08785 (2016)

  36. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn, Springer, Berlin (2009)

    Google Scholar 

  37. Hildebrandt, T., Branke, J.: On using surrogates with genetic programming. Evol. Comput. 23(3), 343–367 (2015)

    Article  Google Scholar 

  38. Ho, Y., Pepyne, D.: Simple explanation of the no-free-lunch theorem and its implications. J. Optim. Theory Appl. 115(3), 549–570 (2002)

    Article  MathSciNet  Google Scholar 

  39. Huang, E., Xu, J., Zhang, S., Chen, C.H.: Multi-fidelity model integration for engineering design. Procedia Comput. Sci. 44, 336–344 (2015)

    Article  Google Scholar 

  40. Hussein, R., Deb, K.: A generative kriging surrogate model for constrained and unconstrained multi-objective optimization. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2016, pp. 573–580 (2016)

    Google Scholar 

  41. Jin, R., Chen, W., Simpson, T.W.: Comparative studies of metamodelling techniques under multiple modelling criteria. Struct. Multidiscip. Optim. 23(1), 1–13 (2001)

    Article  Google Scholar 

  42. Jin, Y.: Surrogate-assisted evolutionary computation: recent advances and future challenges. Swarm Evol. Comput. 1(2), 61–70 (2011)

    Article  Google Scholar 

  43. Jin, Y., Branke, J.: Evolutionary optimization in uncertain environments-a survey. IEEE Trans. Evol. Comput. 9(3), 303–317 (2005)

    Article  Google Scholar 

  44. Joachims, T.: A support vector method for multivariate performance measures. In: Proceedings of the Twenty-Second International Conference on Machine Learning, ICML 2005, pp. 377–384 (2005)

    Google Scholar 

  45. Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13, 455–492 (1998)

    Article  MathSciNet  Google Scholar 

  46. Knowles, J.: Parego: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Trans. Evol. Comput. 10(1), 50–66 (2006)

    Article  Google Scholar 

  47. Knowles, J., Nakayama, H.: Meta-modeling in multiobjective optimization. In: Multiobjective optimization, pp. 245–284. Springer, Berlin (2008)

    Chapter  Google Scholar 

  48. Le, M.N., Ong, Y.S., Menzel, S., Jin, Y., Sendhoff, B.: Evolution by adapting surrogates. Evol. Comput. 21(2), 313–340 (2013)

    Article  Google Scholar 

  49. Loshchilov, I., Schoenauer, M., Sebag, M.: Dominance-based Pareto-surrogate for multi-objective optimization. In: Proceedings of the 8th International Conference on Simulated Evolution and Learning, SEAL 2010, LNCS, vol. 6457, pp. 230–239 (2010)

    Google Scholar 

  50. Mezura-Montes, E., Coello Coello, C.A.: A survey of constraint-handling techniques based on evolutionary multiobjective optimization (2006)

    Google Scholar 

  51. Montgomery, D.C.: Design and Analysis of Experiments, 5th edn, Wiley (2001)

    Google Scholar 

  52. Müller, J., Shoemaker, C.A.: Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems. J. Glob. Optim. 60(2), 123–144 (2014)

    Article  MathSciNet  Google Scholar 

  53. Murphy, K.P.: Machine Learning: A Probabilistic Perspective. MIT press (2012)

    Google Scholar 

  54. Nelson, A., Alonso, J., Pulliam, T.: Multi-fidelity aerodynamic optimization using treed meta-models. In: Proceedings of the 25th AIAA Applied Aerodynamics Conference, pp. 1–19. American Institute of Aeronautics and Astronautics (2012)

    Google Scholar 

  55. Rao, S.V.N., Manju, S.: Optimal pumping locations of skimming wells. Hydrol. Sci. J. 52(2), 352–361 (2007)

    Article  Google Scholar 

  56. Robinson, T., Eldred, M., Willcox, K., Haimes, R.: Surrogate-based optimization using multifidelity models with variable parameterization and corrected space mapping. Aiaa J. 46(11), 2814–2822 (2008)

    Article  Google Scholar 

  57. Romero, P.A., Krause, A., Arnold, F.H.: Navigating the protein fitness landscape with Gaussian processes. Proc. Natl. Acad. Sci. 110(3), E193–E201 (2013)

    Article  MathSciNet  Google Scholar 

  58. Sanchez, E., Pintos, S., Queipo, N.V.: Toward an optimal ensemble of kernel-based approximations with engineering applications. In: Proceedings of the 2006 IEEE International Joint Conference on Neural Network, pp. 2152–2158 (2006)

    Google Scholar 

  59. Shan, S., Wang, G.G.: Space exploration and global optimization for computationally intensive design problems: a rough set based approach. Struct. Multidiscip. Optim. 28(6), 427–441 (2004)

    Article  Google Scholar 

  60. Shan, S., Wang, G.G.: Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions. Struct. Multidiscip. Optim. 41(2), 219–241 (2010)

    Article  MathSciNet  Google Scholar 

  61. Shi, L., Rasheed, K.: A survey of fitness approximation methods applied in evolutionary algorithms. In: Computational Intelligence in Expensive Optimization Problems, pp. 3–28. Springer, Berlin (2010)

    Chapter  Google Scholar 

  62. Simpson, T., Toropov, V., Balabanov, V., Viana, F.: Design and analysis of computer experiments in multidisciplinary design optimization: a review of how far we have come—or not. In: Proceedings of the 12th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, pp. 1–22 (2012)

    Google Scholar 

  63. Singh, P., Couckuyt, I., Ferranti, F., Dhaene, T.: A constrained multi-objective surrogate-based optimization algorithm. In: Proceedings of the IEEE Congress on Evolutionary Computation, CEC 2014, pp. 3080–3087 (2014)

    Google Scholar 

  64. Smith, J., Stone, C., Serpell, M.: Exploiting diverse distance metrics for surrogate-based optimisation of ordering problems: A case study. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2016, pp. 701–708 (2016)

    Google Scholar 

  65. Smola, A., Vapnik, V.: Support vector regression machines. Adv. Neural Inf. Process. Syst. 9, 155–161 (1997)

    Google Scholar 

  66. Sutton, R.S., Barto, A.G.: Introduction to Reinforcement Learning, 1st edn, MIT Press (1998)

    Google Scholar 

  67. Swersky, K., Duvenaud, D., Snoek, J., Hutter, F., Osborne, M.: Raiders of the lost architecture: kernels for Bayesian optimization in conditional parameter spaces. arXiv:1409.4011 (2014)

  68. Teixeira, C., Covas, J.A., Stützle, T., Gaspar-Cunha, A.: Multi-objective ant colony optimization for the twin-screw configuration problem. Eng. Optim. 44(3), 351–371 (2012)

    Article  Google Scholar 

  69. Tenne, Y., Armfield, S.W.: A versatile surrogate-assisted memetic algorithm for optimization of computationally expensive functions and its engineering applications. In: Success in Evolutionary Computation, pp. 43–72. Springer Berlin Heidelberg (2008)

    Google Scholar 

  70. Theiler, J., Galdrikian, B., Longtin, A., Eubank, S., Farmer, J.D.: Using surrogate data to detect nonlinearity in time series. Technical Report, Los Alamos National Laboratory, NM (United States) (1991)

    Google Scholar 

  71. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Methodolgical) 58(1), 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  72. Ursem, R.K.: Multinational gas: Multimodal optimization techniques in dynamic environments. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2000, pp. 19–26 (2000)

    Google Scholar 

  73. Vapnik, V.N., Chervonenkis, A.J.: Theory of Pattern Recognition. Nauka (1974)

    Google Scholar 

  74. Voutchkov, I., Keane, A., Bhaskar, A., Olsen, T.M.: Weld sequence optimization: the use of surrogate models for solving sequential combinatorial problems. Comput. Methods Appl. Mech. Eng. 194(30–33), 3535–3551 (2005)

    Article  Google Scholar 

  75. Wall, M.E., Rechtsteiner, A., Rocha, L.M.: Singular value decomposition and principal component analysis. In: A Practical Approach To Microarray Data Analysis, pp. 91–109. Springer, Berlin (2003)

    Google Scholar 

  76. Wang, H.: Forward regression for ultra-high dimensional variable screening. J. Am. Stat. Assoc. 104(488), 1512–1524 (2009)

    Article  MathSciNet  Google Scholar 

  77. Whitley, L.D., Mathias, K.E., Rana, S., Dzubera, J.: Building better test functions. In: Proceedings of the Sixth International Conference on Genetic Algorithms, pp. 239–246 (1995)

    Google Scholar 

  78. Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)

    Article  Google Scholar 

  79. Zaefferer, M., Bartz-Beielstein, T.: Efficient global optimization with indefinite kernels. In: Proceedings of the 14th International Conference Parallel Problem Solving from Nature, PPSN XIV, pp. 69–79 (2016)

    Chapter  Google Scholar 

  80. Zaefferer, M., Fischbach, A., Naujoks, B., Bartz-Beielstein, T.: Simulation-based test functions for optimization algorithms. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2017, pp. 905–912 (2017)

    Google Scholar 

  81. Zaefferer, M., Stork, J., Friese, M., Fischbach, A., Naujoks, B., Bartz-Beielstein, T.: Efficient global optimization for combinatorial problems. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2014, pp. 871–878 (2014)

    Google Scholar 

  82. Zerpa, L.E., Queipo, N.V., Pintos, S., Salager, J.L.: An optimization methodology of alkaline-surfactant-polymer flooding processes using field scale numerical simulation and multiple surrogates. J. Pet. Sci. Eng. 47(3–4), 197–208 (2005)

    Article  Google Scholar 

Download references

Acknowledgements

This work is part of a project that has received funding from the European Union’s Horizon 2020 research and innovation program under Grant Agreement no. 692286. Tea Tušar acknowledges financial support from the Slovenian Research Agency (project no. Z2–8177).

Author information

Authors and Affiliations

  1. Institute of Data Science, Engineering, and Analytics, Faculty of Computer Science and Engineering Science, TH Köln, Steinmüllerallee 1, 51643, Gummersbach, Germany

    Jörg Stork, Martina Friese, Martin Zaefferer, Thomas Bartz-Beielstein, Andreas Fischbach, Beate Breiderhoff & Boris Naujoks

  2. Department of Intelligent Systems, Jožef Stefan Institute, Jamova cesta 39, 1000, Ljubljana, Slovenia

    Tea Tušar

Authors
  1. Jörg Stork
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martina Friese
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Martin Zaefferer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas Bartz-Beielstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Andreas Fischbach
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Beate Breiderhoff
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Boris Naujoks
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Tea Tušar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martina Friese .

Editor information

Editors and Affiliations

  1. TH Köln, Cologne, Germany

    Thomas Bartz-Beielstein

  2. Jožef Stefan Institute, Ljubljana, Slovenia

    Bogdan Filipič

  3. Jožef Stefan Institute, Ljubljana, Slovenia

    Peter Korošec

  4. University Lille, Lille, France

    El-Ghazali Talbi

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Stork, J. et al. (2020). Open Issues in Surrogate-Assisted Optimization. In: Bartz-Beielstein, T., Filipič, B., Korošec, P., Talbi, EG. (eds) High-Performance Simulation-Based Optimization. Studies in Computational Intelligence, vol 833. Springer, Cham. https://doi.org/10.1007/978-3-030-18764-4_10

Download citation

Publish with us

Policies and ethics

Search

Navigation