Abstract
Global optimization involves the difficult task of the identification of global extremities of mathematical functions. Such problems are often encountered in practice in various fields, e.g., molecular biology, physics, industrial chemistry. In this work, we develop five different parallel Simulated Annealing (SA) algorithms and compare them on an extensive test bed used previously for the assessment of various solution approaches in global optimization. The parallel SA algorithms consist of various categories: the asynchronous approach where no information is exchanged among parallel runs and the synchronous approaches where solutions are exchanged using genetic operators, or where solutions are transmitted only occasionally, or where highly coupled synchronization is achieved at every iteration. One of these approaches, which occasionally applies partial information exchanges (controlled in terms of solution quality), provides particularly notable results for functions with vast search spaces of up to 400 dimensions. Previous attempts with other approaches, such as sequential SA, adaptive partitioning algorithms and clustering algorithms, to identify the global optima of these functions have failed without exception.
Similar content being viewed by others
References
Aarts, E.H. and Korst, J.H.M. (1991), Boltzmann machines as a model for parallel annealing, Algorithmica, 6: 437-465.
Aarts, E.H., de Bont, F., Haberts, E. and van Laarhoven, P. (1986), Parallel implementations of the statistical cooling algorithm, VLSI Journal, 4: 209-238.
Ali, M.M. and Storey, C. (1994), Topographical Multi Level Single Linkage. Journal of Global Optimization, 5: 349-358.
Aluffi-Pentini, F., Parisi, V. and Zirilli, F. (1985), Global optimization and stochastic differential equations. Journal of Optimization Theory and Applications, 47: 1-16.
Androulakis, G.S. and Vrahatis, M.N. (1996), OPTAC: a portable software packaage for analyzing and comparing optimization methods by visualization. Journal of Computational and Applied Mathematics 72: 41-62.
Balardi, F. and Orlando, S. (1989), Strategies for a massively parallel implementaion of simulated annealing, Parallel Architectures and Languages, PARLE '89: 273-287.
Baluja, S. (1994), Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning. Working Paper, CMU-CS-94-163, School of Computer Science, Carnegie Mellon University.
Bilchev, G. and Parmee, I. (1996), Inductive search. IEEE Trans. 832-836.
Botsaris, C.A. (1996), A curvilinear optimization method based upon iterative estimation of the eigensystem of the hessian matrix. J. of Mathematical Analysis and Applications, 63: 396-411.
Breiman, L. and Cutler, A. (1993), A deterministic algorithm for global optimization. Mathematical Programming, 58: 179-199.
Casotto, A. Romeo, F. and Sangiovanni-Vincentelli, A. (1987), A parallel simulated annealing algorithm for placement of macro-cells, IEEE Trans. on Computer Aided Design, 6: 838-847.
Chen, H., Flann, N.S. and Watson, D.W. (1998), Parallel genetic simulated annealing: A massively parallel SIMD algorithm, IEEE Trans. on Parallel and Distributed Systems, 9: 126-136.
Corana, A., Marchesi, M., Marini, C. and Ridella, S. (1987), Minimizing multimodal functions of continuous variables with the simulated annealing algorithm, ACM Trans. of Mathl. Software 13: 262-279.
De Jong, K. (1975), An analysis of the behaviour of a class of genetic adaptive systems. Ph.D. Thesis, University of Michigan, Ann Arbor.
Dekkers, A. and Aarts, E. (1991), Global optimization and simualted annealing. Mathematical Programming, 50: 367-393.
Demirhan, M., Özdamar, L., Helvacio?glu, L. and Birbil, ¸S.I. (1999), FRACTOP: A Geometric partitioning metaheuristic for global optimization, Journal of Global Optimization, 14: 415-435.
Diekmann, R., Lüling, R. and Simon, J. (91193), Problem independent distributed simulated annealing and its applications. Working Paper, University of Paderborn, Dept. of Mathematics and Computer Science, Germany.
Easom, E. (1990), A survey of global optimization techniques. M. Eng. Thesis, University of Louisville, Louisville, KY.
Frost, R. (1993), Ensemble based simulated annealing, ftp.sdsc.edu/pub/sdsc/math/Ebsa, La Jolla, CA, University of California, San Diego.
Greening, D.R. (1990), Parallel simulated annealing techniques, Physica, D42: 293-306.
Jansson, C. and Knüppel, O. (1994), Numerical Results for a self-validating global optimization method. Working Paper, No. 94.1, Technical University of Hamburg, Harburg.
Horst, R. and Tuy, H. (1996), Global Optimization-Deterministic Approaches, 3rd ed. Springer Verlag.
Ingber, L. (1994), Simulated annealing: Practice versus theory. J. Mathl. Comput. Modelling, 18(11): 29-57.
Ingber, L. (1995), Adaptive simulated annealing (ASA): Lessons learned. To appear in Control and Cybernetics, (Polish Journal).
Rinnooy Kan, A.H.G. and Timmer, G.T. (1994), Stochastic methods for global optimization. American J. of Mathematical Management Science 4: 7-40.
Kearfott, B. (1979), An efficient degree-computation method for a generalized method of bisection. Numerical Mathematics 32: 109-127.
Kirkpatrick, A., Gelatt, Jr., C.D. and Vechi, M.P. (1983), Optimization by simulated annealing, Science 220:671-680.
Kliewer, G. and Tschöke, S. (1998), A general parallel simulated annealing library and its applications in industry. Working Paper, University of Paderborn, Dept. of Mathematics and Computer Science, Germany.
Kravitz, S.A. and Rutenbar, R.A. (1987), Placement by simulated annealing on a multi-processor, IEEE Trans. on Computer Aided Design 6: 534-549.
Kurbel, K., Schneider, B. and Singh, K. (1998), Solving optimization problems by parallel recombinative simulated annealing on a parallel computer-An application to standard cell placement in VSLI design, IEEE Trans. on Systems, Man and Cybernetics-Part B-Cybernetics 28: 454-460.
Lee, S-Y. and Lee, K.G. (1996), Synchronous and asynchronous parallel simulated annealing with multiple Markov Chains. IIE Transactions on Parallel and Distributed Systems 7.
Levy, A.V., Montalvo, A., Gomez and S., Calderon, A. (1981), Topics in Global Optimization: Lecture Notes in Mathematics, No. 909, Springer-Verlag, Berlin.
Lin, F.T., Kao, C.Y. and Hsu, C.C. (1991), Incorporating genetic algorithms into simulated annealing, F.J Cantu-Ortiz et al. (eds), Proceedings of the Int. Symposium on Artificial Intelligence: pp. 290-297.
Michalewicz, Z. (1994), Genetic Algorithms+Data Structures=Evolution Programs, Springer Verlag, Berlin.
Moore, R.E. (1979), Methods and applications of interval analysis. SIAM, Philadelphia.
More, B. J., Garbow, B.S. and Hillstrom, K.E. (1981), Testing unconstrained optimization. ACM Trans. Math. Software 7: 17-41.
Moore, R.E. and Ratschek, H. (1988), Inclusion functions and global optimization II. Mathematical Programming 41: 341-356.
Özdamar, L. and Birbil, I. ¸S. (1998), A hybrid genetic algorithm for the capacitated lot sizing and loading problem, European Journal of Operational Research 110: 525-547.
Özdamar, L. and Demirhan, M. (2000a), Experiments with new stochastic global optimization search techniques, Computers and OR 27: 841-865.
Özdamar, L and Demirhan, M. (2000b), Comparison of partition evaluation measures in an adaptive partitioning algorithm for global optimization, to appear in Fuzzy Sets and Systems.
Özdamar, L. (1999), A genetic algorithm approach for the multi-mode resource-constrained project scheduling problem under general resource categories. IEEE Transactions on Systems, Man and Cybernetics 29: 44-69.
Pinter, J. (1992), Convergence qualification of adaptive partitioning algorithms in global optimization, Mathematical Programming 56: 343-360.
Press, W.H., Teukolsky, S.A., Vetterling, W.T. and Flannery, B.P. (1992), Numerical Recipes, The Art of Scientific Computing,. Cambridge University Press, New York.
Price, W.L. (1998), A controlled random search procedure for global optimization, in L.C.W. Dixon and G.P. Szegö (eds), Towards Global Optimization 2, North-Holland, Amsterdam.
Rosenbrock, H.H. (1970), State-Space and Multivariable Theory, Nelson, London.
Schaffer, J.D. (1989), A study of control parameters affecting online performance of genetic algorithms for function optimization. Proc. of the Third Int. Con. on Genetic Algorithms: 51-60.
Schwefel, H. (1991), Numerical Optimization of Computer Models, Wiley, New York.
Srinivas, M. and Patnaik, L.M. (1994), Adaptive probabilities of crossover and mutation in genetic algorithms. IEEE Transactions on Systems, Man and Cybernetics 24: 656-667.
Stenger, F. (1975), Computing the topological degree of a mapping in @n, Numerical Mathematics 25: 23-38.
Stuckman, B.E. and Easom, E.E. (1992), A comparison of Bayesian/Sampling global optimization techniques, IEEE Transactions on Systems, Man and Cybernetics 22: 1024-1032.
Törn, A. (1978), A search clustering approach to global optimization, in L.C.W. Dixon and G.P. Szegö (eds), Towards Global Optimization 2, North-Holland, Amsterdam.
Törn, A. and Zilinskas, A. (1989), Global Optimization. Lecture Notes in Computer Science, Springer-Verlag, Berlin.
Törn, A. and Viitanen, S. (1994), Topographical global optimization using pre-sampled points. Journal of Global Optimization 5: 267-276.
Törn, A., Ali, M.M. and Viitanen, S. (1999), Stochastic global optimization: Problem classes and solution techniques, Journal of Global Optimization 14: 437-447.
Wendt, O. and König, W. (1998), Cooperative simulated annealing: How much cooperation is enough?, Working Paper, Frankfurt University.
Wingo, D.R. (1983), Estimating the location of the Cauchy distribution by numerical global optimization. Communications in Statistics Part B. Simulation and Computation 12: 201-212.
Wodrich, M. and Bilchev, G. (1997), Cooperative distributed search: The ant's way. Working Paper, University of Cape Town, South Africa.
Zacharias, C.R., Lemes, M.R. and Pino, A.D. (1998), Combining genetic algorithms and simulated annealing: a molecular geometry optimization study. THEOCHEM-Journal of Molecular Structure 430: 29-39.
Zilinskas, A. (1981), Two algorithms for one-dimensional multimodal minimization, Optimization 12, 53-63.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Onbaşoğlu, E., Özdamar, L. Parallel Simulated Annealing Algorithms in Global Optimization. Journal of Global Optimization 19, 27–50 (2001). https://doi.org/10.1023/A:1008350810199
Issue Date:
DOI: https://doi.org/10.1023/A:1008350810199