Cited By
View all- Wu HWang CLu CZhan T(2024)HCT: a hybrid CNN and transformer network for hyperspectral image super-resolutionMultimedia Systems10.1007/s00530-024-01387-930:4Online publication date: 20-Jun-2024
Optimizing with Attractor: A Tutorial
Author: 
Weiqi LiAuthors Info & Claims
Weiqi LiAuthors Info & ClaimsArticle No.: 235, Pages 1 - 41
Abstract
This tutorial presents a novel search system—the Attractor-Based Search System (ABSS)—that can solve the Traveling Salesman Problem very efficiently with optimality guarantee. From the perspective of dynamical systems, a heuristic local search algorithm for an NP-complete combinatorial problem is a discrete dynamical system. In a local search system, an attractor drives the search trajectories into the vicinity of a globally optimal point in the solution space, and the convergence of local search trajectories makes the search system become a global and deterministic system. The attractor contains a small set of the most promising solutions to the problem. The attractor can reduce the problem size exponentially, and thus make the exhaustive search feasible. Therefore, this new search paradigm is called optimizing with attractor. The ABSS consists of two search phases: local search phase and exhaustive search phase. The local search process is used to quickly construct the attractor in the solution space, and the exhaustive search process is used to completely search the attractor to identify the optimal solution. Therefore, the exact optimal solution can be found quickly by combining local search and exhaustive search. This tutorial introduces the concept of an attractor in a local search system, and describes the process of optimizing with the attractor, using the Traveling Salesman Problem as the study platform.
References
[1]
Emile H. L. Aarts and Jan K. Lenstra. 2003. Local Search in Combinatorial Optimization. Princeton University Press, Princeton.
[2]
Kathleen T. Alligood, Tim D. Sauer, and James A. Yorke. 1997. Chaos: Introduction to Dynamical Systems. Springer, New York.
[3]
David L. Applegate, Robert E. Bixby, Vašek Chvátal, and William J. Cook. 2006. The Traveling Salesman Problem: A Computational Study. Princeton University Press, Princeton, NJ, USA.
[4]
David L. Applegate, Robert E. Bixby, Vašek Chvátal, and William J. Cook. Concorde website. (March 2015). Retrieved November 15, 2023 from https://www.math.uwaterloo.ca/tsp/concorde/index.html
[5]
Sanjeev Arora and Boaz Barak. 2009. Computational Complexity: A Modern Approach. Cambridge University Press, Cambridge.
[6]
J. Auslander, N. P. Bhatia, and P. Seibert. 1964. Attractor in Dynamical Systems. NASA Technical Report NASA-CR-59858.
[7]
Gérad M. Baudet. 1978. On the branching factor of the alpha-beta pruning algorithm. Artificial Intelligence 10, 2 (1978), 173–199.
[8]
Ricardo P. Beausoleil, Gulnara Baldoquin, and Rodolfo A. Montejo. 2008. Multi-start and path relinking methods to deal with multi-objective knapsack problem. Annals of Operations Research 157, 1 (2008), 105–133.
[9]
Edward J. Beltrami. 1998. Mathematics for Dynamic Modeling. Academic Press, San Diego.
[10]
Mauro Birattari, Paola Pellegrini, and Marco Dorigo. 2007. On the invariance of ant colony optimization. IEEE Transactions on Evolutionary Computation 11, 6 (2007), 732–742.
[11]
Christian Blum and Andrea Roli. 2003. Metaheuristics in combinatorial optimization: Overview and conceptual comparison. ACM Computing Surveys 35, 3 (2003), 268–308.
[12]
Kenneth D. Boese, Andrew B. Kahng, and Sushaker Muddu. 1994. New adaptive multistart techniques for combinatorial global optimizations. Operations Research Letter 16, 2 (1994), 101–113.
[13]
Michael Brin and Garrett Stuck. 2016. Introduction to Dynamical Systems. Cambridge University Press, Cambridge.
[14]
Karl Bringmann, Tobias Friedrich, Frank Neumann, and Markus Wagner. 2011. Approximation-guided evolutionary multi-objective optimization. In Proceedings of the 22nd International Conference on Artificial Intelligence, Vol. 2, 1198–1203.
[15]
Richard J. Brown. 2018. A Modern Introduction to Dynamical Systems. Oxford University Press, Oxford.
[16]
Barun Chandra, Howard Karloff, and Craig Tovey. 1999. New results on the old k-opt algorithm for the traveling salesman problem. SIAM Journal on Computing 28, 6 (1999), 1998–2029.
[17]
Carlos A. Coello Coello, Gary B. Lamont, and David A. Van Veldhuisen. 2007. Evolutionary Algorithms for Solving Multi-Objective Problems. Springer, Berlin.
[18]
Jared L. Cohon. 2004. Multiobjective Programming and Planning. Courier Dover Publications, Mineola.
[19]
Stephen Cook. 2022. The P versus NP Problem. Clay Mathematics Institute. http://www.claymath.org/sites/default/files/pvsnp.pdf
[20]
William Cook. 2012. In Pursuit of the Traveling Salesman: Mathematics at the Limits of Computation. Princeton University Press, Princeton.
[21]
William Cook, William H. Cunningham, William R. Pulleyblank, and Alexander Schrijver. 1998. Combinatorial Optimization. John Wiley & Sons, New York.
[22]
Thomas Cormen. 2001. Introduction to Algorithms. MIT Press, Cambridge.
[23]
Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. 2009. Introduction to Algorithms. MIT Press, Cambridge.
[24]
David Corne, Marco Dorigo, and Fred Glover. 1999. New Ideas in Optimization. McGraw-Hill, London.
[25]
Thomas M. Cover and Joy A. Thomas. 2006. Elements of Information Theory. Wiley, New York.
[26]
G. A. Croes. 1958. A method for solving traveling-salesman problem. Operations Research 6, 6 (1958), 791–812.
[27]
Attila Dénes and Géza Makey. 2011. Attractors and basins of dynamical systems. Electronic Journal of Qualitative Theory of Differential Equations 20, 20 (2011), 1–11.
[28]
Marco Dorigo. 1992. Optimization, Learning and Natural Algorithms. Ph.D. Thesis, Dip Electronica e Informazione, Politecnico Di Milano, Italy.
[29]
Marco Dorigo and Luca M. Gambardella. 1997. Ant colony system: A cooperative learning approach to the traveling salesman problem. IEEE Transactions on Evolutionary Computation 1, 1 (1997), 53–66.
[30]
Marco Dorigo and Thomas Stützle. 2004. Ant Colony Optimization. The MIT Press, Cambridge.
[31]
Ding-Zhu Du and Ker-l Ko. 2000. Theory of Computational Complexity. John Wiley & Sons, New York.
[32]
Ding-Zhu Du, Panos M. Pardalos, and Weili Wu. 2008. History of optimization. In Encyclopedia of Optimization. Christodoylos Floudas and Panos M. Pardalos (Eds.). Springer, Boston.
[33]
Stefan Edelkamp and Richard E. Korf. 1998. The branching factor of regular search space. In Proceedings of the 15thNational Conference on Artificial Intelligence. 292–304.
[34]
Matthias Ehrgott. 2005. Multicriteria Optimization (2nd ed.). Springer, New York.
[35]
Thomas A. Feo and Mauricio G. C. Resende. 1989. A probabilistic heuristic for a computationally difficult set covering problem. Operations Research Letters, 8, 2 (1989), 67–71.
[36]
Thomas A. Feo and Mauricio G. C. Resende. 1995. Greedy randomized adaptive search procedure. Journal of Global Optimization 2 (1995), 1–27.
[37]
Sophie T. Fischer. 1995. A note on the complexity of local search problems. Information Processing Letters 53, 2 (1995), 69–75.
[38]
Marshall L. Fisher and Alexander H. G. Rinnooy Kan. 1988. The design, analysis, and implementation of heuristics. Management Science 34, 3 (1988), 263–265.
[39]
Lance Fortnow. 2009. The status of the P versus NP problem. Communications of the ACM 52, 9 (2009), 78–86.
[40]
Lance Fortnow. 2013. The Golden Ticket—P, NP, and the Search for the Impossible. Princeton University Press. Princeton.
[41]
Paul A. Gagniuc. 2017. Markov Chains: From Theory to Implementation and eExperimentation. John Wiley & Sons, New York.
[42]
Robert G. Gallager. 1991. Information Theory and Reliable Communication. Wiley, New York.
[43]
Michael R. Garey and David Johnson. 1979. Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freemen and Company, New York.
[44]
Michel Gendreau and Jean-Yves Potvin. 2019. Handbook of Metaheuristics,Vol. 272. International Series in Operations Research & Management Science. Springer, New York.
[45]
Fred Glover and Abraham P. Punnen. 1997. The traveling salesman problem: New solvable cases and linkages with the development of approximation algorithms. Journal of the Operational Research Society 48, 5 (1997), 502–510.
[46]
Oded Goldreich. 2008. Computational Complexity: A Conceptual Perspective. Cambridge University Press, Cambridge.
[47]
Oded Goldreich. 2010. P, NP, and NP-Completeness. Cambridge University Press, Cambridge.
[48]
Jonatan Gomez. 2019. Stochastic global optimization algorithms: A systematic formal approach. Information Science 472 (2019), 53–76.
[49]
Peter Grassberger and Itamar Procaccia. 1983. Characterization of strange attractor. Physical Review Letters 50, 5 (1983), 346–349.
[50]
Gregory Gutin and Abraham P. Punnen. 2007. The Traveling Salesman Problem and Its Variations. Springer, New York, NY.
[51]
András György and Levente Kocsis. 2011. Efficient multi-start strategies for local search algorithms. Journal of Artificial Intelligence Research 41, 2 (2011), 407–444.
[52]
Doug Hains, Darrell Whitley, and Adele Howe. 2001. Revisiting the big valley search space structure in the TSP. Journal of Operational Research Society 62, 2 (2011), 305–312.
[53]
Juris Hartmanis and John E. Hopcroft. 1971. An overview of the theory of computational complexity. Journal of the ACM 18, 3 (1971), 444–475.
[54]
Wilfred K. Hastings. 1970. Monte Carlo sampling methods using Markov chains and their applications. Biametnika 57, 1 (1970), 97–109.
[55]
Jane Hawkins. 2021. Attractors in dynamical systems. In Ergodic Dynamics. Graduate Texts in Mathematics, Vol. 289. Springer, New York.
[56]
Keld Helsgaun. 2000. An effective implementation of the Lin–Kernighan traveling salesman heuristic. European Journal of Operational Research 126, 1 (2000), 106–130.
[57]
Keld Helsgaun. 2009. General k-opt submoves for the Lin–Kernighan TSP heuristic. Mathematical Programming Computation 1 (2009), 119–163.
[58]
Israel N. Herstein. 1964. Topics in Algebra. Blaisdell Publishing Company, Waltham, MA.
[59]
Holger H. Hoos and Thomas Stűtzle. 2004. Stochastic Local Search: Foundations and Applications. Morgan Kaufmann, New York.
[60]
Reiner Horst and Panos M. Pardalos. 1995. Handbook of Global Optimization: Nonconvex Optimization and Its Application. Springer, New York.
[61]
Reiner Horst, Panos M. Pardalos, and Nguyen V. Thoai. 2000. Introduction to Global Optimization: Nonconvex Optimization and Its Applications. Springer, New York.
[62]
David S. Johnson. 1990. Local optimization and the Traveling Salesman Problem. In Automata, Languages and Programming (ICALP1990). Lecture Notes in Computer Science, Vol. 443. Springer, Berlin, 446–461.
[63]
David S. Johnson and Lyle A. McGeoch. 2007. Experimental analysis of heuristics for the STSP. In The Traveling Salesman Problem and Its Variations. Combinatorial Optimization. Gregory Gutin and Abraham P. Punnen (Eds.). Springer, Boston, 369–443.
[64]
Scott Kirkpatrick and Gérard Toulouse. 1985. Configuration space analysis of traveling salesman problems. Journal de Physique 46, 8 (1985), 1277–1292.
[65]
Vassilli N. Kolokoltsov. 2010. Nonlinear Markov Process and Kinetic Equations. Cambridge University Press, Cambridge.
[66]
Richard E. Korf. 1985. Depth-first iterative deepening: An optimal admissible tree search. Artificial Intelligence 27, 1 (1985), 97–109.
[67]
Bernhard Korte and Jens Vygen. 2018. Combinatorial Optimization: Theory and Algorithms. Springer, New York.
[68]
Dirk P. Kroese, Thomas Taimre, and Zdravko I. Botev. 2011. Handbook of Monte Carlo Methods. John Wiley & Sons, New York.
[69]
Manuel Laguna and Rafael Martí. 1999. GRASP and path relinking for a 2-player straight line crossing minimization. INFORMS Journal on Computing 11, 1 (1999), 44–52.
[70]
Kenneth Lange. 2010. Local and global convergence. Numerical Analysis for Statisticians (Statistics and Computing). Springer, New York, 277–296.
[71]
Eugene L. Lawler, Jan K. Lenstra, Alexander Rinnooy-Kan, and David Shmoys. 1985. The Traveling Salesman Problems: A Guided Tour of Combinatorial Optimization. John Wiley & Sons, New York.
[72]
Shen Lin. 1965. Computer solution of the traveling salesman problem. The Bell System Technical Journal 44, 10 (1965), 2245–2268.
[73]
Shen Lin and Brian W. Kernighan. 1973. An effective heuristic algorithm for the traveling-salesman problem. Operations Research 21, 2 (1973), 498–516.
[74]
Samir W. Mahfoud. 1996. Niching Methods for Genetic Algorithms. University of Illinois Press, Champaign.
[75]
Rafael Martí, Hose A. Lozano, Alexander Mendiburu, and Leticia Hernando. 2015. Multi-start methods. In Handbook of Heuristics. Springer, Boston, MA, USA, 1–21.
[76]
Rafael Martí, Jose M. Moreno-Vega, and Abraham Duarte. 2010. Advanced multi-start methods. In Handbook of Metaheuristics. Springer, Boston, 265–281.
[77]
Rafael Martí, Mauricio G. C. Resende, and Celso C. Ribeiro. 2013. Multi-start methods for combinatorial optimization. European Journal of Operational Research 226, 1 (2013), 1–8.
[78]
Olivier Martin, Steve W. Otto, and Edward W. Felten. 1992. Large step Markov chains for the TSP incorporating local search heuristics. Operations Research Letters 11, 4 (1992), 219–224.
[79]
Lyle A. McGeoch. 2016. Implementation challenge for TSP heuristics. In Encyclopedia of Algorithms. Springer, New York, 952–954.
[80]
Sean Meyn and Richard L. Tweedie. 2009. Markov Chains and Stochastic Stability. Cambridge University Press, Cambridge.
[81]
Marc Mézard and Giorgio Parisi. 1986. A replica analysis of the traveling salesman problem. Journal de Physique 47, 8 (1986), 1285–1296.
[82]
Zbigniew Michalewicz and David B. Fogel. 2004. How to Solve It: Modern Heuristics. Springer, Berlin.
[83]
Anthony N. Michel, Kaining Wang, and Bo Hu. 2001. Qualitative Theory of Dynamical Systems. Marcel Dekker, New York.
[84]
Kaisa Miettinen. 1999. Nonlinear Multiobjective Optimization. Kluwer Academic Publishers, Bordrecht.
[85]
John Milnor. 1985. On the concept of attractor. Communications in Mathematical Physics 99, 2 (1985), 177–196.
[86]
John Milnor. 2010. Collected Papers of John Milnor VI: Dynamical Systems (1953–2000). American Mathematical Society, Providence.
[87]
Samuel A. Mulder and Donald C. Wunsch. 2003. Million city traveling salesman problem solution by divide and conquer clustering with adaptive resonance neural networks. Neural Networks 16, 5-6 (2003), 827–832.
[88]
Yuichi Nagate and Shigenobu Kobayashi. 2013. A powerful genetic algorithm using edge assembly crossover for the traveling salesman problem. INFORMS Journal on Computing 25, 2 (2013), 346–363.
[89]
Monmarché Nicolas, Guinand Frédéric, and Siarry Patrick. 2010. Artificial Ants. Wiley, New York.
[90]
Gabriela Ochoa and Nadarajen Veerapen. 2016. Deconstructing the big valley search space hypothesis. In Proceedings of Evolutionary Computation in Combinatorial Optimization (EvoCOP ’16). Lecture Notes in Computer Science, Vol. 9595. Springer.
[91]
Christos H. Papadimitriou. 1993. Computational Complexity. Pearson, New York.
[92]
Christos H. Papadimitriou and Kenneth Steiglitz. 1977. On the complexity of local search for the traveling salesman problem. SIAM Journal on Computing 6, 1 (1977), 76–83.
[93]
Christos H. Papadimitriou and Kenneth Steiglitz. 1999. Combinatorial Optimization: Algorithms and Complexity. Dover Publications, New York.
[94]
Panos M. Pardalos and H. Edwin Romeijn. 2002. Handbook of Global Optimization Volume 2: Nonconvex Optimization and Its Application. Springer, New York.
[95]
Judea Pearl. 1982. The solution for the branching factor of the alpha-beta pruning algorithm and its optimality. Communications of the ACM 25, 8 (1982), 559–564.
[96]
Judea Pearl. 1984. Heuristics: Intelligent Search Strategies for Computer Problem Solving. Addison-Wesley, New York.
[97]
Ben Pfaff. 2004. An Introduction to Binary Search Trees and Balanced Trees. Free Software Foundation, Inc., Boston.
[98]
Colin R. Reeves. 1993. Modern Heuristic Techniques for Combinatorial Problems. John Wiley & Sons, New York.
[99]
Colin R. Reeves. 1999. Landscapes, operators and heuristic search. Annals of Operations Research 86 (1999), 473–490.
[100]
César Rego, Dorabela Gamboa, Fred Glover, and Colin Osterman. 2011. Traveling salesman problem heuristics: Leading methods, implementations and latest advances. European Journal of Operational Research 211, 3 (2011), 427–441.
[101]
Mauricio G. C. Resende, Rafael Martí, Michael Gallego, and Abraham Duarte. 2010. GRASP and path relinking for the max-min diversity problem. Computer & Operations Research 37, 3 (2010), 498–508.
[102]
Mauricio G. C. Resende and Celso C. Ribeiro. 2010. Greedy randomized adaptive search procedures: Advances, hybridizations, and applications. In Handbook of Metaheuristics. Springer, New York, 283–319.
[103]
Mauricio G. C. Resende and Celso C. Ribeiro. 2016. Optimization by GRASP. Springer, New York.
[104]
Mauricio G. C. Resende and Celso C. Ribeiro. 2016. Path-relinking. In Optimization by GRASP. Springer, New York, 167–188.
[105]
Sartaj Sahni and Teofilo Gonzales. 1976. P-complete approximation problem. Journal of the ACM 23, 3 (1976), 555–565.
[106]
Danilo Sanches, Darrell Whitley, and Renato Tinós. 2017. Improving an exact solver for the traveling salesman problem using partition crossover. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’17). 337–344.
[107]
Sam L. Savage. 1976. Some theoretical implications of local optimization. Mathematical Programming 10, 1 (1976) 354–366.
[108]
Johannes J. Schneider and Scott Kirkpatrick. 2006. Stochastic Optimization. Springer, New York.
[109]
Claude E. Shammon. 1948. A mathematical theory of communication. Bell System Technical Journal 27 (1948), 379–423 and 623–656.
[110]
Ofer M. Shir. 2012. Niching in evolutionary algorithms. In Handbook of Natural Computing. Springer, Berlin.
[111]
Karthik Sindhya, Kalyanmoy Deb, and Kaisa Miettinen. 2011. Improving convergence of evolutionary multi-objective optimization with local search: A concurrent-hybrid algorithm. Natural Computing 10 (2011), 1407–1430.
[112]
Michael Sipser. 1992. The history and status of the P verses NP question. In Proceedings of the 24th Annual ACM Symposium on Theory of Computing. 603–618.
[113]
Michael Sipser. 2006. Introduction to the Theory of Computation. Thomson Course Technology, Boston.
[114]
Nicolas Sourlas. 1986. Statistical mechanics and the traveling salesman problem. Europhysics Letter 2, 12 (1986), 919–923.
[115]
Bradley S. Stewart, Ching-Fang Liaw, and Chelsea C. White. 1994. A bibliography of heuristic search research through 1992. IEEE Transactions on Systems, Man, and Cybernetic 24, 2 (1994), 268–293.
[116]
El-Ghazali Talbi. 2009. Metaheuristics: From Design to Implementation. Wiley, New York.
[117]
Stelios H. Zanakis, James R. Evans, and Alkis A. Vazacopoulos. 1989. Heuristic methods and applications: A categorized survey. European Journal of Operations Research 43, 1 (1989), 88–110.
[118]
Anatoly Zhigljavaky and Antanas Žilinskas. 2008. Stochastic Global Optimization. Springer, New York.
[119]
Constantin Zopouniclis and Panos M. Pardalos. 2010. Handbook of Multicriteria Analysis (Applied Optimization, 103). Springer, New York.
[120]
Geoffrey Zweig. 1995. An effective tour construction and improvement procedure for the traveling salesman problem. Operations Research 43, 6 (1995), 1049–1057.
Index Terms
- Optimizing with Attractor: A Tutorial
Recommendations
A new approach to the traveling salesman problem
ACM SE '22: Proceedings of the 2022 ACM Southeast ConferenceThe intrinsic difficulty of the Traveling Salesman Problem (TSP) is associated with the combinatorial explosion of potential solutions in the solution space. The significant difficulty with search algorithms for the TSP is how to quickly find the ...
Solving the maximum clique problem by k-opt local search
SAC '04: Proceedings of the 2004 ACM symposium on Applied computingThis paper presents a local search algorithm based on variable depth search, called the k-opt local search, for the maximum clique problem. The k-opt local search performs add and drop moves, each of which can be interpreted as 1-opt move, to search a k-...
Approximate Local Search in Combinatorial Optimization
Local search algorithms for combinatorial optimization problems are generally of pseudopolynomial running time, and polynomial-time algorithms are not often known for finding locally optimal solutions for NP-hard optimization problems. We introduce the ...
Comments
Information & Contributors
Information
Published In
September 2024
980 pages
ISSN:0360-0300
EISSN:1557-7341
DOI:10.1145/3613649
- Editors:
- David Atienza,
- Michela Milano
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 April 2024
Online AM: 15 February 2024
Accepted: 31 January 2024
Revised: 04 October 2023
Received: 30 November 2022
Published in CSUR Volume 56, Issue 9
Check for updates
Author Tags
Qualifiers
- Tutorial
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 219Total Downloads
- Downloads (Last 12 months)219
- Downloads (Last 6 weeks)32
Other Metrics
Citations
Cited By
View all- Wu HWang CLu CZhan T(2024)HCT: a hybrid CNN and transformer network for hyperspectral image super-resolutionMultimedia Systems10.1007/s00530-024-01387-930:4Online publication date: 20-Jun-2024
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in