Hierarchical Neural Constructive Solver for Real-world TSP Scenarios
Authors: 
Yong Liang Goh, Zhiguang Cao, Yining Ma, Yanfei Dong, Mohammed Haroon Dupty, Wee Sun LeeAuthors Info & Claims
Yong Liang Goh, Zhiguang Cao, Yining Ma, Yanfei Dong, Mohammed Haroon Dupty, Wee Sun LeeAuthors Info & ClaimsPages 884 - 895
Abstract
Existing neural constructive solvers for routing problems have predominantly employed transformer architectures, conceptualizing the route construction as a set-to-sequence learning task. However, their efficacy has primarily been demonstrated on entirely random problem instances that inadequately capture real-world scenarios. In this paper, we introduce realistic Traveling Salesman Problem (TSP) scenarios relevant to industrial settings and derive the following insights: (1) The optimal next node (or city) to visit often lies within proximity to the current node, suggesting the potential benefits of biasing choices based on current locations. (2) Effectively solving the TSP requires robust tracking of unvisited nodes and warrants succinct grouping strategies. Building upon these insights, we propose integrating a learnable choice layer inspired by Hypernetworks to prioritize choices based on the current location, and a learnable approximate clustering algorithm inspired by the Expectation-Maximization algorithm to facilitate grouping the unvisited cities. Together, these two contributions form a hierarchical approach towards solving the realistic TSP by considering both immediate local neighbourhoods and learning an intermediate set of node representations. Our hierarchical approach yields superior performance compared to both classical and recent transformer models, showcasing the efficacy of the key designs.
Supplemental Material
MOV File - Hierarchical Neural Constructive Solver for Real-World TSP Scenarios
Current neural solvers trained on random uniform distributions cannot exploit structure in real-world data. We propose a hierarchical approach to solve such problems by looking at the immediate locality and by learning an intermediate set of representations capable of representing the remaining problem. Such an approach improves the neural solvers; our contributions can also extend current solvers and approaches. Additionally, we show that the intermediate representations grow to surround the points nicely as training progresses. Holistically, our approach is generic and the benefits extend beyond just routing problems.
- Download
- 109.27 MB
References
[1]
[n. d.]. https://www.math.uwaterloo.ca/tsp/world/countries.html
[2]
David Applegate. 2003. Concorde: A code for solving traveling salesman problems. http://www. tsp. gatech. edu/concorde. html (2003).
[3]
Irwan Bello, Hieu Pham, Quoc V Le, Mohammad Norouzi, and Samy Bengio. 2016. Neural combinatorial optimization with reinforcement learning. arXiv preprint arXiv:1611.09940 (2016).
[4]
Marco Caserta and Stefan Voß. 2014. A hybrid algorithm for the DNA sequencing problem. Discrete Applied Mathematics 163 (2014), 87--99.
[5]
Guillaume Chaslot, Sander Bakkes, Istvan Szita, and Pieter Spronck. 2008. Montecarlo tree search: A new framework for game ai. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol. 4. 216--217.
[6]
Jinho Choo, Yeong-Dae Kwon, Jihoon Kim, Jeongwoo Jae, André Hottung, Kevin Tierney, and Youngjune Gwon. 2022. Simulation-guided beam search for neural combinatorial optimization. Advances in Neural Information Processing Systems 35 (2022), 8760--8772.
[7]
Paulo R d O Costa, Jason Rhuggenaath, Yingqian Zhang, and Alp Akcay. 2020. Learning 2-opt heuristics for the traveling salesman problem via deep reinforcement learning. In Asian conference on machine learning. PMLR, 465--480.
[8]
Yanfei Dong, Mohammed Haroon Dupty, Lambert Deng, Zhuanghua Liu, Yong Liang Goh, and Wee Sun Lee. 2024. Differentiable Cluster Graph Neural Network. arXiv:2405.16185 [cs.LG]
[9]
Zhang-Hua Fu, Kai-Bin Qiu, and Hongyuan Zha. 2021. Generalize a small pretrained model to arbitrarily large TSP instances. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35. 7474--7482.
[10]
Chengrui Gao, Haopu Shang, Ke Xue, Dong Li, and Chao Qian. 2023. Towards generalizable neural solvers for vehicle routing problems via ensemble with transferrable local policy. arXiv preprint arXiv:2308.14104 (2023).
[11]
David Ha, Andrew Dai, and Quoc V. Le. 2016. HyperNetworks. arXiv:1609.09106 [cs.LG]
[12]
André Hottung, Yeong-Dae Kwon, and Kevin Tierney. 2021. Efficient active search for combinatorial optimization problems. arXiv preprint arXiv:2106.05126 (2021).
[13]
Yan Jin, Yuandong Ding, Xuanhao Pan, Kun He, Li Zhao, Tao Qin, Lei Song, and Jiang Bian. 2023. Pointerformer: Deep Reinforced Multi-Pointer Transformer for the Traveling Salesman Problem. arXiv preprint arXiv:2304.09407 (2023).
[14]
Chaitanya K Joshi, Thomas Laurent, and Xavier Bresson. 2019. An efficient graph convolutional network technique for the travelling salesman problem. arXiv preprint arXiv:1906.01227 (2019).
[15]
Minsu Kim, Junyoung Park, and Jinkyoo Park. 2022. Sym-nco: Leveraging symmetricity for neural combinatorial optimization. Advances in Neural Information Processing Systems 35 (2022), 1936--1949.
[16]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[17]
Scott Kirkpatrick and Gérard Toulouse. 1985. Configuration space analysis of travelling salesman problems. Journal de Physique 46, 8 (1985), 1277--1292.
[18]
Wouter Kool, Herke van Hoof, Joaquim Gromicho, and Max Welling. 2022. Deep policy dynamic programming for vehicle routing problems. In International conference on integration of constraint programming, artificial intelligence, and operations research. Springer, 190--213.
[19]
Wouter Kool, Herke Van Hoof, and Max Welling. 2018. Attention, learn to solve routing problems! arXiv preprint arXiv:1803.08475 (2018).
[20]
Ratnesh Kumar and Zhonghui Luo. 2003. Optimizing the operation sequence of a chip placement machine using TSP model. IEEE Transactions on Electronics Packaging Manufacturing 26, 1 (2003), 14--21.
[21]
Yeong-Dae Kwon, Jinho Choo, Byoungjip Kim, Iljoo Yoon, Youngjune Gwon, and Seungjai Min. 2020. Pomo: Policy optimization with multiple optima for reinforcement learning. Advances in Neural Information Processing Systems 33 (2020), 21188--21198.
[22]
Yining Ma, Zhiguang Cao, and Yeow Meng Chee. 2023. Learning to Search Feasible and Infeasible Regions of Routing Problems with Flexible Neural k-Opt. In Thirty-seventh Conference on Neural Information Processing Systems.
[23]
Yining Ma, Jingwen Li, Zhiguang Cao, Wen Song, Hongliang Guo, Yuejiao Gong, and Yeow Meng Chee. 2022. Efficient Neural Neighborhood Search for Pickup and Delivery Problems. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence. 4776--4784.
[24]
Yining Ma, Jingwen Li, Zhiguang Cao,Wen Song, Le Zhang, Zhenghua Chen, and Jing Tang. 2021. Learning to iteratively solve routing problems with dual-aspect collaborative transformer. Advances in Neural Information Processing Systems 34 (2021), 11096--11107.
[25]
Todd K Moon. 1996. The expectation-maximization algorithm. IEEE Signal processing magazine 13, 6 (1996), 47--60.
[26]
Mohammadreza Nazari, Afshin Oroojlooy, Lawrence Snyder, and Martin Takác. 2018. Reinforcement learning for solving the vehicle routing problem. Advances in neural information processing systems 31 (2018).
[27]
Gerhard Reinelt. [n. d.]. http://comopt.ifi.uni-heidelberg.de/software/TSPLIB95/
[28]
Zhiqing Sun and Yiming Yang. 2023. Difusco: Graph-based diffusion solvers for combinatorial optimization. arXiv preprint arXiv:2302.08224 (2023).
[29]
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. Advances in neural information processing systems 27 (2014).
[30]
Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2019. Transformer dissection: a unified understanding of transformer's attention via the lens of kernel. arXiv preprint arXiv:1908.11775 (2019).
[31]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
[32]
Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. Advances in neural information processing systems 28 (2015).
[33]
Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning 8 (1992), 229--256.
[34]
YaoxinWu,Wen Song, Zhiguang Cao, Jie Zhang, and Andrew Lim. 2021. Learning improvement heuristics for solving routing problems. IEEE transactions on neural networks and learning systems 33, 9 (2021), 5057--5069.
[35]
Haoran Ye, Jiarui Wang, Helan Liang, Zhiguang Cao, Yong Li, and Fanzhang Li. 2024. Glop: Learning global partition and local construction for solving largescale routing problems in real-time. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 20284--20292.
[36]
Cong Zhang, Yaoxin Wu, Yining Ma,Wen Song, Zhang Le, Zhiguang Cao, and Jie Zhang. 2023. A review on learning to solve combinatorial optimisation problems in manufacturing. IET Collaborative Intelligent Manufacturing 5, 1 (2023), e12072.
Index Terms
- Hierarchical Neural Constructive Solver for Real-world TSP Scenarios
Index terms have been assigned to the content through auto-classification.
Recommendations
Hierarchical Solving Method for Large Scale TSP Problems
Advances in Neural Networks – ISNN 2014AbstractThis paper presents a hierarchical algorithm for solving large-scale traveling salesman problem (TSP), the algorithm first uses clustering algorithms to large-scale TSP problem into a number of small-scale collections of cities, and then put this ...
A backbone based TSP heuristic for large instances
We introduce a reduction technique for large instances of the traveling salesman problem (TSP). This approach is based on the observation that tours with good quality are likely to share many edges. We exploit this observation by neglecting the less ...
Reducing Path TSP to TSP
We present a black-box reduction from the path version of the traveling salesman problem (Path TSP) to the classical tour version (TSP). More precisely, given an $\alpha$-approximation algorithm for TSP, then, for any $\epsilon >0$, we obtain an $(\alpha+\...
Comments
Information & Contributors
Information
Published In
August 2024
6901 pages
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 August 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
KDD '24: The 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
August 25 - 29, 2024
Barcelona, Spain
Acceptance Rates
Overall Acceptance Rate 1,133 of 8,635 submissions, 13%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 257Total Downloads
- Downloads (Last 12 months)257
- Downloads (Last 6 weeks)65
Reflects downloads up to 25 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in