Pareto Improver: Learning Improvement Heuristics for Multi-Objective Route Planning
Pages 1033 - 1043
Abstract
As a research hotspot across logistics, operations research, and artificial intelligence, route planning has become a key technology for intelligent transportation systems. Recently, data-driven machine learning heuristics, including learning construction methods and learning improvement methods, have achieved remarkable success in solving single-objective route planning problems. However, many practical route planning scenarios must simultaneously consider multiple conflict objectives. For example, modern logistics companies often need to simultaneously minimize time budget, transportation cost, and vehicle pollution. Several learning construction methods are proposed for solving classical multi-objective route planning (MORP) problems, yet no learning improvement heuristics have been developed so far, even though they are acknowledged to be more efficient in narrowing the optimality gap. To fill this gap, this paper proposes a learning improvement MORP method, Pareto Improver (PI). PI employs a population-based mechanism to approximate the Pareto front with a single deep reinforcement learning model. The experimental results on various MORP problems show that PI can significantly outperform other state-of-the-art methods.
References
[1]
P. Li, X. Wang, H. Gao, X. Xu, M. Iqbal, and K. Dahal, “A dynamic and scalable user-centric route planning algorithm based on polychromatic sets theory,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 3, pp. 2762–2772, Mar. 2022.
[2]
J. Li, L. Xin, Z. Cao, A. Lim, W. Song, and J. Zhang, “Heterogeneous attentions for solving pickup and delivery problem via deep reinforcement learning,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 3, pp. 2306–2315, Mar. 2022.
[3]
Y. Lv, Y. Duan, W. Kang, Z. Li, and F.-Y. Wang, “Traffic flow prediction with big data: A deep learning approach,” IEEE Trans. Intell. Transp. Syst., vol. 16, no. 2, pp. 865–873, Apr. 2015.
[4]
Z. Wanget al., “Multiobjective optimization-aided decision-making system for large-scale manufacturing planning,” IEEE Trans. Cybern., vol. 52, no. 8, pp. 8326–8339, Aug. 2022.
[5]
Y. Wu, W. Song, Z. Cao, J. Zhang, and A. Lim, “Learning improvement heuristics for solving routing problems,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 9, pp. 5057–5069, Sep. 2022.
[6]
T. L. Morin and R. E. Marsten, “Branch-and-bound strategies for dynamic programming,” Oper. Res., vol. 24, no. 4, pp. 611–627, Aug. 1976.
[7]
M. Fischetti, A. Lodi, and P. Toth, “Solving real-world ATSP instances by branch-and-cut,” in Combinatorial Optimization—Eureka, You Shrink! Cham, Switzerland: Springer, 2003, pp. 64–77.
[8]
G. Kim, Y. S. Ong, T. Cheong, and P. S. Tan, “Solving the dynamic vehicle routing problem under traffic congestion,” IEEE Trans. Intell. Transp. Syst., vol. 17, no. 8, pp. 2367–2380, Aug. 2016.
[9]
Y. Bengio, A. Lodi, and A. Prouvost, “Machine learning for combinatorial optimization: A methodological tour d’horizon,” Eur. J. Oper. Res., vol. 290, no. 2, pp. 405–421, Apr. 2021.
[10]
W. Kool, H. van Hoof, and M. Welling, “Attention, learn to solve routing problems!,” in Proc. Int. Conf. Learn. Represent., 2018, pp. 1–25.
[11]
Y.-D. Kwon, J. Choo, B. Kim, I. Yoon, Y. Gwon, and S. Min, “POMO: Policy optimization with multiple optima for reinforcement learning,” in Proc. Adv. Neural Inf. Process. Syst., vol. 33, 2020, pp. 21188–21198.
[12]
J. J. Q. Yu, W. Yu, and J. Gu, “Online vehicle routing with neural combinatorial optimization and deep reinforcement learning,” IEEE Trans. Intell. Transp. Syst., vol. 20, no. 10, pp. 3806–3817, Oct. 2019.
[13]
J. Shi, Y. Gao, W. Wang, N. Yu, and P. A. Ioannou, “Operating electric vehicle fleet for ride-hailing services with reinforcement learning,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 11, pp. 4822–4834, Nov. 2019.
[14]
J. Zhao, M. Mao, X. Zhao, and J. Zou, “A hybrid of deep reinforcement learning and local search for the vehicle routing problems,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 11, pp. 7208–7218, Nov. 2020.
[15]
R. Zhanget al., “Learning to solve multiple-TSP with time window and rejections via deep reinforcement learning,” IEEE Trans. Intell. Transp. Syst., vol. 24, no. 1, pp. 1325–1336, Jan. 2022.
[16]
C. Ghorai, S. Shakhari, and I. Banerjee, “A SPEA-based multimetric routing protocol for intelligent transportation systems,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 11, pp. 6737–6747, Nov. 2020.
[17]
K. Li, T. Zhang, and R. Wang, “Deep reinforcement learning for multiobjective optimization,” IEEE Trans. Cybern., vol. 51, no. 6, pp. 3103–3114, Jun. 2020.
[18]
Z. Zhang, Z. Wu, H. Zhang, and J. Wang, “Meta-learning-based deep reinforcement learning for multiobjective optimization problems,” IEEE Trans. Neural Netw. Learn. Syst., early access, Feb. 16, 2022. 10.1109/TNNLS.2022.3148435.
[19]
H. Wu, J. Wang, and Z. Zhang, “MODRL/D-AM: Multiobjective deep reinforcement learning algorithm using decomposition and attention model for multiobjective optimization,” in Proc. Int. Symp. Intell. Comput. Appl. Cham, Switzerland: Springer, 2019, pp. 575–589.
[20]
Y. Shaoet al., “Multi-objective neural evolutionary algorithm for combinatorial optimization problems,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 4, pp. 2133–2143, Apr. 2021.
[21]
T. Lust and J. Teghem, “The multiobjective traveling salesman problem: A survey and a new approach,” in Proc. Adv. Multi-Objective Nature Inspired Comput. Cham, Switzerland: Springer, 2010, pp. 119–141.
[22]
P. Lacomme, C. Prins, and M. Sevaux, “A genetic algorithm for a bi-objective capacitated arc routing problem,” Comput. Oper. Res., vol. 33, no. 12, pp. 3473–3493, Dec. 2006.
[23]
M. Ehrgott and X. Gandibleux, “A survey and annotated bibliography of multiobjective combinatorial optimization,” OR Spectr., vol. 22, no. 4, pp. 425–460, Nov. 2000.
[24]
H.-L. Liu, F. Gu, and Q. Zhang, “Decomposition of a multiobjective optimization problem into a number of simple multiobjective subproblems,” IEEE Trans. Evol. Comput., vol. 18, no. 3, pp. 450–455, Jun. 2013.
[25]
L. Paquete, M. Chiarandini, and T. Stützle, “Pareto local optimum sets in the biobjective traveling salesman problem: An experimental study,” in Metaheuristics for Multiobjective Optimisation. Cham, Switzerland: Springer, 2004, pp. 177–199.
[26]
L. Paquete and T. Stützle, “A two-phase local search for the biobjective traveling salesman problem,” in Proc. Int. Conf. Evol. Multi-Criterion Optim. Cham, Switzerland: Springer, 2003, pp. 479–493.
[27]
K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist multiobjective genetic algorithm: NSGA-II,” IEEE Trans. Evol. Comput., vol. 6, no. 2, pp. 182–197, Apr. 2002.
[28]
Q. Zhang and H. Li, “MOEA/D: A multiobjective evolutionary algorithm based on decomposition,” IEEE Trans. Evol. Comput., vol. 11, no. 6, pp. 712–731, Dec. 2007.
[29]
A. Jaszkiewicz, “On the performance of multiple-objective genetic local search on the 0/1 knapsack problem—A comparative experiment,” IEEE Trans. Evol. Comput., vol. 6, no. 4, pp. 402–412, Aug. 2002.
[30]
R. Kumar and P. Singh, “Pareto evolutionary algorithm hybridized with local search for biobjective TSP,” in Hybrid Evolutionary Algorithms. Cham, Switzerland: Springer, 2007, pp. 361–398.
[31]
A. Jaszkiewicz and P. Zielniewicz, “Pareto memetic algorithm with path relinking for bi-objective traveling salesperson problem,” Eur. J. Oper. Res., vol. 193, no. 3, pp. 885–890, Mar. 2009.
[32]
V. Konda and J. Tsitsiklis, “Actor-critic algorithms,” in Proc. Adv. Neural Inf. Process. Syst., vol. 12, 1999, pp. 1–7.
[33]
A. Vaswaniet al., “Attention is all you need,” in Proc. Adv. Neural Inf. Process. Syst., vol. 30, 2017, pp. 1–11.
[34]
A. Nichol, J. Achiam, and J. Schulman, “On first-order meta-learning algorithms,” 2018, arXiv:1803.02999.
[35]
V. Mnihet al., “Asynchronous methods for deep reinforcement learning,” in Proc. Int. Conf. Mach. Learn., 2016, pp. 1928–1937.
[36]
J. Castro-Gutierrez, D. Landa-Silva, and J. Moreno Pérez, “Nature of real-world multi-objective vehicle routing with evolutionary algorithms,” in Proc. IEEE Int. Conf. Syst., Man, Cybern., Oct. 2011, pp. 257–264.
[37]
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2014, arXiv:1412.6980.
[38]
X. Caiet al., “The collaborative local search based on dynamic-constrained decomposition with grids for combinatorial multiobjective optimization,” IEEE Trans. Cybern., vol. 51, no. 5, pp. 2639–2650, May 2019.
[39]
Z. Wang, Q. Zhang, A. Zhou, M. Gong, and L. Jiao, “Adaptive replacement strategies for MOEA/D,” IEEE Trans. Cybern., vol. 46, no. 2, pp. 474–486, Feb. 2015.
[40]
C. Audet, J. Bigeon, D. Cartier, S. Le Digabel, and L. Salomon, “Performance indicators in multiobjective optimization,” Eur. J. Oper. Res., vol. 292, no. 2, pp. 397–422, Jul. 2021.
[41]
M. P. Hansen and A. Jaszkiewicz, Evaluating the Quality of Approximations to the Non-Dominated Set. Princeton, NJ, USA: Citeseer, 1994.
[42]
Z. Wang, Q. Zhang, Y.-S. Ong, S. Yao, H. Liu, and J. Luo, “Choose appropriate subproblems for collaborative modeling in expensive multiobjective optimization,” IEEE Trans. Cybern., vol. 53, no. 1, pp. 483–496, Jan. 2021.
Index Terms
- Pareto Improver: Learning Improvement Heuristics for Multi-Objective Route Planning
Index terms have been assigned to the content through auto-classification.
Recommendations
Pareto-based Multi-Objective Machine Learning
HIS '07: Proceedings of the 7th International Conference on Hybrid Intelligent SystemsMachine learning is inherently a multi-objective task. Traditionally, however, either only one of the objectives is adopted as the cost function or multiple objectives are aggregated to a scalar cost function. This can mainly attributed to the fact that ...
Distributional pareto-optimal multi-objective reinforcement learning
NIPS '23: Proceedings of the 37th International Conference on Neural Information Processing SystemsMulti-objective reinforcement learning (MORL) has been proposed to learn control policies over multiple competing objectives with each possible preference over returns. However, current MORL algorithms fail to account for distributional preferences over ...
A numerical method for constructing the Pareto front of multi-objective optimization problems
In this paper, a new numerical method is presented for constructing an approximation of the Pareto front of multi-objective optimization problems. This method is based on the well-known scalarization approach by Pascoletti and Serafini. The proposed ...
Comments
Information & Contributors
Information
Published In
1558-0016 © 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Press
Publication History
Published: 22 September 2023
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 09 Nov 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in