A Simulation-Aided Deep Reinforcement Learning Approach for Optimization of Automated Sorting Center Processes
Authors: 
Deepak Mohapatra, Aritra Pal, Ankush Ojha, Supratim Ghosh, Marichi Agarwal, 
Chayan SarkarAuthors Info & Claims
Deepak Mohapatra, Aritra Pal, Ankush Ojha, Supratim Ghosh, Marichi Agarwal, Chayan SarkarAuthors Info & ClaimsPages 2795 - 2806
Abstract
Operations in a parcel sorting center (SC) are multi-fold which lead to multiple NP-hard optimization problems, namely, parcel-chute assignment, online bin-packing, scheduling, and routing. The advent of multi-agent robotics has accelerated the process of automation in sorting centers which has led to the requirement of sophisticated algorithms to optimize these operations within an SC. To this end, we propose RL --- SORT: a simulation-aided deep reinforcement learning based algorithm which jointly optimizes the parcel-chute assignment and online roller-cage (RC) packing problems. Through experimentation on our simulation framework, we show that RL --- SORT not only outperforms baselines, but also has a low computational burden. Further, it is able to significantly reduce the number of RCs used, thereby, reducing the transportation costs.
References
[1]
Balcou, C., and V. Yadavalli. 2017. "A supply chain management model to optimise the sorting capability of a third party logistics distribution centre". South African Journal of Business Management 48(1):77--76.
[2]
Benzi, F., E. Bassi, F. Marabelli, N. Belloni, and M. Lombardi. 2019. "IIoT-based Motion Control Efficiency in Automated Warehouses". In 2019 AEIT International Annual Conference (AEIT). September 18th--20th, Florence, Italy, 1--6.
[3]
Crainic, T. G., G. Perboli, and R. Tadei. 2008. "Extreme point-based heuristics for three-dimensional bin packing". Informs Journal on computing 20(3):368--384.
[4]
Fedtke, S., and N. Boysen. 2017. "Layout planning of sortation conveyors in parcel distribution centers". Transportation Science 51(1):3--18.
[5]
Ghosh, S., A. Pal, P. Kumar, A. Ojha, A. Paranjape, S. Barat, and H. Khadilkar. 2021. "A simulation driven optimization algorithm for scheduling sorting center operations". In Winter Simulation Conference, edited by S. Kim, B. Feng, K. Smith, S. Masoud, Z. Zheng, C. Szabo, and M. Loper, 1--12. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
[6]
Gupta, A., S. Ghosh, and A. Dhara. 2022. "Deep Reinforcement Learning Algorithm for Fast Solutions to Vehicle Routing Problem with Time-Windows". In 5th ACM Joint International Conference on Data Science & Management of Data (9th ACM IKDD CODS and 27th COMAD), 236--240. Virtual.
[7]
Harrison, R. 2019. "Dynamically integrating manufacturing automation with logistics". In 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation. September 10th--13th, Zaragoza, Spain, 21--22.
[8]
Hubbs, C., C. Li, N. Sahinidis, I. Grossmann, and J. Wassick. 2020. "A deep reinforcement learning approach for chemical production scheduling". Computers and Chemical Engineering 141(1):106982.
[9]
Jarrah, A. I., X. Qi, and J. F. Bard. 2016. "The destination-loader-door assignment problem for automated package sorting centers". Transportation Science 50(4):1314--1336.
[10]
Khir, R., A. Erera, and A. Toriello. 2021. "Two-stage sort planning for express parcel delivery". IISE Transactions 53(12):1353--1368.
[11]
Kim, J.-B., H.-B. Choi, G.-Y. Hwang, K. Kim, Y.-G. Hong, and Y.-H. Han. 2020. "Sortation Control Using Multi-Agent Deep Reinforcement Learning in N-Grid Sortation System". Sensors 20(12):3401.
[12]
McWilliams, D. L., P. M. Stanfield, and C. D. Geiger. 2005. "The parcel hub scheduling problem: A simulation-based solution approach". Computers & Industrial Engineering 49(3):393--412.
[13]
Motor Intelligence 2022, April. "Courier, Express, and Parcel (CEP) Market - Growth, Trends, CoVID-19 Impact, and Forecasts (2022--2027)". Technical report.
[14]
Ojha, A., M. Agarwal, A. Singhal, C. Sarkar, S. Ghosh, and R. Sinha. 2021. "A generalized algorithm and framework for online 3-dimensional bin packing in an automated sorting center". In Proceedings of the 7th IEEE Indian Control Conference. December 20th--22nd, Mumbai, India, 135--140.
[15]
Werners, B., and T. Wülfing. 2010. "Robust optimization of internal transports at a parcel sorting center operated by Deutsche Post World Net". European Journal of Operational Research 201(2):419--426.
[16]
Zhao, H., Q. She, C. Zhu, Y. Yang, and K. Xu. 2021. "Online 3D Bin Packing with Constrained Deep Reinforcement Learning". In Proceedings of the AAAI Conference on Artificial Intelligence, Volume 35(1), 741--749. Virtual.
Recommendations
Learning Global Optimization by Deep Reinforcement Learning
Intelligent SystemsAbstractLearning to Optimize (L2O) is a growing field that employs a variety of machine learning (ML) methods to learn optimization algorithms automatically from data instead of developing hand-engineered algorithms that usually require hyperparameter ...
Deep Reinforcement Learning: From Q-Learning to Deep Q-Learning
Neural Information ProcessingAbstractAs the two hottest branches of machine learning, deep learning and reinforcement learning both play a vital role in the field of artificial intelligence. Combining deep learning with reinforcement learning, deep reinforcement learning is a method ...
Placement Optimization with Deep Reinforcement Learning
ISPD '20: Proceedings of the 2020 International Symposium on Physical DesignPlacement Optimization is an important problem in systems and chip design, which consists of mapping the nodes of a graph onto a limited set of resources to optimize for an objective, subject to constraints. In this paper, we start by motivating ...
Comments
Information & Contributors
Information
Published In
December 2022
3536 pages
Sponsors
In-Cooperation
- IIE: Institute of Industrial Engineers
- INFORMS-SIM: Institute for Operations Research and the Management Sciences: Simulation Society
- SCS: Society for Computer Simulation
Publisher
IEEE Press
Publication History
Published: 02 March 2023
Check for updates
Qualifiers
- Research-article
Conference
WSC '22
Sponsor:
Acceptance Rates
Overall Acceptance Rate 3,413 of 5,075 submissions, 67%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 20Total Downloads
- Downloads (Last 12 months)3
- Downloads (Last 6 weeks)0
Reflects downloads up to 18 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in