Calibrating Simulation Models with Sparse Data: Counterfeit Supply Chains during Covid-19
Authors: 
Isabelle M. van Schilt, Jan H. Kwakkel, Alexander Verbraeck, Jelte P. MenseAuthors Info & Claims
Isabelle M. van Schilt, Jan H. Kwakkel, Alexander Verbraeck, Jelte P. MenseAuthors Info & ClaimsPages 496 - 507
Abstract
COVID-19 related crimes like counterfeit Personal Protective Equipment (PPE) involve complex supply chains with partly unobservable behavior and sparse data, making it challenging to construct a reliable simulation model. Model calibration can help with this, as it is the process of tuning and estimating the model parameters with observed data of the system. A subset of model calibration techniques seems to be able to deal with sparse data in other fields: Genetic Algorithms and Bayesian Inference. However, it is unknown how these techniques perform when accurately calibrating simulation models with sparse data. This research analyzes the quality-of-fit of these two model calibration techniques for a counterfeit PPE simulation model given an increasing degree of data sparseness. The results demonstrate that these techniques are suitable for calibrating a linear supply chain model with randomly missing values. Further research should focus on other techniques, larger set of models, and structural uncertainty.
References
[1]
Aggarwal, C. C., A. Hinneburg, and D. A. Keim. 2001. "On the Surprising Behavior of Distance Metrics in High Dimensional Space". In International Conference on Database Theory, edited by J. Van den Bussche and V. Vianu, 420--434. London, UK: Springer.
[2]
Csilléry, K., M. G. Blum, O. E. Gaggiotti, and O. François. 2010. "Approximate Bayesian Computation (ABC) in Practice". Trends in Ecology & Evolution 25(7):410--418.
[3]
de Groot, L., and A. Hübl. 2021. "Developing a Calibrated Discrete Event Simulation Model of Shops of a Dutch Phone and Subscription Retailer During COVID-19 to Evaluate Shift Plans to Reduce Waiting Times". In Proceedings of the 2021 Winter Simulation Conference, edited by S. Kim, B. Feng, K. Smith, S. Masoud, Z. Zheng, C. Szabo, and M. Loper, 1--12. Phoenix, Arizona: Institute of Electrical and Electronics Engineers, Inc.
[4]
De Santis, A., T. Giovannelli, S. Lucidi, M. Messedaglia, and M. Roma. 2022. "A Simulation-Based Optimization Approach for the Calibration of a Discrete Event Simulation Model of an Emergency Department". Annals of Operations Research:1--30.
[5]
Frank, M., C. Laroque, and T. Uhlig. 2013. "Reducing Computation Time in Simulation-Based Optimization of Manufacturing Systems". In Proceedings of the 2013 Winter Simulations Conference, 2710--2721. Washington, District of Columbia: Institute of Electrical and Electronics Engineers, Inc.
[6]
Gelman, A., and D. B. Rubin. 1992. "Inference from Iterative Simulation Using Multiple Sequences". Statistical Science 7(4):457--472.
[7]
Hazen, B. T., C. A. Boone, J. D. Ezell, and L. A. Jones-Farmer. 2014. "Data Quality for Data Science, Predictive Analytics, and Big Data in Supply Chain Management: An Introduction to the Problem and Suggestions for Research and Applications". International Journal of Production Economics 154:72--80.
[8]
Hofmann, M. 2005. "On the Complexity of Parameter Calibration in Simulation Models". The Journal of Defense Modeling and Simulation 2(4):217--226.
[9]
Huang, Y. 2013. Automated Simulation Model Generation. Ph.D. Thesis, Faculty of Technology, Policy and Management, Delft University of Technology, Delft, Netherlands.
[10]
Jacobs, P. H. M. 2005. The DSOL Simulation Suite. Ph.D. Thesis, Faculty of Technology, Policy and Management, Delft University of Technology, Delft, Netherlands. http://resolver.tudelft.nl/uuid:4c5586e2-85a8-4e02-9b50-7c6311ed1278.
[11]
Jalali, H., I. Van Nieuwenhuyse, and V. Picheny. 2017. "Comparison of Kriging-Based Algorithms for Simulation Optimization with Heterogeneous Noise". European Journal of Operational Research 261(1):279--301.
[12]
Khondoker, M., R. Dobson, C. Skirrow, A. Simmons, and D. Stahl. 2016. "A Comparison of Machine Learning Methods for Classification Using Simulation with Multiple Real Data Examples from Mental Health Studies". Statistical Methods in Medical Research 25(5):1804--1823.
[13]
Kuipers, L. 2021. "Increasing Supply Chain Visibility With Limited Data Availability: Data Assimilation In Discrete Event Simulation". Master's thesis, Faculty of Technology, Policy and Management, Delft University of Technology, Delft, Netherlands. https://resolver.tudelft.nl/uuid:5f68b82f-205e-4509-9a64-22082c46065f.
[14]
Liu, Z., D. Rexachs, F. Epelde, and E. Luque. 2017. "A Simulation and Optimization Based Method for Calibrating Agent-Based Emergency Department Models Under Data Scarcity". Computers & Industrial Engineering 103:300--309.
[15]
Malleson, N. 2014. "Calibration of Simulation Models". Encyclopedia of Criminology & Criminal Justice 40:115--118.
[16]
Mirjalili, S. 2019. "Genetic Algorithm". In Evolutionary Algorithms and Neural Networks. Studies in Computational Intelligence, Volume 780, 43--55. Cham: Springer.
[17]
Mirkes, E. M., J. Allohibi, and A. Gorban. 2020. "Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality". Entropy 22(10):1--31.
[18]
Omar, I. A., M. Debe, R. Jayaraman, K. Salah, M. Omar, and J. Arshad. 2022. "Blockchain-Based Supply Chain Traceability for COVID-19 Personal Protective Equipment". Computers & Industrial Engineering 167:107995.
[19]
Ören, T. I. 1981. "Concepts and Criteria to Assess Acceptability of Simulation Studies: A Frame of Reference". Communications of the Association for Computing Machinery 24(4):180--189.
[20]
Park, B., and H. Qi. 2005. "Development and Evaluation of a Procedure for the Calibration of Simulation Models". Transportation Research Record 1934(1):208--217.
[21]
Powell, M. J. 1964. "An Efficient Method for Finding the Minimum of a Function of Several Variables Without Calculating Derivatives". The Computer Journal 7(2):155--162.
[22]
Puchinger, J., and G. R. Raidl. 2005. "Combining Metaheuristics and Exact Algorithms in Combinatorial Optimization: A survey and Classification". In First International Work-Conference on the Interplay Between Natural and Artificial Computation, 41--53. Las Palmas, Spain: Springer.
[23]
Ren, Y., and Y. Wu. 2013. "An Efficient Algorithm for High-Dimensional Function Optimization". Soft Computing 17(6):995--1004.
[24]
Sadegh, M., and J. A. Vrugt. 2014. "Approximate Bayesian Computation Using Markov Chain Monte Carlo Simulation: DREAM (ABC)". Water Resources Research 50(8):6767--6787.
[25]
Schmitt, A., and M. Singh. 2009. "Quantifying Supply Chain Disruption Risk Using Monte Carlo and Discrete-Event Simulation". In Proceedings of the 2009 Winter Simulation Conference, edited by M. Rossetti, R. R. Hill, and B. Johansson, 1237 -- 1248. Austin, Texas: Institute of Electrical and Electronics Engineers, Inc.
[26]
Shannon, R. E. 1998. "Introduction to the Art and Science of Simulation". In Proceedings of the 1998 Winter Simulation Conference, edited by D. Medeiros, E. F. Watson, J. S. Carson, and M. S. Manivannan, 7--14. Washington, District of Columbia: Institute of Electrical and Electronics Engineers, Inc.
[27]
Slowik, A., and H. Kwasnicka. 2020. "Evolutionary Algorithms and Their Applications to Engineering Problems". Neural Computing and Applications 32(16):12363--12379.
[28]
Suárez, J. L., S. García, and F. Herrera. 2021. "A Tutorial on Distance Metric Learning: Mathematical Foundations, Algorithms, Experimental Analysis, Prospects and Challenges". Neurocomputing 425:300--322.
[29]
Vassiliadis, V. S., and R. Conejeros. 2009. "Powell Method". In Encyclopedia of Optimization, edited by C. A. Floudas and P. M. Pardalos, 3012--3013. Boston, MA: Springer.
[30]
Vrugt, J. A. 2016. "Markov Chain Monte Carlo Simulation Using the DREAM Software Package: Theory, Concepts, and MATLAB Implementation". Environmental Modelling & Software 75:273--316.
[31]
Vrugt, J. A., and K. J. Beven. 2018. "Embracing Equifinality with Efficiency: Limits of Acceptability Sampling Using the DREAM (LOA) Algorithm". Journal of Hydrology 559:954--971.
[32]
Whitley, D. 1994. "A Genetic Algorithm Tutorial". Statistics and Computing 4(2):65--85.
[33]
Wigan, M. R. 1972. "The Fitting, Calibration, and Validation of Simulation Models". Simulation 18(5):188--192.
[34]
Xie, X. 2018. Data Assimilation in Discrete Event Simulations. Ph.D. Thesis, Faculty of Technology, Policy and Management, Delft University of Technology, Delft, Netherlands.
[35]
Zhong, J., and W. Cai. 2015. "Differential Evolution with Sensitivity Analysis and the Powell's Method for Crowd Model Calibration". Journal of Computational Science 9:26--32.
Recommendations
A simulation and optimization based method for calibrating agent-based emergency department models under data scarcity
Highlights- A method to calibrate an emergency department model with incomplete data.
- ...
AbstractTo tackle the problem of efficiently managing increasingly complex systems, simulation models have been widely used. This is because simulation is safer, less expensive, and faster than field implementation and experimenting. To ...
Validating and Calibrating Agent-Based Models: A Case Study
In this paper we deal with some validation and calibration experiments on a modified version of the Complex Adaptive Trivial System (CATS) model proposed in Gallegati et al. (2005 Journal of Economic Behavior and Organization, 56 , 489---512). The CATS ...
Comments
Information & Contributors
Information
Published In
December 2022
3536 pages
Sponsors
In-Cooperation
- IIE: Institute of Industrial Engineers
- INFORMS-SIM: Institute for Operations Research and the Management Sciences: Simulation Society
- SCS: Society for Computer Simulation
Publisher
IEEE Press
Publication History
Published: 02 March 2023
Check for updates
Qualifiers
- Research-article
Conference
WSC '22
Sponsor:
Acceptance Rates
Overall Acceptance Rate 3,413 of 5,075 submissions, 67%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 8Total Downloads
- Downloads (Last 12 months)5
- Downloads (Last 6 weeks)0
Reflects downloads up to 11 Aug 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in