research-article

Load-Aware Dynamic Time Synchronization in Parallel Discrete Event Simulation

Authors:

Abdel-Hameed A. Badawy,

Nandakishore Santhi,

Stephan Eidenbenz,

Dmitry PonomarevAuthors Info & Claims

SIGSIM-PADS '21: Proceedings of the 2021 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation

Pages 95 - 105

https://doi.org/10.1145/3437959.3459249

Published: 01 June 2021 Publication History

Abstract

Traditional Parallel Discrete Event Simulation (PDES) systems employ a monolithic approach for choosing their thread synchronization protocol. They either implement a Time Window-based conservative synchronization or an optimistic event processing capability based on the Time Warp synchronization. In this paper, we show that this binary choice is suboptimal and unnecessary, particularly in the realistic situation where the load distribution across the simulation domain changes over time. We thus propose a new PDES synchronization scheme, called Hybrid PDES, that dynamically switches between conservative and optimistic synchronization protocols based on the simulation run time characteristics.

The primary objective of Hybrid PDES is to exploit the optimistic event processing as long as it is beneficial for the system performance and scalability. We implement Hybrid PDES in Python- and Lua-based Simian PDES engines and demonstrate up to 3X performance improvements on Intel Knights Landing and AMD EPYC processors based on the Phold, La-pdes and PPT-GPU simulation applications.

Supplementary Material

MP4 File (PADS_21_Recording_Ali.mp4)

Load-Aware Dynamic Time Synchronization in Parallel Discrete Event Simulation

Download
37.40 MB

References

[1]

P. Andelfinger, Y. Xu, W. Cai, D. Eckhoff, and A. Knoll. 2018. Fast-Forwarding Agent States to Accelerate Microscopic Traffic Simulations. In Proceedings of the 2018 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation (Rome, Italy). ACM, 113--124.

Digital Library

[2]

Y. Arafa, A. A. Badawy, G. Chennupati, N. Santhi, and S. Eidenbenz. 2019. PPTGPU: Scalable GPU Performance Modeling. IEEE Computer Architecture Letters 18, 1 (2019), 55--58.

[3]

K. Bahulkar, J. Wang, N. Abu-Ghazaleh, and D. Ponomarev. 2012. Partitioning on Dynamic Bahavior for Parallel Discrete Event Simulation. In 26th IEEE/ACM/SCS Workshop on Principles of Advanced and Distributed Simulations (PADS).

Digital Library

[4]

D. Bauer, C. Carothers, and A. Holder. 2009. Scalable Time Warp on Bluegene Supercomputer. In Proc. of the ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS).

Digital Library

[5]

S. Carnà, S. Ferracci, E. De Santis, A. Pellegrini, and F. Quaglia. 2019. Hardware- Assisted Incremental Checkpointing in Speculative Parallel Discrete Event Simulation. In 2019 Winter Simulation Conference (WSC). 2759--2770.

Digital Library

[6]

H. Chen, Y.Yao, andW. Tang. 2015. Can MIC Find Its Place in theWorld of PDES?. In Proceedings of International Symposium on Distributed Simulation and Real Time Systems (DS-RT).

Digital Library

[7]

G. Chennupati, N. Santhi, and S. Eidenbenz. 2019. Scalable Performance Prediction of Codes with Memory Hierarchy and Pipelines. In Proceedings of the 2019 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation (Chicago, IL, USA). 13--24.

Digital Library

[8]

G. Chrysos. 2012. Intel Xeon Phi x100 Family Coprocessor - the Architecture. In Intel white paper.

[9]

NVIDIA Corporation. Jun. 2017. Volta Tesla V100 GPU Architecture Whitepaper. http://images.nvidia.com/content/volta-architecture/pdf/volta-architecturewhitepaper. pdf

[10]

E. Deelman and B. K. Szymanski. 1998. Dynamic load balancing in parallel discrete event simulation for spatially explicit problems. In Proceedings. Twelfth Workshop on Parallel and Distributed Simulation PADS '98. 46--53.

Digital Library

[11]

A. S. Dhodapkar and J. E. Smith. 2003. Comparing program phase detection techniques. In Proceedings. 36th Annual IEEE/ACM International Symposium on Microarchitecture, 2003. MICRO-36. 217--227.

Digital Library

[12]

P. M. Dickens, D. M. Nicol, P. F. Reynolds, and J. M. Duva. 1996. Analysis of Bounded Time Warp and Comparison with YAWNS. ACM Trans. Model. Comput. Simul. 6, 4 (Oct. 1996), 297--320.

Digital Library

[13]

A. Eker, B. Williams, K. Chiu, and D. Ponomarev. 2019. Controlled Asynchronous GVT: Accelerating Parallel Discrete Event Simulation on Many-Core Clusters. In Proceedings of 48th International Conference on Parallel Processing (ICPP 2019). 1--10.

Digital Library

[14]

A. Eker, B. Williams, K. Chiu, and D. Ponomarev. 2020. Demand-Driven PDES: Exploiting Locality in Simulation Models. In Proceedings of the 2020 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation. ACM.

Digital Library

[15]

A. Eker, B. Williams, N. Mishra, D. Thakur, K. Chiu, D. Ponomarev, and N. Abu- Ghazaleh. 2018. Performance Implications of Global Virtual Time Algorithms on a Knights Landing Processor. In 2018 IEEE/ACM 22nd International Symposium on Distributed Simulation and Real Time Applications (DS-RT). 1--10.

Digital Library

[16]

R. Fujimoto. 1989. Time Warp on a Shared Memory Multiprocessor. Transactions of Society for Computer Simulation (July 1989), 211--239.

Digital Library

[17]

R. Fujimoto. 1990. Parallel Discrete Event Simulation. Commun. ACM 33, 10 (Oct. 1990), 30--53.

Digital Library

[18]

R. Fujimoto. 1990. Performance of Time Warp under synthetic workloads. Proceedings of the SCS Multiconference on Distributed Simulation 22, 1 (Jan. 1990), 23--28.

[19]

E. Galli, L. Cuéllar, S. Eidenbenz, M. Ewers, S. Mniszewski, and C. Teuscher. 2009. ActivitySim: Large-Scale Agent-Based Activity Generation for Infrastructure Simulation. In Proceedings of the 2009 Spring Simulation Multiconference. Society for Computer Simulation International, Article 16, 9 pages.

Digital Library

[20]

S. Grauer-Gray, L. Xu, R. Searles, S. Ayalasomayajula, and J. Cavazos. 2012. Auto-tuning a high-level language targeted to GPU codes. In Innovative Parallel Computing (InPar '12). 1--10. https://doi.org/10.1109/InPar.2012.6339595

[21]

T. R. Henderson, S. Roy, S. Floyd, and G. F. Riley. 2006. Ns-3 Project Goals. In Proceeding from the 2006 Workshop on Ns-2: The IP Network Simulator (Pisa, Italy). ACM, Article 13.

Digital Library

[22]

M. Ianni, R. Marotta, D. Cingolani, A. Pellegrini, and F. Quaglia. 2018. The Ultimate Share-Everything PDES System. In 2018 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation. 73--84.

Digital Library

[23]

D. Jagtap, K. Bahulkar, D. Ponomarev, and N. Abu-Ghazaleh. 2012. Characterizing and understanding pdes behavior on tilera architecture. In 2012 ACM/IEEE/SCS 26thWorkshop on Principles of Advanced and Distributed Simulation. IEEE, 53--62.

Digital Library

[24]

D. Jagtap, N.Abu-Ghazaleh, and D.Ponomarev. 2012. Optimization of Parallel Discrete Event Simulator for Multi-core Systems. In International Parallel and Distributed Processing Symposium.

Digital Library

[25]

D. Jefferson, B. Beckman, F. Wieland, L. Blume, M. Di Loreto, P. Hontalas, P. Laroche, K. Sturdevant, J. Tupman, V. Warren, J. Wedel, H. Younger, and S. Bellenot. 1987. Distributed Simulation and the Time Warp Operating System. In Proceedings of the 12??? SIGOPS - Symposium of Operating Systems Principles. 77--93.

Digital Library

[26]

D. R. Jefferson and P. D. Barnes. 2017. Virtual time III: Unification of conservative and optimistic synchronization in parallel discrete event simulation. In 2017 Winter Simulation Conference (WSC). 786--797.

Digital Library

[27]

P. D. Barnes Jr, C. D. Carothers, D. R. Jefferson, and J. M. LaPre. 2013. Warp speed: executing time warp on 1,966,080 cores. In Proceedings of the 2013 ACM SIGSIM conference on Principles of advanced discrete simulation. ACM, 327--336.

Digital Library

[28]

L. Kroc, S. Eidenbenz, and J. P. Smith. 2009. Sessionsim: Activity-based session generation for network simulation. In Proceedings of the 2009 Winter Simulation Conference (WSC). IEEE, 3169--3180.

Digital Library

[29]

L. Li and C. Tropper. 2007. A design-driven partitioning algorithm for distributed verilog simulation. In 21st International Workshop on Principles of Advanced and Distributed Simulation (PADS'07). IEEE, 211--218.

Digital Library

[30]

J. Linden, P. Bauer, S. Engblom, and B. Jonsson. 2019. Exposing Inter-process Information for Efficient PDES of Spatial Stochastic Systems on Multicores. ACM Transactions on Modeling and Computer Simulation 29, 2, 0--25.

Digital Library

[31]

B. D. Lubachevsky. 1989. Efficient Distributed Event-Driven Simulations of Multiple-Loop Networks. Commun. ACM 32, 1 (Jan. 1989), 111--123.

Digital Library

[32]

E. J. Martello, A. Terra, R. Parizotto, and B. Mello. 2020. Closing the Gap Between Lookahead and Checkpointing to Provide Hybrid Synchronization. In Anais do XLVII Seminário Integrado de Software e Hardware (Cuiabá). SBC, Porto Alegre, RS, Brasil, 104--115.

[33]

F. Mattern. 1993. Efficient Algorithms for Distributed Snapshots and Global Virtual Time Approximation. J. Parallel and Distrib. Comput. 18, 4 (Aug. 1993), 423--434.

Digital Library

[34]

Eric Mikida and Laxmikant V Kale. 2018. Adaptive methods for irregular parallel discrete event simulation workloads. In Proceedings of the 2018 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation. ACM.

Digital Library

[35]

E. Park, S. Eidenbenz, N. Santhi, G. Chapuis, and B. Settlemyer. 2015. Parameterized benchmarking of parallel discrete event simulation systems: Communication, computation, and memory. In 2015 Winter Simulation Conference (WSC). 2836-- 2847.

Digital Library

[36]

K. S. Perumalla, A. J. Park, and V. Tipparaju. 2014. Discrete Event Execution with One-Sided and Two-Sided GVT Algorithms on 216,000 Processor Cores. ACM Trans. Model. Comput. Simul. 24, 3, Article 16 (June 2014), 25 pages.

Digital Library

[37]

K. S. Perumalla and S. K. Seal. 2012. Discrete event modeling and massively parallel execution of epidemic outbreak phenomena. SIMULATION 88, 7 (2012), 768--783.

Digital Library

[38]

F. Quaglia, A. Pellegrini, and R. Vitali. 2014. Reshuffling PDES platforms for multi/many-coremachines: A perspective with focus on load sharing. In Modeling and Simulation-Based Systems Engineering Handbook, Daniele Gianni, Andrea D'Ambrogio, and Andreas Tolk (Eds.). CRC Press, 203--232.

[39]

D. M. Rao. 2014. Accelerating Parallel Agent-Based Epidemiological Simulations. In Proceedings of the 2nd ACM SIGSIM Conference on Principles of Advanced Discrete Simulation (Denver, Colorado, USA). ACM, 127--138.

Digital Library

[40]

N. Santhi, S. Eidenbenz, and J. Liu. 2015. The Simian concept: Parallel Discrete Event Simulation with interpreted languages and just-in-time compilation. In 2015 Winter Simulation Conference (WSC). 3013--3024.

Digital Library

[41]

A. Sodani, R. Gramunt, J. Corbal, H. Kim, K. Vinod, S. Chinthamani, S. HUtsell, R. Agarwal, and Y. Liu. 2016. Knights Landing: Second-Generation Intel Xeon Phi Product. In IEEE Micro.

Digital Library

[42]

S. Srinivasan and P. F. Reynolds. 1998. Elastic Time. ACM Trans. Model. Comput. Simul. 8, 2 (April 1998), 103--139.

Digital Library

[43]

S. Thulasidasan, S. Kasiviswanathan, S. Eidenbenz, E. Galli, S. Mniszewski, and P. Romero. 2009. Designing systems for large-scale, discrete-event simulations: Experiences with the FastTrans parallel microsimulator. In 2009 International Conference on High Performance Computing (HiPC). IEEE, 428--437.

[44]

S. Thulasidasan, S. Kasiviswanathan, S. Eidenbenz, and P. Romero. 2010. Explicit spatial scattering for load balancing in conservatively synchronized parallel discrete event simulations. In 2010 IEEE Workshop on Principles of Advanced and Distributed Simulation. IEEE, 1--8.

Digital Library

[45]

J. Wang, D. Jagtap, N. Abu-Ghazaleh, and D. Ponomarev. 2014. Parallel discrete event simulation for multi-core systems: Analysis and optimization. IEEE Transactions on Parallel and Distributed Systems 25, 6 (2014), 1574--1584.

Digital Library

[46]

B. Williams, D. Ponomarev, N. Abu-Ghazaleh, and P. Wilsey. 2017. Performance characterization of parallel discrete event simulation on knights landing processor. In Proceedings of the 2017 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation. ACM, 121--132.

Digital Library

[47]

L. F. Wilson and W. Shen. 1998. Experiments in load migration and dynamic load balancing in speedes. In 1998 Winter Simulation Conference, Vol. 1. IEEE, 483--490.

Digital Library

Cited By

Trabes GWainer GGil-Costa V(2023)A Parallel Algorithm to Accelerate DEVS Simulations in Shared Memory ArchitecturesIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.325608334:5(1609-1620)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1109/TPDS.2023.3256083
Hu HXu ZWang YLiu F(2023)Fast and Scalable Gate-Level Simulation in Massively Parallel Systems2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD)10.1109/ICCAD57390.2023.10323959(1-9)Online publication date: 28-Oct-2023
https://doi.org/10.1109/ICCAD57390.2023.10323959
Eker ATimmerman DWilliams BChiu KPonomarev D(2021)GVT-Guided Demand-Driven Scheduling in Parallel Discrete Event SimulationProceedings of the 50th International Conference on Parallel Processing10.1145/3472456.3472470(1-10)Online publication date: 9-Aug-2021
https://dl.acm.org/doi/10.1145/3472456.3472470

Index Terms

Load-Aware Dynamic Time Synchronization in Parallel Discrete Event Simulation
1. Computing methodologies
  1. Modeling and simulation
    1. Simulation types and techniques
      1. Discrete-event simulation
      2. Massively parallel and high-performance simulations

Recommendations

Virtual Time III, Part 1: Unified Virtual Time Synchronization for Parallel Discrete Event Simulation
Algorithms for synchronization of parallel discrete event simulation have historically been divided between conservative methods that require lookahead but not rollback, and optimistic methods that require rollback but not lookahead. In this paper we ...
Simultaneous events and lookahead in simulation protocols

A discrete event simulation model may contain several events that have the same timestamp, referred to as simultaneous events. In general, the results of a simulation depend on the order in which simultaneous events are executed. Simulation languages and ...
A PDEVS simulator supporting multiple synchronization protocols

With the ever-increasing complexity of simulation models, parallel simulation becomes necessary to perform simulation within reasonable time bounds. The built-in parallelism of Parallel DEVS is often insufficient to tackle this problem on its own. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGSIM-PADS '21: Proceedings of the 2021 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation

May 2021

181 pages

ISBN:9781450382960

DOI:10.1145/3437959

General Chairs:
Saikou Diallo
Old Dominion University, USA
,
Andreas Tolk
The MITRE Corporation, USA
,
Program Chair:
Philippe Giabbanelli
Miami University, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSIM: ACM Special Interest Group on Simulation and Modeling

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 June 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGSIM-PADS '21

Sponsor:

SIGSIM

SIGSIM-PADS '21: SIGSIM Conference on Principles of Advanced Discrete Simulation

May 31 - June 2, 2021

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 398 of 779 submissions, 51%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
137
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)5

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Trabes GWainer GGil-Costa V(2023)A Parallel Algorithm to Accelerate DEVS Simulations in Shared Memory ArchitecturesIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.325608334:5(1609-1620)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1109/TPDS.2023.3256083
Hu HXu ZWang YLiu F(2023)Fast and Scalable Gate-Level Simulation in Massively Parallel Systems2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD)10.1109/ICCAD57390.2023.10323959(1-9)Online publication date: 28-Oct-2023
https://doi.org/10.1109/ICCAD57390.2023.10323959
Eker ATimmerman DWilliams BChiu KPonomarev D(2021)GVT-Guided Demand-Driven Scheduling in Parallel Discrete Event SimulationProceedings of the 50th International Conference on Parallel Processing10.1145/3472456.3472470(1-10)Online publication date: 9-Aug-2021
https://dl.acm.org/doi/10.1145/3472456.3472470

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents