Developing and Evaluating In Situ Visualization Algorithms using Containers
Authors: 
Michael Will, Quincy Wofford, 
John Patchett, David Rogers, Jonas Lukasczyk, Christoph GarthAuthors Info & Claims
Michael Will, Quincy Wofford, John Patchett, David Rogers, Jonas Lukasczyk, Christoph GarthAuthors Info & ClaimsPages 6 - 11
Abstract
Fundamental research into in situ visualization and analysis techniques is difficult to access for visualization researchers due to the overwhelming complexity and effort of constructing and managing in situ software stacks that allow reproducible evaluation of novel in situ visualization and analysis techniques. To address this problem, we describe EZ-ISAV, a work-in-progress towards a framework for easy construction of customizable in situ pipelines in container images. Designed for portability and ease of use, these images are intended to serve as proof-of-concept cases for in situ visualization and analysis research. Furthermore, we describe the EZ-ISAV repository, an open repository of turn-key, ready-made container images that can strongly reduce the overhead of developing and evaluating in situ techniques and provide improved reproducibility and portability of in situ visualization research.
References
[1]
2020. The Pantheon Project: Reproducible Workflows for Extreme Scale Science. http://pantheonscience.org/
[2]
2021. Docker Documentation: Rootless Mode. https://docs.docker.com/engine/security/rootless/
[3]
James Ahrens, Sébastien Jourdain, Patrick O’Leary, John Patchett, David H Rogers, and Mark Petersen. 2014. An image-based approach to extreme scale in situ visualization and analysis. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE Press, 424–434.
[4]
Martin Alnæs, Jan Blechta, Johan Hake, August Johansson, Benjamin Kehlet, Anders Logg, Chris Richardson, Johannes Ring, Marie E Rognes, and Garth N Wells. 2015. The FEniCS project version 1.5. Archive of Numerical Software 3, 100 (2015).
[5]
Mark Potsdam Andy Bauer, Andrew Wissinkand Buvana Jayaraman. 2016. ParaView Catalyst Computes Particle Paths In Situ. https://blog.kitware.com/paraview-catalyst-computes-particle-paths-in-situ/
[6]
Utkarsh Ayachit, Andrew Bauer, Berk Geveci, Patrick O’Leary, Kenneth Moreland, Nathan Fabian, and Jeffrey Mauldin. 2015. ParaView Catalyst: Enabling In Situ Data Analysis and Visualization. In Proceedings of the First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (Austin, TX, USA) (ISAV2015). Association for Computing Machinery, New York, NY, USA, 25–29. https://doi.org/10.1145/2828612.2828624
[7]
Gaurav Banga, Peter Druschel, and Jeffrey C Mogul. 1999. Resource containers: A new facility for resource management in server systems. In OSDI, Vol. 99. 45–58.
[8]
Kevin J Bowers, BJ Albright, L Yin, B Bergen, and TJT Kwan. 2008. Ultrahigh performance three-dimensional electromagnetic relativistic kinetic plasma simulation. Physics of Plasmas 15, 5 (2008), 055703.
[9]
Kevin J Bowers, Brian J Albright, Benjamin Bergen, Lin Yin, Kevin J Barker, and Darren J Kerbyson. 2008. 0.374 Pflop/s trillion-particle kinetic modeling of laser plasma interaction on Roadrunner. In SC’08: Proceedings of the 2008 ACM/IEEE conference on Supercomputing. IEEE, 1–11.
[10]
Kevin J Bowers, Brian J Albright, Lin Yin, W Daughton, Vadim Roytershteyn, B Bergen, and TJT Kwan. 2009. Advances in petascale kinetic plasma simulation with VPIC and Roadrunner. In Journal of Physics: Conference Series, Vol. 180. IOP Publishing, 012055.
[11]
Minh Thanh Chung, Nguyen Quang-Hung, Manh-Thin Nguyen, and Nam Thoai. 2016. Using Docker in high performance computing applications. In 2016 IEEE Sixth International Conference on Communications and Electronics (ICCE). 52–57. https://doi.org/10.1109/CCE.2016.7562612
[12]
Paolo Di Tommaso, Maria Chatzou, Evan W Floden, Pablo Prieto Barja, Emilio Palumbo, and Cedric Notredame. 2017. Nextflow enables reproducible computational workflows. Nature biotechnology 35, 4 (2017), 316–319.
[13]
T. Gamblin, M. LeGendre, M. R. Collette, G. L. Lee, A. Moody, B. R. de Supinski, and S. Futral. 2015. The Spack package manager: bringing order to HPC software chaos. In SC15: International Conference for High-Performance Computing, Networking, Storage and Analysis. IEEE Computer Society, Los Alamitos, CA, USA, 1–12. https://doi.org/10.1145/2807591.2807623
[14]
Lisa Gerhardt, Wahid Bhimji, Shane Canon, Markus Fasel, Doug Jacobsen, Mustafa Mustafa, Jeff Porter, and Vakho Tsulaia. 2017. Shifter: Containers for HPC. Journal of Physics: Conference Series 898 (oct 2017), 082021. https://doi.org/10.1088/1742-6596/898/8/082021
[15]
Michael A Heroux, Jonathan Carter, Rajeev Thakur, Lois McInnes, James Ahrens, Todd Munson, J Robert Neely, and Jeffrey S Vetter. 2020. ECP software technology capability assessment report, February 2019. Technical Report. Oak Ridge National Lab.(ORNL), Oak Ridge, TN (United States).
[16]
Michael A Heroux, Jonathan Carter, Rajeev Thakur, Lois McInnes, James Ahrens, J Robert Neely, and Jeffrey S Vetter. 2019. ECP software technology capability assessment report, February 2019. Technical Report. Oak Ridge National Lab.(ORNL), Oak Ridge, TN (United States).
[17]
Michael Kerrisk. 2012. LCE: The failure of operating systems and how we can fix it. https://lwn.net/Articles/524952/
[18]
Gregory M. Kurtzer, Vanessa Sochat, and Michael W. Bauer. 2017. Singularity: Scientific containers for mobility of compute. PLOS ONE 12, 5 (05 2017), 1–20. https://doi.org/10.1371/journal.pone.0177459
[19]
M Larsen, J Ahrens, U Ayachit, E Brugger, H Childs, B Geveci, and C Harrison. 2017. The ALPINE In Situ Infrastructure: Ascending from the Ashes of Strawman. In Proceedings of the In Situ Infrastructures on Enabling Extreme-Scale Analysis and Visualization. 42–46. https://doi.org/10.1145/3144769.3144778
[20]
Samuel Leventhal, Mark Kim, and David Pugmire. 2019. PAVE: An In Situ Framework for Scientific Visualization and Machine Learning Coupling. In 2019 IEEE/ACM 5th International Workshop on Data Analysis and Reduction for Big Scientific Data (DRBSD-5). 8–15. https://doi.org/10.1109/DRBSD-549595.2019.00007
[21]
Shaomeng Li, Nicole Marsaglia, Christoph Garth, Jonathan Woodring, John Clyne, and Hank Childs. 2018. Data Reduction Techniques for Simulation, Visualization and Data Analysis. Computer Graphics Forum 37, 6 (Sept. 2018), 422–447. https://doi.org/10.1111/cgf.13336
[22]
Anders Logg and Garth N. Wells. 2010. DOLFIN: Automated Finite Element Computing. ACM Trans. Math. Softw. 37, 2, Article 20 (April 2010), 28 pages. https://doi.org/10.1145/1731022.1731030
[23]
Andrew Mallinson, David A Beckingsale, WP Gaudin, JA Herdman, JM Levesque, and Stephen A Jarvis. 2013. Cloverleaf: Preparing hydrodynamics codes for exascale. The Cray User Group 2013(2013).
[24]
Dirk Merkel. 2014. Docker: lightweight linux containers for consistent development and deployment. Linux journal 2014, 239 (2014), 2.
[25]
Richard B Neale, Chih-Chieh Chen, Andrew Gettelman, Peter H Lauritzen, Sungsu Park, David L Williamson, Andrew J Conley, Rolando Garcia, Doug Kinnison, Jean-Francois Lamarque, 2010. Description of the NCAR community atmosphere model (CAM 5.0). NCAR Tech. Note NCAR/TN-486+ STR 1, 1 (2010), 1–12.
[26]
Patrick O’Leary, James Ahrens, Sébastien Jourdain, Scott Wittenburg, David H. Rogers, and Mark Petersen. 2016. Cinema image-based in situ analysis and visualization of MPAS-ocean simulations. Parallel Comput. 55(2016), 43–48. https://doi.org/10.1016/j.parco.2015.10.005 Visualization and Data Analytics for Scientific Discovery.
[27]
Patrick Payne and Marc Charest. 2019. A Multi-Material Exact Intersection ALE Hydrodynamics Code for Existing and Emerging Architectures. In APS April Meeting Abstracts, Vol. 2019. K01–043.
[28]
Reid Priedhorsky and Tim Randles. 2017. Charliecloud: Unprivileged Containers for User-Defined Software Stacks in HPC. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (Denver, Colorado) (SC ’17). Association for Computing Machinery, New York, NY, USA, Article 36, 10 pages. https://doi.org/10.1145/3126908.3126925
[29]
David Rogers. [n.d.]. Pantheon/E4S/Ascent Miniapp Workflow. https://github.com/cinemascienceworkflows/2021-04_Miniapp-Ascent Citation is for workflow only. For other citations, see attribution in workflow documentation.
[30]
Julien Tierny, Guillaume Favelier, Joshua A. Levine, Charles Gueunet, and Michael Michaux. 2018. The Topology ToolKit. IEEE Transactions on Visualization and Computer Graphics 24 (2018), 832–842.
Index Terms
- Developing and Evaluating In Situ Visualization Algorithms using Containers
Index terms have been assigned to the content through auto-classification.
Recommendations
Leveraging NVIDIA Omniverse for In Situ Visualization
High Performance ComputingAbstractTypical in situ visualization approaches involve rendering images of the simulation data in step with the simulation itself, using in situ visualization tools such as ParaView Catalyst, VisIt libsim, or SENSEI. For these approaches, one has to ...
Hardware-Agnostic Interactive Exascale In Situ Visualization of Particle-In-Cell Simulations
PASC '23: Proceedings of the Platform for Advanced Scientific Computing ConferenceThe volume of data generated by exascale simulations requires scalable tools for analysis and visualization. Due to the relatively low I/O bandwidth of modern HPC systems, it is crucial to work as close as possible with simulated data via in situ ...
Python-based in situ analysis and visualization
ISAV '18: Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and VisualizationThis work focuses on enabling the use of Python-based methods for the purpose of performing in situ analysis and visualization. This approach facilitates access to and use of a rapidly growing collection of Python-based, third-party libraries for ...
Comments
Information & Contributors
Information
Published In
November 2021
36 pages
ISBN:9781450387156
DOI:10.1145/3490138
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 15 November 2021
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ISAV'21
ISAV'21: In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization
November 15, 2021
MO, St. Louis, USA
Acceptance Rates
Overall Acceptance Rate 23 of 63 submissions, 37%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 111Total Downloads
- Downloads (Last 12 months)40
- Downloads (Last 6 weeks)1
Reflects downloads up to 30 Aug 2024
Other Metrics
Citations
Cited By
View allView Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format