Automatic Search Guided Code Optimization Framework for Mixed-Precision Scientific Applications
Authors: 
Jienan Yao, Wei XueAuthors Info & Claims
Jienan Yao, Wei XueAuthors Info & ClaimsPages 399 - 403
Abstract
The rapid development in machine learning (ML) has prompted demand for low-precision arithmetic hardware that can deliver faster computing speed. Weather simulation applications typically exhibit higher sensitivity towards small perturbation on the input data, but the inherent uncertainty paves the way for opportunities in mixed-precision computing (MPC) by trading accuracy for performance. To determine an acceptable precision allocation for variables without degrading the simulation quality, challenges include exploring an exponential search space of mixed-precision configurations. We propose a mixed-precision code tuning framework for automatic search of suitable precision configurations for weather modeling applications with black-box optimization algorithms. These search algorithms demonstrate adequate sample efficiency with favorable optimality in the search space of two weather modeling miniapps, and the search results achieve up to 30% performance gain that stays within the tolerance level, offering a workflow to facilitate the identification of variables sensitive to precision change.
Supplemental Material
MP4 File
Recording of "Automatic Search Guided Code Optimization Framework for Mixed-Precision Scientific Applications" at Workshop on Enabling Predictive Science with Optimization and Uncertainty Quantification in HPC 2023
- Download
- 208.45 MB
References
[1]
H. M. Arnold, I. M. Moroz, and T. N. Palmer. 2013. Stochastic parametrizations and model uncertainty in the Lorenz ’96 system. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 371, 1991 (2013), 20110479. https://doi.org/10.1098/rsta.2011.0479 arXiv:https://royalsocietypublishing.org/doi/pdf/10.1098/rsta.2011.0479
[2]
Qinglei Cao, Sameh Abdulah, Rabab Alomairy, Yu Pei, Pratik Nag, George Bosilca, Jack Dongarra, Marc G. Genton, David E. Keyes, Hatem Ltaief, and Ying Sun. 2022. Reshaping geostatistical modeling and prediction for extreme-scale environmental applications. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (Dallas, Texas) (SC ’22). IEEE Press, Article 2, 12 pages.
[3]
Andrew Dawson and Peter D. Düben. 2017. rpe v5: an emulator for reduced floating-point precision in large numerical simulations. Geoscientific Model Development 10, 6 (2017), 2221–2230. https://doi.org/10.5194/gmd-10-2221-2017
[4]
Nikolaus Hansen, Sibylle D. Müller, and Petros Koumoutsakos. 2003. Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES). Evolutionary Computation 11, 1 (2003), 1–18. https://doi.org/10.1162/106365603321828970
[5]
Tsuyoshi Ichimura, Kohei Fujita, Ryota Kusakabe, Kentaro Koyama, Sota Murakami, Yuma Kikuchi, Takane Hori, Muneo Hori, Hikaru Inoue, Takafumi Nose, Takahiro Kawashima, and Maddegedara Lalith. 2022. Extreme scale earthquake simulation with uncertainty quantification. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (Dallas, Texas) (SC ’22). IEEE Press, Article 4, 11 pages.
[6]
Milan Klöwer, Miha Razinger, Juan J. Dominguez, Peter D. Düben, and Tim N. Palmer. 2021. Compressing atmospheric data into its real information content. Nature Computational Science 1, 11 (2021), 713–724.
[7]
Pradeep V. Kotipalli, Ranvijay Singh, Paul Wood, Ignacio Laguna, and Saurabh Bagchi. 2019. AMPT-GA: Automatic Mixed Precision Floating Point Tuning for GPU Applications. In Proceedings of the ACM International Conference on Supercomputing (Phoenix, Arizona) (ICS ’19). Association for Computing Machinery, New York, NY, USA, 160–170. https://doi.org/10.1145/3330345.3330360
[8]
Fred Kucharski, Franco Molteni, Martin P. King, Riccardo Farneti, In-Sik Kang, and Laura Feudale. 2013. On the Need of Intermediate Complexity General Circulation Models: A “SPEEDY” Example. Bulletin of the American Meteorological Society 94, 1 (2013), 25 – 30. https://doi.org/10.1175/BAMS-D-11-00238.1
[9]
Ignacio Laguna and Ganesh Gopalakrishnan. 2022. Finding Inputs that Trigger Floating-Point Exceptions in GPUs via Bayesian Optimization. In SC22: International Conference for High Performance Computing, Networking, Storage and Analysis. 1–14. https://doi.org/10.1109/SC41404.2022.00038
[10]
Ignacio Laguna, Paul C. Wood, Ranvijay Singh, and Saurabh Bagchi. 2019. Gpumixer: Performance-driven floating-point tuning for gpu scientific applications. In ISC High Performance 2019, Frankfurt/Main, Germany, May 17, 2019, Proceedings 34. Springer, 227–246.
[11]
Harshitha Menon, Michael O. Lam, Daniel Osei-Kuffuor, Markus Schordan, Scott Lloyd, Kathryn Mohror, and Jeffrey Hittinger. 2018. ADAPT: Algorithmic Differentiation Applied to Floating-Point Precision Tuning. In SC18: International Conference for High Performance Computing, Networking, Storage and Analysis. 614–626. https://doi.org/10.1109/SC.2018.00051
[12]
Franco Molteni. 2003. Atmospheric simulations using a GCM with simplified physical parametrizations. I: model climatology and variability in multi-decadal experiments. Climate Dynamics 20 (2003), 175–191.
[13]
Matthew R Norman and USDOE. 2020. miniWeather. https://doi.org/10.11578/dc.20201001.88
[14]
NVIDIA. 2017. NVIDIA Tesla V100 GPU Architecture. https://images.nvidia.com/content/volta-architecture/pdf/volta-architecture-whitepaper.pdf.
[15]
E. Adam Paxton, Matthew Chantry, Milan Klöwer, Leo Saffin, and Tim Palmer. 2022. Climate Modeling in Low Precision: Effects of Both Deterministic and Stochastic Rounding. Journal of Climate 35, 4 (2022), 1215 – 1229. https://doi.org/10.1175/JCLI-D-21-0343.1
[16]
Cindy Rubio-González, Cuong Nguyen, Hong Diep Nguyen, James Demmel, William Kahan, Koushik Sen, David H. Bailey, Costin Iancu, and David Hough. 2013. Precimonious: Tuning assistant for floating-point precision. In SC ’13: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis. 1–12. https://doi.org/10.1145/2503210.2503296
[17]
Oriol Tintó Prims, Mario C. Acosta, Andrew M. Moore, Miguel Castrillo, Kim Serradell, Ana Cortés, and Francisco J. Doblas-Reyes. 2019. How to use mixed precision in ocean models: exploring a potential reduction of numerical precision in NEMO 4.0 and ROMS 3.6. Geoscientific Model Development 12, 7 (2019), 3135–3148. https://doi.org/10.5194/gmd-12-3135-2019
[18]
Haoyu Xu, Tao Zhang, Yiqi Luo, Xin Huang, and Wei Xue. 2018. Parameter calibration in global soil carbon models using surrogate-based optimization. Geoscientific Model Development 11, 7 (2018), 3027–3044. https://doi.org/10.5194/gmd-11-3027-2018
Index Terms
- Automatic Search Guided Code Optimization Framework for Mixed-Precision Scientific Applications
Recommendations
Mixed-precision explicit stabilized Runge–Kutta methods for single- and multi-scale differential equations
AbstractMixed-precision algorithms combine low- and high-precision computations in order to benefit from the performance gains of reduced-precision without sacrificing accuracy. In this work, we design mixed-precision Runge–Kutta–Chebyshev (...
Highlights- Mixed-precision algorithms can obtain high-accuracy results at a low-precision cost.
Black-box optimization benchmarking of IPOP-saACM-ES and BIPOP-saACM-ES on the BBOB-2012 noiseless testbed
GECCO '12: Proceedings of the 14th annual conference companion on Genetic and evolutionary computationIn this paper, we study the performance of IPOP-saACM-ES and BIPOP-saACM-ES, recently proposed self-adaptive surrogate-assisted Covariance Matrix Adaptation Evolution Strategies. Both algorithms were tested using restarts till a total number of function ...
Bi-population CMA-ES agorithms with surrogate models and line searches
GECCO '13 Companion: Proceedings of the 15th annual conference companion on Genetic and evolutionary computationIn this paper, three extensions of the BI-population Covariance Matrix Adaptation Evolution Strategy with weighted active covariance matrix update (BIPOP-aCMA-ES) are investigated. First, to address expensive optimization, we benchmark a recently ...
Comments
Information & Contributors
Information
Published In
November 2023
2180 pages
ISBN:9798400707858
DOI:10.1145/3624062
Copyright © 2023 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 12 November 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
SC-W 2023
SC-W 2023: Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis
November 12 - 17, 2023
CO, Denver, USA
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 168Total Downloads
- Downloads (Last 12 months)168
- Downloads (Last 6 weeks)31
Reflects downloads up to 09 Nov 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatGet Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in