Entropy-driven Optimal Sub-sampling of Fluid Dynamics for Developing Machine-learned Surrogates
Authors: 
Wesley Brewer, Daniel Martinez, Muralikrishnan Gopalakrishnan Meena, Aditya Kashi, Katarzyna Borowiec, Siyan Liu, Christopher Pilmaier, Greg Burgreen, Shanti BhushanAuthors Info & Claims
Wesley Brewer, Daniel Martinez, Muralikrishnan Gopalakrishnan Meena, Aditya Kashi, Katarzyna Borowiec, Siyan Liu, Christopher Pilmaier, Greg Burgreen, Shanti BhushanAuthors Info & ClaimsPages 73 - 80
Abstract
Optimal sub-sampling of large datasets from fluid dynamics simulations is essential for training reduced-order machine learned models. A method using Shannon entropy was developed to weight flow features according to their level of information content, such that the most informative features can be extracted and used for training a surrogate model. The method is demonstrated in the canonical flow over a cylinder problem simulated with OpenFOAM. Both time-independent predictions and temporal forecasting were investigated as well as two types of prediction targets: local per-grid-point predictions and global per-time-step predictions. When tested on training a surrogate model, results indicate that our entropy-based sampling method typically outperforms random sampling and yields more reproducible results in less iterations. Finally, the method was used to train a surrogate model for modeling turbulence in magnetohydrodynamic flows, which revealed various challenges and opportunities for future research.
References
[1]
Google AI. 2023. Time series forecasting. https://www.tensorflow.org/tutorials/structured_data/time_series
[2]
Prasanna Balaprakash, Michael Salim, Thomas D Uram, Venkat Vishwanath, and Stefan M Wild. 2018. DeepHyper: Asynchronous hyperparameter search for deep neural networks. In 2018 IEEE 25th international conference on high performance computing (HiPC). IEEE, 42–51.
[3]
Gal Berkooz, Philip Holmes, and John L. Lumley. 1993. The Proper Orthogonal Decomposition in the Analysis of Turbulent Flows. Annual Review of Fluid Mechanics 25 (Jan. 1993), 539–575. https://www.annualreviews.org/doi/abs/10.1146/annurev.fl.25.010193.002543
[4]
Shanti Bhushan, Greg W Burgreen, Wesley Brewer, and Ian D Dettwiller. 2021. Development and validation of a machine learned turbulence model. Energies 14, 5 (2021), 1465.
[5]
Mathew Boyer, Wesley Brewer, Jeff Finckenor, Chris Brackbill, Daniel Martinez, and Andrew Wissink. 2023. Development of a Machine-Learned Cruise Guide Indicator for Rotorcraft. In Proceedings of the 79th Annual Forum of the Vertical Flight Society. West Palm Beach, Florida. https://doi.org/10.4050/F-0079-2023-18164
[6]
Wesley Brewer, Daniel Martinez, Mathew Boyer, Dylan Jude, Andy Wissink, Ben Parsons, Junqi Yin, and Valentine Anantharaj. 2021. Production deployment of machine-learned rotorcraft surrogate models on hpc. In 2021 IEEE/ACM Workshop on Machine Learning in High Performance Computing Environments (MLHPC). IEEE, 21–32.
[7]
Yang Chen. 2020. Active learning over dnn: Automated engineering design optimization for fluid dynamics based on self-simulated dataset. arXiv preprint arXiv:2001.08075 (2020).
[8]
Miles MP Couchman, Stephen M de Bruyn Kops, and P Caulfield Colm-cille. 2023. Mixing across stable density interfaces in forced stratified turbulence. Journal of Fluid Mechanics 961 (2023), A20.
[9]
Liang Deng, Jianqiang Chen, Yueqing Wang, Xinhai Chen, Fang Wang, and Jie Liu. 2022. MVU-Net: a multi-view U-Net architecture for weakly supervised vortex detection. Engineering Applications of Computational Fluid Mechanics 16, 1 (2022), 1567–1586.
[10]
Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96). 226–231.
[11]
Paul Fischer, Stefan Kerkemeier, Misun Min, Yu-Hsiang Lan, Malachi Phillips, Thilina Rathnayake, Elia Merzari, Ananias Tomboulides, Ali Karakus, Noel Chalmers, 2022. NekRS, a GPU-accelerated spectral element Navier–Stokes solver. Parallel Comput. 114 (2022), 102982.
[12]
Kai Fukami, Kazuto Hasegawa, Taichi Nakamura, Masaki Morimoto, and Koji Fukagata. 2021. Model order reduction with neural networks: Application to laminar and turbulent flows. SN Computer Science 2 (2021), 1–16.
[13]
Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In International conference on machine learning. 1050–1059.
[14]
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015).
[15]
Geoffrey E Hinton and Ruslan R Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. science 313, 5786 (2006), 504–507.
[16]
Edwin T Jaynes. 1957. Information theory and statistical mechanics. Physical review 106, 4 (1957), 620.
[17]
Michael G Kapteyn, David J Knezevic, DBP Huynh, Minh Tran, and Karen E Willcox. 2022. Data-driven physics-based digital twins via a library of component-based reduced-order models. Internat. J. Numer. Methods Engrg. 123, 13 (2022), 2986–3003.
[18]
Mariia Karabin and Danny Perez. 2020. An entropy-maximization approach to automated training set generation for interatomic potentials. The Journal of Chemical Physics 153, 9 (2020).
[19]
T. Kohonen. 1990. The self-organizing map. Proc. IEEE 78, 9 (1990), 1464–1480.
[20]
Bryan Lim, Sercan Ö Arık, Nicolas Loeff, and Tomas Pfister. 2021. Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting 37, 4 (2021), 1748–1764.
[21]
Bryan Lim and Stefan Zohren. 2021. Time-series forecasting with deep learning: a survey. Philosophical Transactions of the Royal Society A 379, 2194 (2021), 20200209.
[22]
Fan Liu, Wensheng Zhou, Bingxuan Liu, Ke Li, Kai Zhang, Chenming Cao, Guoyu Qin, Chen Cao, and Renfeng Yang. 2022. Flow Field Description and Simplification Based on Principal Component Analysis Downscaling and Clustering Algorithms. Frontiers in Earth Science 9 (2022), 804617.
[23]
Siyan Liu, Pei Zhang, Dan Lu, and Guannan Zhang. 2021. PI3NN: Out-of-distribution-aware prediction intervals from three neural networks. arXiv preprint arXiv:2108.02327 (2021).
[24]
J. MacQueen. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1. 281–297.
[25]
Krithika Manohar, Bingni W. Brunton, J. Nathan Kutz, and Steven L. Brunton. 2018. Data-Driven Sparse Sensor Placement for Reconstruction: Demonstrating the Benefits of Exploiting Known Patterns. IEEE Control Systems Magazine 38, 3 (June 2018), 63–86. https://doi.org/10.1109/MCS.2018.2810460 Conference Name: IEEE Control Systems Magazine.
[26]
Daniel A. Martinez-Gonzalez, Dylan Jude, and Andrew Wissink. 2022. ROAM-ML: A reduced order aerodynamic module augmented with neural network digital surrogates. AIAA SCITECH 2022 Forum (2022).
[27]
David Montes de Oca Zapiain, Mitchell A Wood, Nicholas Lubbers, Carlos Z Pereyra, Aidan P Thompson, and Danny Perez. 2022. Training data selection for accuracy and transferability of interatomic potentials. npj Computational Materials 8, 1 (2022), 189.
[28]
Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever. 2021. Deep double descent: Where bigger models and more data hurt. Journal of Statistical Mechanics: Theory and Experiment 2021, 12 (2021), 124003.
[29]
OpenAI. n.d. ChatGPT: An AI Language Model based on GPT-4 Architecture. https://www.openai.com/. Accessed: [May 3, 2023].
[30]
Tom O’Malley, Elie Bursztein, James Long, François Chollet, Haifeng Jin, Luca Invernizzi, G de Marmiesse, Y Fu, J Podivìn, F Schäfer, 2023. Keras Tuner. 2019. Available online: github. com/keras-team/kerastuner (accessed on 2 April 2022) (2023).
[31]
Eliaquim M Ramos, Gabriella M Darze, Francisco RT do Nascimento, José Luiz H Faccini, and Gilson A Giraldi. 2020. Comparison of dynamic mode decomposition and deep learning techniques for two-phase flows analysis. Flow, Turbulence and Combustion 105 (2020), 1345–1379.
[32]
Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po-Yao Huang, Zhihui Li, Brij B Gupta, Xiaojiang Chen, and Xin Wang. 2021. A survey of deep active learning. ACM computing surveys (CSUR) 54, 9 (2021), 1–40.
[33]
Peter J. Schmid. 2010. Dynamic mode decomposition of numerical and experiemental data. Journal of Fluid Mechanics 656 (2010), 5–28. https://doi.org/10.1017/S0022112010001217
[34]
Burr Settles. 2009. Active learning literature survey. Technical Report. University of Wisconsin, Madison.
[35]
Arpan Sircar, Jin Whan Bae, Ethan Peterson, Jerome Solberg, and V Badalasi. 2022. FERMI: A multi-physics simulation environment for fusion reactor blanket. Technical Report. Lawrence Livermore National Lab.(LLNL), Livermore, CA (United States).
[36]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
[37]
Pantelis R Vlachas, Georgios Arampatzis, Caroline Uhler, and Petros Koumoutsakos. 2022. Multiscale simulations of complex systems by learning their effective dynamics. Nature Machine Intelligence 4, 4 (2022), 359–366.
[38]
Henry G Weller, Gavin Tabor, Hrvoje Jasak, and Christer Fureby. 1998. A tensorial approach to computational continuum mechanics using object-oriented techniques. Computers in physics 12, 6 (1998), 620–631.
[39]
Steven R Young, Derek C Rose, Thomas P Karnowski, Seung-Hwan Lim, and Robert M Patton. 2015. Optimizing deep learning hyper-parameters through an evolutionary algorithm. In Proceedings of the workshop on machine learning in high-performance computing environments. 1–5.
Index Terms
- Entropy-driven Optimal Sub-sampling of Fluid Dynamics for Developing Machine-learned Surrogates
Index terms have been assigned to the content through auto-classification.
Recommendations
Query Size Estimation for Joins Using Systematic Sampling
We propose a new approach to the estimation of query result sizes for join queries. The technique, which we have called “systematic sampling—SYSSMP”, is a novel variant of the sampling-based approach. A key novelty of the systematic sampling is that it ...
Flow Visualization in Computational Fluid Dynamics
Several flow visualization techniques using the Lagran gian approach are proposed for analyzing the numerical solutions of unsteady flow fields computed by the Eu lerian approach. We show how these methods can be used to assess the validity of solutions ...
Comments
Information & Contributors
Information
Published In
November 2023
2180 pages
ISBN:9798400707858
DOI:10.1145/3624062
Copyright © 2023 ACM.
Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 12 November 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Department of Energy
Conference
SC-W 2023
SC-W 2023: Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis
November 12 - 17, 2023
CO, Denver, USA
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 87Total Downloads
- Downloads (Last 12 months)87
- Downloads (Last 6 weeks)6
Reflects downloads up to 04 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format