article

Multicore/Multi-GPU Accelerated Simulations of Multiphase Compressible Flows Using Wavelet Adapted Grids

Authors:

Diego Rossinelli,

Babak Hejazialhosseini,

Daniele G. Spampinato, and

Petros KoumoutsakosAuthors Info & Claims

SIAM Journal on Scientific Computing, Volume 33, Issue 2

Pages 512 - 540

https://doi.org/10.1137/100795930

Published: 01 March 2011 Publication History

Abstract

We present a computational method of coupling average interpolating wavelets with high-order finite volume schemes and its implementation on heterogeneous computer architectures for the simulation of multiphase compressible flows. The method is implemented to take advantage of the parallel computing capabilities of emerging heterogeneous multicore/multi-GPU architectures. A highly efficient parallel implementation is achieved by introducing the concept of wavelet blocks, exploiting the task-based parallelism for CPU cores, and by managing asynchronously an array of GPUs by means of OpenCL. We investigate the comparative accuracy of the GPU and CPU based simulations and analyze their discrepancy for two-dimensional simulations of shock-bubble interaction and Richtmeyer-Meshkov instability. The results indicate that the accuracy of the GPU/CPU heterogeneous solver is competitive with the one that uses exclusively the CPU cores. We report the performance improvements by employing up to 12 cores and 6 GPUs compared to the single-core execution. For the simulation of the shock-bubble interaction at Mach 3 with two million grid points, we observe a 100-fold speedup for the heterogeneous part and an overall speedup of 34.

References

[1]

J. M. Alam, N. K.-R. Kevlahan, and O. V. Vasilyev, Simultaneous space-time adaptive wavelet solution of nonlinear parabolic differential equations, J. Computat. Phys., 214 (2006), pp. 829-857.

[2]

D. A. Bader, V. Agarwal, and S. Kang, Computing discrete transforms on the Cell Broadband Engine, Parallel Computing, 35 (2009), pp. 119-137.

[3]

A. Bagabir and D. Drikakis, Mach number effects on shock-bubble interaction, Shock Waves, 11 (2001), pp. 209-218.

[4]

M. Bergdorf and P. Koumoutsakos, A Lagrangian particle-wavelet method, Multiscale Model. Simul., 5 (2006), pp. 980-995.

[5]

M. J. Berger and J. Oliger, Adaptive mesh refinement for hyperbolic partial differential equations, J. Comput. Phys., 53 (1984), pp. 484-512.

[6]

M. Bernaschi, L. Rossi, R. Benzi, M. Sbragaglia, and S. Succi, Graphics processing unit implementation of lattice Boltzmann models for flowing soft systems, Phys. Rev. E, 80 (2009), 066707.

[7]

R. D. Blumofe and C. E. Leiserson, Scheduling multithreaded computations by work stealing, J. ACM, 46 (1999), pp. 720-748.

[8]

T. Brandvik and G. Pullan, Acceleration of a $3$D Euler solver using commodity graphics hardware, in Proceedings of the 46th AIAA Aerospace Sciences Meeting, American Institute of Aeronautics and Astronautics, Reston, VA, 2008, AIAA-2008-607.

[9]

M. Brouillette, The Richtmyer-Meshkov instability, Ann. Rev. Fluid Mech., 34 (2002), pp. 445-468.

[10]

S. Browne, J. Dongarra, N. Garner, G. Ho, and P. Mucci, A portable programming interface for performance evaluation on modern processors, Int. J. High Perform. Comput. Appl., 14 (2000), pp. 189-204.

[11]

I. L. Chern, J. Glimm, O. McBryan, B. Plohr, and S. Yaniv, Front tracking for gas-dynamics, J. Comput. Phys., 62 (1986), pp. 83-110.

[12]

A. Cohen, I. Daubechies, and J. C. Feauveau, Biorthogonal bases of compactly supported wavelets, Comm. Pure Appl. Math., 45 (1992), pp. 485-560.

[13]

M. O. Domingues, S. M. Gomes, O. Roussel, and K. Schneider, An adaptive multiresolution scheme with local time stepping for evolutionary PDEs, J. Comput. Phys., 227 (2008), pp. 3758-3780.

[14]

M. O. Domingues, S. M. Gomes, O. Roussel, and K. Schneider, Space-time adaptive multiresolution methods for hyperbolic conservation laws: Applications to compressible Euler equations, Appl. Numer. Math., 59 (2009), pp. 2303-2321.

[15]

D. L. Donoho, Smooth wavelet decompositions with blocky coefficient kernels, in Recent Advances in Wavelet Analysis, Academic Press, New York, 1993, pp. 259-308.

[16]

B. Einfeldt, On Godunov-type methods for gas dynamics, SIAM J. Numer. Anal., 25 (1988), pp. 294-318.

[17]

E. Elsen, P. LeGresley, and E. Darve, Large calculation of the flow over a hypersonic vehicle using a GPU, J. Comput. Phys., 227 (2008), pp. 10148-10161.

[18]

P. N. Glaskowsky, NVIDIA's Fermi: The First Complete GPU Computing Architecture, Tech. report, NVIDIA, Santa Clara, CA, 2009.

[19]

J. W. Grove, Applications of front tracking to the simulation of shock refractions and unstable mixing, Appl. Numer. Math., 14 (1994), pp. 213-237.

[20]

J. F. Haas and B. Sturtevant, Interaction of weak shock-waves with cylindrical and spherical gas inhomogeneities, J. Fluid Mech., 181 (1987), pp. 41-76.

[21]

T. R. Hagen, K. A. Lie, and J. R. Natvig, Solving the Euler equations on graphics processing units, Computational Science - ICCS 2006, 3994 (2006), pp. 220-227.

[22]

A. Harten, Adaptive multiresolution schemes for shock computations, J. Comput. Phys., 115 (1994), pp. 319-338.

[23]

B. Hejazialhosseini, D. Rossinelli, M. Bergdorf, and P. Koumoutsakos, High order finite volume methods on wavelet-adapted grids with local time-stepping on multicore architectures for the simulation of shock-bubble interactions, J. Comput. Phys., 229 (2010), pp. 8364-8383.

[24]

R. L. Holmes, J. W. Grove, and D. H. Sharp, Numerical investigation of Richtmyer-Meshkov instability using front tracking, J. Fluid Mech., 301 (1995), pp. 51-64.

[25]

M. Hopf and T. Ertl, Hardware accelerated wavelet transformations, in Proceedings of EG/IEEE TCVG Symposium on Visualization, IEEE, Washington, DC, 2000, pp. 93-103.

[26]

X. Y. Hu, B. C. Khoo, N. A. Adams, and F. L. Huang, A conservative interface method for compressible flows, J. Comput. Phys., 219 (2006), pp. 553-578.

[27]

T. Ishihara, T. Gotoh, and Y. Kaneda, Study of high-Reynolds number isotropic turbulence by direct numerical simulation, Ann. Rev. Fluid Mech., 41 (2009), pp. 165-180.

[28]

G. S. Jiang and C. W. Shu, Efficient implementation of weighted ENO schemes, J. Comput. Phys., 126 (1996), pp. 202-228.

[29]

I. C. Kampolis, X. S. Trompoukis, V. G. Asouti, and K. C. Giannakoglou, CFD-based analysis and two-level aerodynamic optimization on graphics processing units, Comput. Methods Appl. Mech. Engrg., 199 (2010), pp. 712-722.

[30]

N. K.-R. Kevlahan and O. V. Vasilyev, An adaptive wavelet collocation method for fluid-structure interaction at high Reynolds numbers, SIAM J. Sci. Comput., 26 (2005), pp. 1894-1915.

[31]

R. LeVeque, Finite Volume Methods for Hyperbolic Problems, Cambridge University Press, Cambridge, UK, 2002.

[32]

J. Liandrat and P. Tchamitchian, Resolution of the $1$D regularized Burgers equation using a spatial wavelet approximation, Tech. report 90-83, 1CASE, NASA Contractor Report 18748880, 1990.

[33]

X. D. Liu, S. Osher, and T. Chan, Weighted essentially nonoscillatory schemes, J. Comput. Phys., 115 (1994), pp. 200-212.

[34]

E. E. Meshkov, Instability of a shock wave accelerated interface between two gases, NASA Tech. Trans., 1970.

[35]

F. Miniati and P. Colella, Block structured adaptive mesh and time refinement for hybrid, hyperbolic plus n-body systems, J. Comput. Phys., 227 (2007), pp. 400-430.

[36]

A. Munshi, The OpenCL specification, version 1.0, Khronos Group Std., Beaverton, OR, 2009.

[37]

Nvidia, NVIDIA CUDA Compute Unified Device Architecture: Programming guide, NVIDIA, Santa Clara, CA, 2007.

[38]

A. Prosperetti and G. Tryggvason, eds., Computational Methods for Multiphase Flow, Cambridge University Press, Cambridge, UK, 2007, Ch. 3.

[39]

L. Qianlong and O. V. Vasilyev, A Brinkman penalization method for compressible flows in complex geometries, J. Comput. Phys., 227 (2007), pp. 946-966.

[40]

J. J. Quirk and S. Karni, On the dynamics of a shock-bubble interaction, J. Fluid Mech., 318 (1996), pp. 129-163.

[41]

R. D. Richtmyer, Taylor instability in shock acceleration of compressible fluids, Commun. Pure Appl. Math., 13 (1960), pp. 297-319.

[42]

D. Rossinelli, M. Bergdorf, G.-H. Cottet, and P. Koumoutsakos, GPU accelerated simulations of bluff body flows using vortex particle methods, J. Comput. Phys., 229 (2010), pp. 3316-3333.

[43]

D. Rossinelli, M. Bergdorf, B. Hejazialhosseini, and P. Koumoutsakos, Wavelet-based adaptive solvers on multi-core architectures for the simulation of complex systems, in Euro-Par '09: Proceedings of the 15th International Euro-Par Conference on Parallel Processing, Springer-Verlag, Berlin, Heidelberg, 2009, pp. 721-734.

[44]

O. Roussel, K. Schneider, A. Tsigulin, and H. Bockhorn, A conservative fully adaptive multiresolution algorithm for parabolic PDEs, J. Comput. Phys., 188 (2003), pp. 493-523.

[45]

R. Saurel and R. Abgrall, A simple method for compressible multifluid flows, SIAM J. Sci. Comput., 21 (1999), pp. 1115-1145.

[46]

C. E. Scheidegger, J. L. D. Comba, R. D. da Cunha, and N. Corporation, Practical CFD simulations on programmable graphics hardware using SMAC, Computer Graphics Forum, 24 (2005), pp. 715-728.

[47]

K. Schneider and O. V. Vasilyev, Wavelet methods in computational fluid dynamics, Ann. Rev. Fluid Mech., 42 (2010), pp. 473-503.

[48]

C. Tenllado, J. Setoain, M. Prieto, L. Pinuel, and F. Tirado, Parallel implementation of the $2$D discrete wavelet transform on graphics processing units: Filter bank versus lifting, IEEE Trans. Parallel Distributed Systems, 19 (2008), pp. 299-310.

[49]

E. F. Toro, Riemann Solvers and Numerical Methods for Fluid Dynamics, Springer-Verlag, Berlin, 1999.

[50]

S. Williams, A. Waterman, and D. Patterson, Roofline: An insightful visual performance model for multicore architectures, Commun. ACM, 52 (2009), pp. 65-76.

[51]

J. H. Williamson, Low-storage Runge-Kutta schemes, J. Comput. Phys., 35 (1980), pp. 48-56.

Cited By

Loffeld JHittinger J(2019)On the arithmetic intensity of high-order finite-volume discretizations for hyperbolic systems of conservation lawsInternational Journal of High Performance Computing Applications10.1177/109434201769187633:1(25-52)Online publication date: 1-Jan-2019
https://dl.acm.org/doi/10.1177/1094342017691876
Margara APezzè MPivkin ISantoro MCarver JCiancarini PHong N(2015)Towards an engineering methodology for multi-model scientific simulationsProceedings of the 2015 International Workshop on Software Engineering for High Performance Computing in Science10.5555/2821003.2821013(51-55)Online publication date: 16-May-2015
https://dl.acm.org/doi/10.5555/2821003.2821013
Nejadmalayeri AVezolainen ABrown-Dymkoski EVasilyev O(2015)Parallel adaptive wavelet collocation method for PDEsJournal of Computational Physics10.1016/j.jcp.2015.05.028298:C(237-253)Online publication date: 1-Oct-2015
https://dl.acm.org/doi/10.1016/j.jcp.2015.05.028
Show More Cited By

Recommendations

GPU accelerated flow solver for direct numerical simulation of turbulent flows

Graphical processing units (GPUs), characterized by significant computing performance, are nowadays very appealing for the solution of computationally demanding tasks in a wide variety of scientific applications. However, to run on GPUs, existing codes ...
Read More
GPU accelerated simulations of bluff body flows using vortex particle methods

We present a GPU accelerated solver for simulations of bluff body flows in 2D using a remeshed vortex particle method and the vorticity formulation of the Brinkman penalization technique to enforce boundary conditions. The efficiency of the method ...
Read More
GPU accelerated simulations of 3D deterministic particle transport using discrete ordinates method

Graphics Processing Unit (GPU), originally developed for real-time, high-definition 3D graphics in computer games, now provides great faculty in solving scientific applications. The basis of particle transport simulation is the time-dependent, multi-...
Read More

Comments

Information & Contributors

Information

Published In

cover image SIAM Journal on Scientific Computing

SIAM Journal on Scientific Computing Volume 33, Issue 2

April 2011

579 pages

ISSN:1064-8275

Issue’s Table of Contents

Publisher

Society for Industrial and Applied Mathematics

United States

Publication History

Published: 01 March 2011

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Loffeld JHittinger J(2019)On the arithmetic intensity of high-order finite-volume discretizations for hyperbolic systems of conservation lawsInternational Journal of High Performance Computing Applications10.1177/109434201769187633:1(25-52)Online publication date: 1-Jan-2019
https://dl.acm.org/doi/10.1177/1094342017691876
Margara APezzè MPivkin ISantoro MCarver JCiancarini PHong N(2015)Towards an engineering methodology for multi-model scientific simulationsProceedings of the 2015 International Workshop on Software Engineering for High Performance Computing in Science10.5555/2821003.2821013(51-55)Online publication date: 16-May-2015
https://dl.acm.org/doi/10.5555/2821003.2821013
Nejadmalayeri AVezolainen ABrown-Dymkoski EVasilyev O(2015)Parallel adaptive wavelet collocation method for PDEsJournal of Computational Physics10.1016/j.jcp.2015.05.028298:C(237-253)Online publication date: 1-Oct-2015
https://dl.acm.org/doi/10.1016/j.jcp.2015.05.028
(2015)MRAG-I2DJournal of Computational Physics10.1016/j.jcp.2015.01.035288:C(1-18)Online publication date: 1-May-2015
https://dl.acm.org/doi/10.1016/j.jcp.2015.01.035
Kalivarapu VWiner E(2015)A study of graphics hardware accelerated particle swarm optimization with digital pheromonesStructural and Multidisciplinary Optimization10.1007/s00158-014-1215-751:6(1281-1304)Online publication date: 1-Jun-2015
https://dl.acm.org/doi/10.1007/s00158-014-1215-7
(2014)Nonlinear preconditioning for efficient and accurate interface capturing in simulation of multicomponent compressible flowsJournal of Computational Physics10.1016/j.jcp.2014.07.034276:C(508-540)Online publication date: 1-Nov-2014
https://dl.acm.org/doi/10.1016/j.jcp.2014.07.034
Rossinelli DHejazialhosseini BHadjidoukas PBekas CCurioni ABertsch AFutral SSchmidt SAdams NKoumoutsakos PGropp WMatsuoka S(2013)11 PFLOP/s simulations of cloud cavitation collapseProceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis10.1145/2503210.2504565(1-13)Online publication date: 17-Nov-2013
https://dl.acm.org/doi/10.1145/2503210.2504565
Hejazialhosseini BRossinelli DConti CKoumoutsakos PHollingsworth J(2012)High throughput software for direct numerical simulations of compressible two-phase flowsProceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis10.5555/2388996.2389018(1-12)Online publication date: 10-Nov-2012
https://dl.acm.org/doi/10.5555/2388996.2389018

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents