Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Time-parallel simulation of the decay of homogeneous turbulence using Parareal with spatial coarsening

  • Special Issue Parallel-in-Time Methods
  • Published:
Computing and Visualization in Science

Abstract

Direct Numerical Simulation of turbulent flows is a computationally demanding problem that requires efficient parallel algorithms. We investigate the applicability of the time-parallel Parareal algorithm to an instructional case study related to the simulation of the decay of homogeneous isotropic turbulence in three dimensions. We combine a Parareal variant based on explicit time integrators and spatial coarsening with the space-parallel Hybrid Navier–Stokes solver. We analyse the performance of this space–time parallel solver with respect to speedup and quality of the solution. The results are compared with reference data obtained with a classical explicit integration, using an error analysis which relies on the energetic content of the solution. We show that a single Parareal iteration is able to reproduce with high fidelity the main statistical quantities characterizing the turbulent flow field.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. We define the parallel efficiency as the ratio of the parallel speedup over the number of processes involved.

  2. Since there is no mean flow in this configuration, we cannot use the classical definition of the Reynolds number. We rather use \(Re_\lambda \)(see e.g., [11, 44]), which characterizes the ratio of turbulent eddy viscosity, over the fluid viscosity. This Reynolds number, at least one order of magnitude lower than the classical one, has therefore no relevance regarding laminar to turbulent transition, but instead informs about the range of length scales present in an already turbulent flow (see e.g., [31] for a complete description).

  3. For isotropic turbulence \(\lambda \) is conveniently expressed as \(\lambda = u^{\prime }\sqrt{15 \langle \mu \rangle /(\langle \rho \rangle \epsilon )}\) (see, e.g., [31, Sec. 6.3, p.199]).

  4. available at https://gitlab.com/tlunet/parallel-in-time.

  5. This value of the theoretical ratio of coarse-to-fine execution is explained by the coarsening in space by a factor of 2 in each direction and by the choice of the coarse time step detailed in Sect. 3.4.

  6. We refer the reader to only Sec. 4.3 for numerical experiments using two iterations (\({\widehat{K}}=2\)) of Parareal with spatial coarsening.

  7. In Hybrid the computation of the right-hand side of the Navier–Stokes equations involves 39 gradient evaluations (3 for (14), 18 for (15), 6 for (16), 9 for (18) and 3 for (19), respectively). Since the RK4 time integration method requires 4 evaluations of right-hand sides, the total amount of gradient evaluations for one time step of the coarse solver is then 156. Finally, 60 stencil evaluations per coarse point are required for the interpolation of the 5 different fields.

References

  1. Arteaga, A., Ruprecht, D., Krause, R.: A stencil-based implementation of Parareal in the C++ domain specific embedded language STELLA. Appl. Math. Comp. 267, 727–741 (2015)

    Article  MathSciNet  Google Scholar 

  2. Aubanel, E.: Scheduling of tasks in the Parareal algorithm. Parallel Comput. 37(3), 172–182 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bal, G.: Analysis of the convergence and of the stability of the Parareal algorithm for partial differential equations. In: Kornhuber, R., et al. (eds.) Domain Decomposition Methods in Science and Engineering, Lecture Notes in Computational Science and Engineering, vol. 40, pp. 426–432. Springer, New York (2005)

    Google Scholar 

  4. Bermejo-Moreno, I., Bodart, J., Larsson, J., Barney, B., Nichols, J., Jones, S.: Solving the compressible Navier–Stokes equations on up to 1.97 million cores and 4.1 trillion grid points. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis. ACM, p. 62 (2013)

  5. Berry, L.A., Elwasif, W., Reynolds-Barredo, J., Samaddar, D., Sanchez, R., Newman, D.E.: Event-based Parareal: a data-flow based implementation of Parareal. J. Comput. Phys. 231(17), 5945–5954 (2012)

    Article  Google Scholar 

  6. Celledoni, E., Kvamsdal, T.: Parallelization in time for thermo-viscoplastic problems in extrusion of aluminium. Int. J. Numer. Methods Eng. 79(5), 576–598 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  7. Choi, H., Moin, P.: Effects of the computational time step on numerical solutions of turbulent flow. J. Comput. Phys. 113(1), 1–4 (1994)

    Article  MATH  Google Scholar 

  8. Croce, R., Ruprecht, D., Krause, R.: Parallel-in-space-and-time simulation of the three-dimensional, unsteady Navier–Stokes equations for incompressible flow. In: Bock, H., Hoang, X., Rannacher, R., Schlöder, J. (eds.) Modeling, Simulation and Optimization of Complex Processes-HPSC 2012, pp. 13–23. Springer, New York (2014)

    Google Scholar 

  9. Dai, X., Maday, Y.: Stable Parareal in time method for first-and second-order hyperbolic systems. SIAM J. Sci. Comput. 35(1), A52–A78 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  10. Dongarra, J., Hittinger, J., Bell, J., Chacon, L., Falgout, R., Heroux, M., Hovland, P., Ng, E., Webster, C., Wild, S.: Applied mathematics research for exascale computing. Tech. Rep. LLNL-TR-651000, Lawrence Livermore National Laboratory, USA (2014). https://science.energy.gov/~/media/ascr/pdf/research/ am/docs/EMWGreport.pdf

  11. Donzis, D.A., Yeung, P.K., Sreenivasan, K.R.: Dissipation and enstrophy in isotropic turbulence: resolution effects and scaling in direct numerical simulations. Phys. Fluids 20, 045108 (2008)

    Article  MATH  Google Scholar 

  12. Eghbal, A., Gerber, A.G., Aubanel, E.: Acceleration of unsteady hydrodynamic simulations using the Parareal algorithm. J. Comput. Sci. 19, 57–76 (2017)

    Article  Google Scholar 

  13. Farhat, C., Cortial, J., Dastillung, C., Bavestrello, H.: Time-parallel implicit integrators for the near real-time prediction of linear structural dynamics responses. Int. J. Numer. Methods Eng. 67(5), 697–724 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  14. Fischer, P.F., Hecht, F., Maday, Y.: A Parareal in time semi-implicit approximation of the Navier–Stokes equations. In: Kornhuber, R., et al. (eds.) Domain Decomposition Methods in Science and Engineering. Lecture Notes in Computational Science and Engineering, vol. 40, pp. 433–440. Springer, New York (2005)

    Chapter  Google Scholar 

  15. Gander, M., Petcu, M.: Analysis of a Krylov subspace enhanced Parareal algorithm for linear problem. In: ESAIM: Proceedings, vol. 25, pp. 114–129 (2009)

  16. Gander, M.J.: 50 years of time parallel time integration. In: Carraro, T., Geiger, M., Körkel, S., Rannacher, R. (eds.) Multiple Shooting and Time Domain Decomposition Methods, pp. 69–114. Springer, New York (2015)

    Chapter  Google Scholar 

  17. Gander, M., Hairer, E.: Nonlinear convergence analysis for the Parareal algorithm. In: Widlund, O.B., Keyes, D.E. (eds.) Domain Decomposition Methods in Science and Engineering, Lecture Notes in Computational Science and Engineering, vol. 60, pp. 45–56. Springer, New York (2008)

    Google Scholar 

  18. Gander, M.J., Vandewalle, S.: Analysis of the Parareal time-parallel time-integration method. SIAM J. Sci. Comput. 29(2), 556–578 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  19. Gropp, W.: MPI: The Complete Reference, vol. 2, The MPI-2 Extensions. MIT Press, Cambridge (1998)

    Google Scholar 

  20. Jamme, S.: Étude de l’interaction entre une turbulence homogène isotrope et une onde de choc. Ph.D. thesis, Institut National Polytechnique de Toulouse, France (1998)

  21. Johnsen, E., Larsson, J., Bhagatwala, A.V., Cabot, W.H., Moin, P., Olson, B.J., Rawat, P.S., Shankar, S.K., Sjögreen, B., Yee, H., et al.: Assessment of high-resolution methods for numerical simulations of compressible turbulence with shock waves. J. Comput. Phys. 229(4), 1213–1237 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  22. Kaneda, Y., Ishihara, T., Yokokawa, M., Itakura, K., Uno, A.: Energy dissipation rate and energy spectrum in high resolution direct numerical simulations of turbulence in a periodic box. Phys. Fluids 15(2), L21–L24 (2003)

    Article  MATH  Google Scholar 

  23. Kreienbuehl, A., Naegel, A., Ruprecht, D., Speck, R., Wittum, G., Krause, R.: Numerical simulation of skin transport using Parareal. Comput. Vis. Sci. 17(2), 99–108 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  24. Larsson, J., Lele, S., Moin, P.: Effect of numerical dissipation on the predicted spectra for compressible turbulence. CTR Annual Research Briefs pp. 47–57 (2007)

  25. Larsson, J., Bermejo-Moreno, I., Lele, S.K.: Reynolds-and Mach-number effects in canonical shock-turbulence interaction. J. Fluid Mech. 717, 293 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  26. Lekien, F., Marsden, J.: Tricubic interpolation in three dimensions. Int. J. Numer. Methods Eng. 63(3), 455–471 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  27. Lele, S.: Compact finite difference schemes with spectral-like resolution. J. Comput. Phys. 103(1), 16–42 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  28. Lions, J.L., Maday, Y., Turinici, G.: A “Parareal” in time discretization of PDE’s. C. R. Math. Acad. Sci. Paris 332(7), 661–668 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  29. Minion, M.: A hybrid Parareal spectral deferred corrections method. Comm. App. Math. Comp. Sci. 5(2), 265–301 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  30. Mula, O.: Some contributions towards the parallel simulation of time dependent neutron transport and the integration of observed data in real time. Ph.D. thesis, University of Paris VI, France (2014)

  31. Pope, S.B.: Turbulent Flows. Cambridge University Press, Cambridge (2000)

    Book  MATH  Google Scholar 

  32. Reynolds-Barredo, J.M., Newman, D.E., Sanchez, R., Samaddar, D., Berry, L.A., Elwasif, W.R.: Mechanisms for the convergence of time-parallelized, Parareal turbulent plasma simulations. J. Comput. Phys. 231(23), 7851–7867 (2012)

    Article  Google Scholar 

  33. Rogallo, R.S.: Numerical experiments in homogeneous turbulence. Tech. rep., NASA Aimes Research Center, NASA Technical Memorandum 81315 (1981)

  34. Ruprecht, D.: A shared memory implementation of pipelined Parareal. arXiv preprint arXiv:1509.06935 (2015)

  35. Ruprecht, D.: Wave propagation characteristics of Parareal (2017). arXiv:1701.01359 [math.NA]

  36. Ruprecht, D.: Convergence of Parareal with spatial coarsening. PAMM 14(1), 1031–1034 (2014)

    Article  Google Scholar 

  37. Ruprecht, D., Krause, R.: Explicit parallel-in-time integration of a linear acoustic-advection system. Comput. Fluids 59, 72–83 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  38. Russell, F.P., Wilkinson, K.A., Kelly, P.H., Skylaris, C.K.: Optimised three-dimensional Fourier interpolation: an analysis of techniques and application to a linear-scaling density functional theory code. Comput. Phys. Commun. 187, 8–19 (2015)

    Article  Google Scholar 

  39. Samaddar, D., Newman, D.E., Sánchez, R.: Parallelization in time of numerical simulations of fully-developed plasma turbulence using the Parareal algorithm. J. Comput. Phys. 229(18), 6558–6573 (2010)

    Article  MATH  Google Scholar 

  40. Schiesser, W.E.: The Numerical Method of Lines: Integration of Partial Differential Equations. Elsevier, London (2012)

    MATH  Google Scholar 

  41. Staff, G.A., Rønquist, E.M.: Stability of the Parareal algorithm. In: Kornhuber, R., et al. (eds.) Domain Decomposition Methods in Science and Engineering, Lecture Notes in Computational Science and Engineering, vol. 40, pp. 449–456. Springer, New York (2005)

    Chapter  Google Scholar 

  42. Steiner, J., Ruprecht, D., Speck, R., Krause, R.: Convergence of Parareal for the Navier–Stokes equations depending on the Reynolds number. In: Abdulle, A., Deparis, S., Kressner, D., Nobile, E., Picasso, M. (eds.) Numerical Mathematics and Advanced Applications—ENUMATH 2013, vol. 103, pp. 195–202. Springer, New York (2015)

    Google Scholar 

  43. Trottenberg, U., Oosterlee, C.W., Schüller, A.: Multigrid. Academic, New York (2001)

    MATH  Google Scholar 

  44. Vincent, A., Meneguzzi, M.: The spatial structure and statistical properties of homogeneous turbulence. J. Fluid Mech. 225, 1–20 (1991)

    Article  MATH  Google Scholar 

  45. Wang, Q., Gomez, S., Blonigan, P., Gregory, A., Qian, E.: Towards scalable parallel-in-time turbulent flow simulations. Phys. Fluids 25(11), 10818 (2013)

    Google Scholar 

Download references

Acknowledgements

The authors thank the two reviewers for their comments. The authors would like to acknowledge CALMIP for the dotation of computing hours on the EOS computer. This work was granted access to the HPC resources of CALMIP under allocation P1425 and P17005. Thibaut Lunet would like to thank Daniel Ruprecht, Michael Minion, Debasmita Samaddar, Jacob B. Schroder, Martin J. Gander, Robert Speck, Andreas Schmidt, Felix Kwok, Beth Wingate and Raymond J. Spiteri for the interesting discussions at the 5th Workshop on Parallel-in-time integration at the Banff International Research Station (BIRS), and the Jülich Supercomputing Center for granting a Student Travel Award for attending this conference.

Author information

Authors and Affiliations

  1. ISAE-SUPAERO, 10 avenue Edouard Belin, BP 54032, 31055 , Toulouse Cedex 4, France

    Thibaut Lunet, Julien Bodart & Xavier Vasseur

  2. CERFACS, 42 avenue Gaspard Coriolis, 31057 , Toulouse Cedex 1, France

    Thibaut Lunet

  3. INPT-IRIT, University of Toulouse and ENSEEIHT, 2 rue Camichel, BP 7122, 31071, Toulouse Cedex 7, France

    Serge Gratton

Authors
  1. Thibaut Lunet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Julien Bodart
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Serge Gratton
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xavier Vasseur
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thibaut Lunet.

Additional information

This research is partly funded by “Région Occitanie/ Pyrénées-Méditerranée” as part of the “Développement de nouvelles stratégies pour le calcul massivement parallèle à l’échelle exa en mécanique des fluides numérique (DENSE)” project.

A. Computation of the energy spectrum

A. Computation of the energy spectrum

We briefly describe the computation of the energy spectrum. More details can be found in the literature (see, e.g., [20, Ap. B]). We denote by \(\hat{\varvec{u}}(\kappa _x,\kappa _y,\kappa _z)\) the Discrete Fourier Transform (DFT) of \(\varvec{u}(x,y,z)\), using 1 (\(1/N_L^3\)) to normalize the direct (inverse) DFT, respectively. We define \(\kappa _x\), \(\kappa _y\) and \(\kappa _z\) the discrete one-dimensional wavenumbers as:

$$\begin{aligned} \kappa _{[.]}= \left\{ n \frac{2\pi }{L} \text { with }n\in \mathbb {Z} \;| -\frac{N_{L}}{2}+1 \le n \le \frac{N_{L}}{2} \right\} , \end{aligned}$$
(37)

and the associated vector norm:

$$\begin{aligned} \kappa =\left||\varvec{\kappa }\right|| = \sqrt{\kappa _x^2+\kappa _y^2+\kappa _z^2}. \end{aligned}$$
(38)

The 3D Fourier space is discretized into three-dimensional shells of thickness \(d\kappa \) (usually \(\frac{2\pi d\kappa }{L}=1\)). To deduce the energy corresponding to the wavenumber \(\kappa _i\), a bin count is then performed using an integer number of shells:

$$\begin{aligned} \begin{aligned} E(\kappa _i) = \frac{1}{2N_L^6} \underset{ \kappa \in [ \kappa _i - \frac{d\kappa }{2}, \kappa _i + \frac{d\kappa }{2} [ }{\sum } |\hat{\varvec{u}}(\kappa _x,\kappa _y,\kappa _z)|^2. \end{aligned} \end{aligned}$$
(39)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lunet, T., Bodart, J., Gratton, S. et al. Time-parallel simulation of the decay of homogeneous turbulence using Parareal with spatial coarsening. Comput. Visual Sci. 19, 31–44 (2018). https://doi.org/10.1007/s00791-018-0295-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00791-018-0295-0

Keywords

Search

Navigation