Abstract
Numerical simulation of three-dimensional incompressible flows at high Reynolds number using the unsteady Navier–Stokes equations is challenging. In order to obtain accurate simulations, very fine meshes are necessary, and such simulations are increasingly important for modern engineering practices, such as understanding the flow behavior around high speed trains, which is the target application of this research. To avoid the time step size constraint imposed by the CFL number and the fine spacial mesh size, we investigate some fully implicit methods, and focus on how to solve the large nonlinear system of equations at each time step on large scale parallel computers. In most of the existing implicit Navier–Stokes solvers, segregated velocity and pressure treatment is employed. In this paper, we focus on the Newton–Krylov–Schwarz method for solving the monolithic nonlinear system arising from the fully coupled finite element discretization of the Navier–Stokes equations on unstructured meshes. In the subdomain, LU or point-block ILU is used as the local solver. We test the algorithm for some three-dimensional complex unsteady flows, including flows passing a high speed train, on a supercomputer with thousands of processors. Numerical experiments show that the algorithm has superlinear scalability with over three thousand processors for problems with tens of millions of unknowns.
References
Alfonsi, G.: Reynolds-averaged Navier-Stokes equations for turbulence modeling. Appl. Mech. Rev. 62, 040802 (2009)
Alfonsi, G.: On direct numerical simulation of turbulent flows. Appl. Mech. Rev. 64, 020802 (2011)
Balay, S., Buschelman, K., Eijkhout, V., Gropp, W.D., Kaushik, D., Knepley, M.G., McInnes, L.C., Smith, B.F., Zhang, H.: PETSc Users Manual. Tech. Rep., Argonne National Laboratory (2012)
Bayraktar, E., Mierka, O., Turek, S.: Benchmark computations of 3D laminar flow around a cylinder with CFX, OpenFOAM and FeatFlow. Int. J. Comput. Sci. Eng. 7, 253–266 (2012)
Bazilevs, Y., Calo, V.M., Hughes, T.J.R., Zhang, Y.: Isogeometric fluid-structure interaction: theory, algorithms, and computations. Comput. Mech. 43, 3–37 (2008)
Cai, X.-C., Gropp, W.D., Keyes, D.E., Melvin, R.G., Young, D.P.: Parallel Newton-Krylov-Schwarz algorithms for the transonic full potential equation. SIAM J. Sci. Comput. 19, 246–265 (1998)
Cai, X.-C., Sarkis, M.: A restricted additive Schwarz preconditioner for general sparse linear systems. SIAM J. Sci. Comput. 21, 792–797 (1999)
Chan, T.F., van der Vorst, H.A.: Approximate and incomplete factorizations. In: Parallel Numerical Algorithms, ICASE/LaRC Interdisciplinary Series in Science and Engineering IV. Centenary Conference, Keyes, D.E., Sameh, A., Venkatakrishnan, V. (eds.) Dordrecht. Kluwer, 167–202 (1997)
Chen, J.H.: Petascale direct numerical simulation of turbulent combustion-fundamental insights towards predictive models. Proc. Combust. Inst. 33, 99–123 (1999)
Eisenstat, S.C., Walker, H.F.: Choosing the forcing terms in an inexact Newton method. SIAM J. Sci. Comput. 17, 16–32 (1996)
Franca, L.P., Frey, S.L.: Stabilized finite element method: II. The incompressible Navier-Stokes equation. Comput. Methods Appl. Mech. Eng. 99, 209–233 (1992)
Friedrich, R., Huttl, T.J., Manhart, M., Wagner, C.: Direct numerical simulation of incompressible turbulent flows. Comput. Fluids 30, 555–579 (2001)
Guermond, J.-L., Oden, J.T., Prudhomme, S.: Mathematical perspectives on large eddy simulation models for turbulent flows. J. Math. Fluid Mech. 6, 194–248 (2004)
Hwang, F.-N., Cai, X.-C.: A parallel nonlinear additive Schwarz preconditioned inexact Newton algorithm for incompressible Navier-Stokes equations. J. Comput. Phys. 204, 666–691 (2005)
Hwang, F.-N., Wu, C.-Y., Cai, X.-C.: Numerical simulation of three-dimensional blood flows using domain decomposition method on parallel computer. J. Chin. Soc. Mech. Eng. 31, 199–208 (2010)
John, V.: On the efficiency of linearization schemes and coupled multigrid methods in the simulation of a 3D flow around a cylinder. Int. J. Numer. Meth. Fluids 50, 845–862 (2006)
Karypis, G.: METIS/ParMETIS webpage. University of Minnesota, http://glaros.dtc.umn.edu/gkhome/views/metis (2012)
Mahesh, K., Costantinescu, G., Moin, P.: A numerical method for large-eddy simulation in complex geometries. J. Comput. Phys. 197, 215–240 (2004)
Moin, P., Mahesh, K.: Direct numerical simulation: a tool in turbulence research. Annu. Rev. Fluid Mech. 30, 539–578 (1998)
Murillo, M., Cai, X.-C.: A fully implicit parallel algorithm for simulating the nonlinear electrical activity of the heart. Numer. Linear Algebra. Appl. 11, 261–277 (2004)
Owen, S.J., Shepherd, J.F.: CUBIT project webpage. http://cubit.sandia.gov/ (2012)
Piomelli, U.: Large-eddy simulation: achievements and challenges. Prog. Aeosp. Sci. 35, 335–362 (1999)
Rahimian, A., Lashuk, I., Veerapaneni, S., Chandramowlishwaran, A., Malhotra, D., Moon, L., Sampath, R., Shringarpure, A., Vetter, J., Vuduc, R., Zorin D., Biros, G.: Petascale direct numerical simulation of blood flow on 200k cores and heterogeneous architectures. In: Proceedings ACM/IEEE Supercomputing Conference, (2010)
Saad, Y.: Iterative Methods for Sparse Linear Systems. PWS Publishing Company, Boston (1996)
Saad, Y., Schultz, M.H.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7, 856–869 (1986)
Sagaut, P.: Large Eddy Simulation for Incompressible Flows. Springer, Berlin (2000)
Schäfer, M., Turek, S.: Benchmark computations of laminar flow around a cylinder. Notes Numer. Fluid Mech. 52, 547–566 (1996)
Toselli, A., Widlund, O.: Domain Decomposition Methods: Algorithms and Theory. Springer, Berlin (2005)
Yokokawa, M., Itakura, K.I., Uno, A., Ishihara, T., Kaneda, Y.: 16.4-Tflops direct numerical simulation of turbulence by a Fourier spectral method on the Earth Simulator. In: Proceedings ACM/IEEE Supercomputing Conference, (2002)
Author information
Authors and Affiliations
Corresponding author
Additional information
The research was supported in part by NSF of USA under grants DMS-0913089 and CCF-1216314, the Knowledge Innovation Program of the Chinese Academy of Sciences (China) under KJCX2-EW-L01, and the international cooperation project of Guangdong provience (China) under 2011B050400037.
Rights and permissions
About this article
Cite this article
Chen, R., Wu, Y., Yan, Z. et al. A Parallel Domain Decomposition Method for 3D Unsteady Incompressible Flows at High Reynolds Number. J Sci Comput 58, 275–289 (2014). https://doi.org/10.1007/s10915-013-9732-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10915-013-9732-x
Keywords
- Three-dimensional unsteady incompressible flows
- Unstructured finite element
- Parallel computing
- Fully implicit
- Domain decomposition