Augier, P, et al.

2019 FluidDyn: A Python Open-Source Framework for

Journal of Research and Teaching in Fluid Dynamics by Simulations, Experiments
open research software and Data Processing. Journal of Open Research Software, 7: 9. DOI:
https://doi.org/10.5334/jors.237

SOFTWARE METAPAPER

FluidDyn: A Python Open-Source Framework for

Research and Teaching in Fluid Dynamics by Simulations,
Experiments and Data Processing
Pierre Augier1, Ashwin Vishnu Mohanan2 and Cyrille Bonamy1
1
University of Grenoble Alpes, CNRS, Grenoble INP, LEGI, Grenoble, FR
2
Linné Flow Centre, Department of Mechanics, KTH, Stockholm, SE
Correspondig author: Pierre Augier (pierre.augier@univ-grenoble-alpes.fr)

FluidDyn is a project to foster open-science and open-source in the fluid dynamics community. It is thought
of as a research project to channel open-source dynamics, methods and tools to do science. We propose a
set of Python packages forming a framework to study fluid dynamics with different methods, in particular
laboratory experiments (package fluidlab), simulations (packages fluidfft, fluidsim and fluidfoam)
and data processing (package fluidimage). In the present article, we give an overview of the specialized
packages of the project and then focus on the base package called fluiddyn, which contains common
code used in the specialized packages. Packages fluidfft and fluidsim are described with greater detail
in two companion papers [4, 5]. With the project FluidDyn, we demonstrate that specialized scientific
code can be written with methods and good practices of the open-source community. The Mercurial
repositories are available in Bitbucket (https://bitbucket.org/fluiddyn/). All codes are documented using
Sphinx and Read the Docs, and tested with continuous integration run on Bitbucket Pipelines and Travis. To
improve the reuse potential, the codes are as modular as possible, leveraging the simple object-oriented
programming model of Python. All codes are also written to be highly efficient, using C++, Cython and
Pythran to speedup the performance of critical functions.

Keywords: Fluid dynamics research with Python; Numerical simulations; Laboratory experiments; Free and
open-source software; modular; object-oriented; collaborative; efficient; tested; documented
Funding statement: This project has indirectly benefited from funding from the foundation Simone et Cino
Del Duca de l’Institut de France, the European Research Council (ERC) under the European Union’s Horizon
2020 research and innovation program (grant agreement No 647018-WATU and Euhit consortium) and the
Swedish Research Council (Vetenskapsrådet): 2013-5191. We have also been able to use supercomputers
of CIMENT/GRICAD, CINES/GENCI (grant 2018-A0040107567) and the Swedish National Infrastructure
for Computing (SNIC).

(1) Overview fantastic opportunities for human collaboration, and of

Introduction course, for science.
Science is mainly a collective activity. We can go further Web related activities alone account for at least 5%
only by “standing on the shoulders of giants” (and of of GDP in the USA and the European Union.1 A huge
a huge number of technicians and scientists). Science amount of money (and work) is invested on the related
is a lot about how to build new knowledge from the technologies. We are familiar with the most prominent
work of others and hence, the exchange of ideas is a companies involved in this dynamics (Google, Facebook,
fundamental aspect. In the last decades, we have lived etc.), but there are also several smaller and sometimes
through a revolution on how people exchange ideas. lesser known organizations. Most of these companies
Computers of all kinds (from smartphones to HPC base part of their work on the open-source paradigm
clusters) connected by a world wide web are used contributing to open-source languages, libraries, software
for human communication and also for many other and operating systems, while also using them — a win-win
applications. It has become nearly effortless to reproduce situation for both the corporations and the community.
and exchange ideas and data. The set of intangibles that This has lead to deep changes in software engineering,
grows when shared and degrades when hoarded, such with a massive use of open-source methods and tools, for
as knowledge and love, has been somehow extended. example distributed version control systems (DVCS) and
The information technology revolution open doors to web-based source development platforms.
Art. 9, page 2 of 6 Augier et al: FluidDyn

The computer performance continues to increase rapid development cycle made the need to decentralize
exponentially, now also with the help of Graphical FluidDyn evident. Now, FluidDyn project hosts a number
Processing Units (GPU). This gave way to a big boom in of specialized packages, namely:
practical uses of data science and machine learning, which
drives a strong research on artificial intelligence. Such • fluiddyn3: The base package which contains
developments contribute to progresses in open-source pure-python code that can be reused in scripts or in
software. To summarize, there is a strong dynamics in specialized FluidDyn packages. It also contains codes
play around the use of computers (in particular with the for miscellaneous command-line utilities useful for a
web) and this creates very efficient tools and methods for typical fluid dynamics user.
collective work and software development. The code of this package is presented in further
These changes in our world also reflect in the way detail in its documentation (https://fluiddyn.readthe-
science is done. Software and programming in science docs.io/) and some prominent features are presented
occupy a much bigger place than before. The role of in the following subsection.
software in science has changed. In the past, coding • Transonic: a pure Python package to accelerate
was sometimes considered as an inferior activity by Python-Numpy code with Pythran and potentially
some scientists. The focus was on the theory and the other Python compilers.
mathematical demonstration, which had to be elegant as • fluidfft (see the companion paper [4]): a package
it gets included in the articles. In contrast, it was normal to which provides C++ and Python classes unifying vari-
write crude code and to just show the results. Nowadays, ous libraries to perform Fast Fourier Transform (FFT)
codes tend to be at the heart of research. in sequential and in parallel.
“Open-science” is a new trend taking advantage of these • fluidsim (see the companion paper [5]): Numerically
new facts. Pioneering attempts are being made to do better oriented framework to run sequential and parallel
science, improving reproducibility and collective efficiency, Computational Fluid Dynamics (CFD) simulations and
by using the open-source methods and tools for science on-the-fly post-processing for a variety of problems
and sharing and collaborating via the world wide web. (Navier-Stokes, Shallow Water, Föppl von Kármán
FluidDyn is a project to foster open-science and equations, to name a few). A study using fluidsim
open-source coding in Python in the field of fluid mechanics. has just been published in Physics of Fluids [3].
The project envisages to provide the technical framework • fluidlab: Package to handle laboratory experiments.
to allow collaborative development of tools useful for the Primarily used to communicate with various hard-
fluid mechanics community, and to do science with open ware devices such as motors and pumps, to handle
methods. We provide examples of solutions for: I/O between sensors, and to store data. Experiments
using fluidlab have been carried out in the DAMTP
• Good coding practice with readable and easy to fluid laboratory (Cambridge, UK. cf [2]), in ENS Lyon
comprehend Python code (PEP 8). laboratory (Lyon, France [6]) and at LEGI (Grenoble,
• Source control management (Mercurial) and forge France [1]).
(Bitbucket) simple for the newcomers. • fluidimage: Scalable image processing package
• Packaging and installation procedure. All packages which implements various algorithms to calibrate
are available from PyPI and can be install simply with cameras, to preprocess images, to do Particle
pip, the standard Python installation tool. Image Velocimetry (PIV) and to postprocess data.
• Licenses: depending on the packages, we choose fluidimage is used to process images taken during
to use the CeCILL-B or the CeCILL licenses. These experiments performed in the Coriolis platform at
licenses are compatible respectively with BSD and LEGI [1].
GPL licenses and adapted to both international and • fluidfoam: Small package to load OpenFoam data
French legal matters.2 and plot them.
• Documentation produced with standard and up-to-date • fluidcoriolis: Small package used to carry out
tools: Sphinx, Anaconda and Jupyter. Built and hosted experiments in the Coriolis platform (a large rotating
online at Read the Docs. Can also be generated offline. platform participating in the European consortiums
• Unittest and continuous integration with Bitbucket Euhit and Hydralab) and open the data obtained (see,
Pipelines and Travis. for example [1]). One of the motivations behind creat-
ing this package is to study how to use open-source to
We hope that such a clean framework will facilitate create and share open-data.
contributions from scientists in the field and that we can
build together a nice, user-friendly and efficient ecosystem A detailed presentation on the above packages can be
specialized in research and teaching in fluid dynamics. found in their respective documentations on the web
and for fluidfft and fluidsim in the two companion
Implementation and architecture papers [4, 5]. The code base was designed to follow
Organization of the code in packages Python 2.7 syntax during its genesis. Now, it has been
FluidDyn was originally intended to be a single package to made forward compatible with Python 3 through the
perform experiments and simulations. Since a typical user use of external package future. We now tend to abandon
may not be involved in both experiments and simulations Python 2.7 support for the next releases of the package.
and also, due to increasing complexity as a virtue of This article will now focus on the base package fluiddyn.
Augier et al: FluidDyn Art. 9, page 3 of 6

API of the Python library fluiddyn The classes of this module allows one to do it with a quite
All functions and classes defined in fluiddyn are pure simple and general API as shown in the tutorial on the
Python elements, meaning that no extensions are module.
implemented in fluiddyn. Thus the package fluiddyn
is extremely easy to install with just a pip install Module fluiddyn.util.mpi
command and no compilation. This simple module makes simultaneous sequential
The package fluiddyn is organized into five and MPI programming a breeze by providing number of
sub-packages: processes, nb_proc = 1 and rank = 0, when used
in sequential mode, otherwise providing the appropriate
• fluiddyn.io: This subpackage provides utilities for values provided by mpi4py package. Use of this module
input/output to different file formats. thwarts coding several if-else clauses. If the program was
• fluiddyn.util: Miscellaneous utilities. using MPI, also defines the variable comm as an alias for
• fluiddyn.calcul: Modules for numerical compu- the MPI.COMM_WORLD communicator.
tations.
• fluiddyn.clusters: Classes to launch jobs on Module fluiddyn.calcul.easyfft
HPC clusters. Thin wrapper for an unified API using classes around the
• fluiddyn.output: Utilities to produce scientific packages pyfftw and scipy.fftpack. It is very easy
outputs (figures and videos). to perform forward and inverse Fast Fourier Transforms
(FFT) in one-, two- and three-dimensions. The FFT can be
Sub-packages io, util and calcul are the largest multithreaded if the environment variable OMP_NUM_
in terms of lines of code and provides the Application THREADS is defined.
Programming Interfaces (API) to support fluidsim,
fluidlab and fluidimage. For the sake of brevity, we shall The fluiddyn command-line utilities
only describe here some of the most important modules. The package fluiddyn also provides few command-line
utilities to perform simple tasks useful for a scientist
Module fluiddyn.util.paramcontainer developing with Python and using the FluidDyn
Sub-package paramcontainer defines class packages.
ParamContainer which is a hierarchical container for
any type of parameters. As shown in this tutorial, various • fluidinfo
strengths of an object of this class include: Displays important information related to software
and hardware. It includes detailed information such
• Support containing and printing documentation on as currently installed FluidDyn packages, other third-
the parameters. party packages, C compiler, MPI and Numpy configu-
• Support printing to console as XML and saving as ration.
XML, HDF5 and NetCDF Files. • fluiddump
• Evaluate data types automatically while loading from Utility to print the hierarchy of HDF5 and NetCDF
saved files. files. It does not depend on the NetCDF4 library.
• Easy exploration in an interactive console. • fluidnbstripout
• Allows modification of default parameters through Very simple layer to stripout Jupyter notebooks of
simple Python script files. An error is raised if the user output, which is useful to keep the notebooks light-
attempts to use an invalid parameter. weight when included in a repository. This tool is
• Graphical User Interface (GUI) frontend with PyQt. based on nbstripout but, in contrast to nbstripout,
by default the notebooks with a file name ending as
Thus, it makes it a much more robust implementation ‘.nbconvert.ipynb’ are excluded.
for saving key parameters, compared to conventional • fluidcluster-help
methods which rely on text or CSV files. Tiny utility to print a short documentation on the
most useful commands to interact with the scheduler
Module fluiddyn.util.serieofarrays of a HPC cluster.
This module provides classes to iterate over files. It is a • fluidmat2py
common task in data processing to understand how to Utility to produce a strange code which is no longer
organize a sequence of files from filenames and formats. Matlab and not yet Python. This strange code is then
Typically, one has to form smaller sets of arrays contained much easier to translate into correct Python than the
in the files. For example, we may have a sequence such as: original Matlab code.

(im1_1.png, im1_2.png, im1_3.png, im2_1.png, im2_2.png, im2_3.png)

from which we could create different subsets like,

((im1_1.png, im1_2.png, im1_3.png), ((im1_1.png, im2_1.png),
(im2_1.png, im2_2.png, im2_3.png)) (im1_2.png, im2_2.png),
(im1_3.png, im2_3.png))
Art. 9, page 4 of 6 Augier et al: FluidDyn

Quality control • Julien Salort (Laboratoire de physique, ENS de Lyon):

The package fluiddyn currently supplies unit tests developer of fluidlab.
covering around 70% of its code. These unit tests are run
regularly through continuous integration on Travis CI with Software location
the most recent releases of fluiddyn’s dependencies and Name: PyPI
on Bitbucket Pipelines inside a static Docker container. Persistent identifier: https://pypi.org/project/fluiddyn
The tests are run using standard Python interpreter with Licence: CeCILL-B, a BSD compatible French licence.
all supported versions. Version published: 0.2.4
For fluiddyn, the code coverage results are displayed Date published: 02/07/2018
at Codecov. Using third-party packages coverage and
tox, it is straightforward to bootstrap the installation Code repository
with dependencies, test with multiple Python versions Name: Bitbucket
and combine the code coverage report, ready for upload. Persistent identifier: https://bitbucket.org/fluiddyn/
It is also possible to run similar isolated tests using tox fluiddyn
or coverage analysis using coverage in a local machine. Licence: CeCILL-B
Up-to-date build status and coverage status are displayed Date published: 2015
on the landing page of the Bitbucket repository. We also
try to follow a consistent code style as recomended by Language
PEP (Python enhancement proposals) 8 and 257. This is English
also inspected using lint checkers such as flake8 and
pylint among the developers. The code is regularly (3) Reuse potential
cleaned up using the Python code formatter black. As a library, fluiddyn has been used in the project’s
specialized packages. The common code base for
(2) Availability packages with such varied applications is a proof of
Operating system fluiddyn’s versatility and generality. It can be used
Windows and any POSIX based OS, such as GNU/Linux in other packages outside the FluidDyn project easily
and macOS. depending on the need. The command-line tools can be
useful to all scientists working with Python. Other use
Programming language cases could be:
Python 2.7, 3.4 or above. For the next versions, we will drop
Python 2.7 support and Python >= 3.6 will be required. • ParamContainer as a generic parameter storage
standard.
Dependencies • Modules within fluiddyn.util for miniature tasks,
We list here only the dependencies of the base package printing in terminal with colours, getting memory
fluiddyn. usage information, detecting if the current Python
session is inside IPython, creating a string with time
• Minimum: Numpy, Matplotlib, psutil, and date, etc.
future, subprocess32 (for Python 2.7 only), • Modules within fluiddyn.io to read and save
h5py, h5netcdf. images of various formats including TIFF files with
• Full functionality: mpi4py, Scipy, pyfftw multiple images; easily ask user with yes/no queries;
(requires FFTW library), pillow. handle different file formats using classes, including
• Optional: OpenCV with Python bindings, CSV, HDF5, Digiflow, Dantec formats.
scikit-image. • Modules inside fluiddyn.clusters subpackage
to make job submission scriptable in HPC clusters
List of contributors with OAR or SLURM job schedulers. It could be even
• Pierre Augier (LEGI): creator of the FluidDyn project, used for non-Python jobs.
developer of majority of the FluidDyn packages,
future-proofing with Python 3 compatibility and There is no formal support mechanism. However, bug
documentation. reports can be submitted at the Issues page on Bitbucket.
• Ashwin Vishnu Mohanan (KTH): developer of the Discussions and questions can be aired on instant
packages fluidsim, fluidfft, fluidimage and messaging channels in Riot (or equivalent with Matrix
fluiddyn; documentation, code coverage and protocol)4 or via IRC protocol on Freenode at #fluiddyn-
continuous integration (Docker, Bitbucket Pipelines users. Discussions can also be exchanged via the official
and Travis CI). mailing list.5
• Cyrille Bonamy (LEGI): developer of fluidfft, flu-
idsim, fluidimage and fluidfoam. Main main- Conclusions
tainer of fluidfoam. FluidDyn is an attempt to set off collaborative dynamics
• Antoine Campagne (LEGI): developer of fluidimage based on open-source development in fluid dynamics
and fluidlab. research. We shall try, with this project, to explore the
• Miguel Calpe Linares (LEGI): developer of fluidsim possibilities of open-source in science and fluid dynamics
and fluidlab. by fully exploiting the new open-source tools and methods.
Augier et al: FluidDyn Art. 9, page 5 of 6

The project is right now in a preliminary stage. Packages Acknowledgements

are actively evolving with interesting features and a We thank the CNRS to finance the work of Pierre Augier
framework for collaborative development, packaging, while giving freedom in terms of scientific project.
documenting and testing is now well set. However, the Similarly, Ashwin Vishnu Mohanan could not have been
community around the project is currently tiny and as involved in this project without the kindness of Erik
now, we have to work on attracting users and developers Lindborg. We thank Antoine Campagne, Miguel Calpe
since an active community is a criteria for success and Linares and Julien Salort for their implications in the
sustenance of an open-source project. development of some of the FluidDyn packages. We are
Will people use the FluidDyn tools and collaborate grateful to Bitbucket for providing us with a high quality
through the project FluidDyn? There are clearly many forge compatible with Mercurial, free of cost. Finally,
challenges and potential barriers: we thank Gabriel Moreau and Joël Sommeria for their
constant will to discuss and share their knowledge.
• Some habits in the community.
• Knowledge and skills in the community. For example, Competing Interests
only few people use issue tracker and pull requests. The authors have no competing interests to declare.
• Lack of a business model for open-source software
in science. A good quality open-source software has References
a cost that institutions should be willing to fund and 1. Campagne, A, Alfredsson, H, Chassagne, R,
support. Micard, D, Mordant, N, Segalini, A, Sommeria, J,
• Lack of recognition of the work spent in open- Viboud, S, Mohanan, A V, Lindborg, E and Augier,
source, in particular for scientists without permanent P 2016 First report of the MILESTONE experiment:
positions. Strongly stratified turbulence and mixing efficiency in
the coriolis platform, VIIIth International Symposium
On our side, we also have positive points. The quality on Stratified Flows (ISSF, San Diego, USA).
of the tools we use (Python and its scientific ecosystem, 2. Leclercq, C, Partridge, J L, Augier, P, Dalziel, S
Mercurial, Read the Docs, Jupyter, …) is impressive. B and Kerswell, R R 2016 ‘Using stratification to
Scientific code is done to be read and to transmit ideas. mitigate end effects in quasi-Keplerian TaylorCouette
To this effect, Python is among the best languages today. flow’. J. Fluid Mech, 791: 608–630. DOI: https://doi.
Python starts to be a standard tool in fluid dynamics, org/10.1017/jfm.2016.44
especially used for CFD (Dedalus, SpectralDNS, TriFlow, 3. Lindborg, E and Mohanan, A V 2017 ‘A two-
PyLBM, Oasis, PyFR, FEniCS, Cassiopee, pyCGNS, etc.) and dimensional toy model for geophysical turbulence’.
data analysis (OpenPTV, PyPIV). Moreover, we can benefit Phys. Fluids, 29(11): 111114.
from the dynamics of Python and of emerging subjects 4. Mohanan, A V, Bonamy, C and Augier, P 2019a
like deep learning and the Internet of Things. Finally, if we ‘FluidFFT: Common API (C++ and Python) for
manage to gather a community of users and of developers, Fast Fourier Transform libraries’. J. Open Research
the collective efficiency related to open-source methods Software (DOI).
and tools can be a strong booster. 5. Mohanan, A V, Bonamy, C and Augier, P 2019b
‘FluidSim: Modular, objectoriented python package for
Notes high-performance CFD simulations’. J. Open Research
1
See the report by Internet Association titled “Refreshing Software (DOI).
Our Understanding of the Internet Economy”. 6. Salort, J, Rusaouëen, É, Robert, L, Du Puits, R,
2
http://www.cecill.info/licences.en.html. Loesch, A, Pirotte, O, Roche, P-E, Castaing, B and
3
We use FluidDyn (with capital letters) to name the pro- Chillà, F 2018 ‘A local sensor for joint temperature and
ject and fluiddyn for the base package. velocity measurements in turbulent flows’. Review of
4
https://matrix.to/#/#fluiddyn-users:matrix.org. Scientific Instruments, 89(1): 015005. DOI: https://doi.
5
https://www.freelists.org/list/fluiddyn. org/10.1063/1.4989430
Art. 9, page 6 of 6 Augier et al: FluidDyn

How to cite this article: Augier, P, Mohanan, A V and Bonamy, C 2019 FluidDyn: A Python Open-Source Framework for
Research and Teaching in Fluid Dynamics by Simulations, Experiments and Data Processing. Journal of Open Research
Software, 7: 9. DOI: https://doi.org/10.5334/jors.237

Submitted: 02 July 2018 Accepted: 15 February 2019 Published: 01 April 2019

Copyright: © 2019 The Author(s). This is an open-access article distributed under the terms of the Creative Commons
Attribution 4.0 International License (CC-BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium,
provided the original author and source are credited. See http://creativecommons.org/licenses/by/4.0/.

Journal of Open Research Software is a peer-reviewed open access journal published by

Ubiquity Press
OPEN ACCESS