PYFR installations at NSC


Level of support

Tier 3 NSC will not be able to help you much with this program, either because we lack the in-house experience, or because it is a test installation. In general, these types of installations are untested, and will not be updated unless you send a request to NSC.

Please see the page describing our software support categories for more information.
You can also contact support@nsc.liu.se for further information.

Official homepage: http://www.pyfr.org

PyFR is an open-source Python based framework for solving advection-diffusion type problems on streaming architectures using the Flux Reconstruction approach of Huynh. The framework is designed to solve a range of governing systems on mixed unstructured grids containing various element types. It is also designed to target a range of hardware platforms via use of an in-built domain specific language derived from the Mako templating engine.

Please contact NSC Support if you have any questions or problems.

How to use

Load the pyfr module corresponding to the version you want to use, e.g

module load pyfr/1.6.0

For a complete description how to run PYFR, please consult the official user guide: http://pyfr.org/user_guide.php

Current Restrictions

Currently, we only support to run PYFR on multicore CPUs. The GPU version will be provided soon. Concerning mesh files, support for CGNS files is still missing. Only the standard format Gmesh is available. It is not fully clear yet, in how far PYFR supports OPENMP parallelization.

PYFR Basics

PyFR 1.6.0 uses three distinct file formats:
    .ini — configuration file
    .pyfrm — mesh file
    .pyfrs — solution file

1. pyfr import    - convert a Gmsh .msh file or CGNS .cgns file into a PyFR .pyfrm file
2. pyfr partition - partition an existing mesh. On Triolith, we use Metis 5 to partition the mesh
3. pyfr run       - run the simulation

PYFR examples

PYFR provides a few basic examples. These examples are included in the official distribution. On Triolith you may copy the examples to your local directory. Please find the examples included for version PYFR 1.6.0 at:

/software/apps/pyfr/PyFR-1.6.0/examples

Three testcases are provided by default: couette_flow_2d, euler_vortex_2d, inc_cylinder_2d

How to run a simple example

Practical example how to run the testcase “couette_flow_2d” on Triolith.

Directory of testcase files: /software/apps/pyfr/PyFR-1.6.0/examples/couette_flow_2d

Copy the mesh file and configuration file into your local directory

  • Mesh file: couette_flow_2d.msh
  • Configuration file: couette_flow_2d.ini

PYFR uses so called backends, depending if you want to run it on CPUs or GPUs. Currently, PYFR only runs on regular CPUs on Triolith. This backend is referred to as “openmp”. Support for GPUs will be provided in the near future.

In the configuration file, one has to edit the settings for the “backend-openmp”. For Triolith, edit the following settings:

[backend-openmp]
cc = gcc
cblas = /software/apps/OpenBLAS/0.2.19/g62/lib/libopenblas_seq_ilp64.so

Convert the Gmsh mesh file into PYFR mesh file format

pyfr import couette_flow_2d.msh couette_flow_2d.pyfrm

Run PYFR in serial mode

pyfr run -b openmp -p couette_flow_2d.pyfrm couette_flow_2d.ini

Run PYFR with MPI

Split the mesh into domains

pyfr partition <number of domains> couette_flow_2d.pyfrm .

Attention: When splitting the mesh, the different domains are stored all together in one common mesh file with the original name of the mesh. That means, it overwrites the original mesh file. By default the new mesh file will be saved in the current directory “.” If you prefer to keep a separate copy of the file, that contains the splitted domains, create a new directory and provide the directory name in the command line, e.g.

pyfr partition <number of domains> couette_flow_2d.pyfrm ./new_directory

This will create a second mesh file with the name “couette_flow_2d.pyfrm” in “./new_directory”

Run in parallel

mpirun -n <number of domains> pyfr run -b openmp -p couette_flow_2d.pyfrm couette_flow_2d.ini

Options:

  • “-b openmp” : Use the openmp backend for multicore CPUs on Triolith.
  • “-p” : Enable PYFR to print the progress of the computation on screen. Without this option,
    there will be no information on the progress.