Web : http://LearnCAx.

com
Email : info@learncax.com

LearnCAx Blog : Parallelization of UDFs in ANSYS FLUENT

Subhransu Majhi1
LearnCAx : Inspire | Educate | Mentor
Centre for Computational Technologies Pvt. Ltd., Pune, India

Note: You may download the serial as well as parallel UDF (C Program) files mentioned in
the test case below and their high resolution illustrative image comparing the two by
visiting the original blog

Hello Friends, in recent times I have been constantly writing about ANSYS FLUENT
Customization for CFD Simulations. And in continuation to it, today I am sharing a new
piece of knowledge which is short & simple but highly useful for ANSYS FLUENT users
having basic knowledge of UDFs - Parallelization of UDFs

Many of the advance ANSYS FLUENT users write UDF (User Defined Function) to ask ANSYS
FLUENT to do some special jobs. These UDFs are very popular in ANSYS FLUENT user
community. Most of the UDFs that we create are programmed for serial mode of ANSYS
FLUENT since it only involves basic C program coding with required marcos. However, to
utilize the power of parallel processing, we need to convert the serial UDF into parallel
UDF. Read this blog for a step by step process to convert serial UDF to parallel UDF.

1
Subhransu Majhi (subhransu@cctech.co.in), LearnCAx, Centre for Computational Technologies Pvt. Ltd

1 Akshay Residency, 50 Anand Park, Aundh, Pune – 411007, India © Copyrights : http://LearnCAx.com
 Web : http://LearnCAx.com
Email : info@learncax.com

Today we will understand how to convert the serial UDFs to run in Parallel mode in few
easy steps. To begin with, let us first understand what it means when we say parallel
mode UDFs. When you run a UDF into a serial FLUENT process, it gets executed by that
single process just once. Whereas, if you run a UDF under parallel mode of FLUENT, then
FLUENT executes your UDF (C program) on the host process as well as on the ‘n’ node
processes. It means that the same set of C program written in your UDF is read ‘n + 1’
times. A very common problem which can arise in such situation is that if in your UDF you
are creating a file and writing some text into it, then all these processes will try to create
this file at the same location and will try to insert the same text all at the same time.
Similarly, if we try to run a UDF which calculates the average pressure using the formula
“force/area” whose values are fetched from the mesh, then the host will perform “0/0”
since mesh data is not present in the host process at all. Also in most of the cases you
must have experienced that when a UDF meant for serial version of FLUENT is run on the
parallel version of FLUENT then FLUENT crashes with some error message. So it’s obvious
that you need to modify the UDF for supporting it in parallel version of FLUENT.

Let us now see how we can modify a UDF meant to be run on a single process to run on
multiple processes i.e. parallel version of FLUENT. You will get the below information
from ANSYS UDF guide but I thought of writing down the same in short with the help of the
issue I faced.

Before we begin, let us first see what all are we going to cover –

 Compiler Directives
 Predicates
 Global Reduction Macros

Firstly, let’s understand the scenario. In one of my projects, one of the objectives was to
find out the thermal information of a heat emitting device. So, at the end of the
simulation I was using a DEFINE-ON-DEMAND macro based UDF to perform the following
operations –

 Read a file containing IDs of the list of inlets & outlets of the heating device (all IDs
are written in a single line and are seperated with spaces between them)
 Calculate the surface max temperature, surface min temperature and area weighted
average of temperature for those IDs
 Write down the above calculated values into a file to be used for report generation

Now, since a need arised to speed up the simulation, I started shifting the FLUENT run into
parallel mode, but all of a sudden my serial mode UDF stopped working as expected.
Hence, I went thru ANSYS FLUENT's UDF guide and started learning about modify UDFs for
parallel use. I now have the modified UDF which works both in parallel as well as in serial
mode of FLUENT.

Let us watch side by side the serial UDF vs its parallel version and finally understand all
the necessary modifications done to it.

1 Akshay Residency, 50 Anand Park, Aundh, Pune – 411007, India © Copyrights : www.LearnCAx.com
 Web : http://LearnCAx.com
Email : info@learncax.com

1 Akshay Residency, 50 Anand Park, Aundh, Pune – 411007, India © Copyrights : www.LearnCAx.com
 Web : http://LearnCAx.com
Email : info@learncax.com

Now let us understand the modifications one by one –

Modification 1: Type - Compiler Directives: These are used to execute commands only on
certain types of processes, viz. Serial, Host and/or Parallel. In the above UDF we don’t
need to use the host process to do any activity, hence we are excluding it by the
#if !RP_HOST compiler directive. This will make the Serial (in serial mode) as well as the
Node process (in parallel mode) execute the complete DEFINE_ON_DEMAND UDF when ever
it is executed from FLUENT application on demand by the user. A compliler directive
needs to be ended with #endif. If you look at the end of the UDF you will find that I have
used #endif to close this compiler directive. You can wrap any part of your C program
using a compiler directive. There are following types of compiler directives -

 #if RP_HOST - code written inside this will be performed only by the HOST process
 #if RP_NODE - code written inside this will be performed only by the NODE process
 #if PARALLEL - code written inside this will be performed only if parallel version of FLUENT is started
 #if !RP_HOST - code written inside this will be performed only by the SERIAL & NODE processes
 #if !RP_NODE - code written inside this will be performed only by the SERIAL & HOST processes
 #if !PARALLEL - code written inside this will be performed only if serial version of FLUENT is started

Modification 2: Type - Compiler Directives: Similar to “Modification 1” but this time we

are using it for executing the wrapped if condition only if FLUENT is opened in parallel
mode.

Modification 3: Type - Predicates: The use of predicates are similar to that of compiler
directives. Predicates are pre-defined FLUENT parallel version's macros, which lets you run
a set of codes for a specific process or on the process where the selected object's primary
face is present. Below is the list of Predicates available in FLUENT's parallel version –

 MULTIPLE_COMPUTE_NODE_P
 ONE_COMPUTE_NODE_P
 ZERO_COMPUTE_NODE_P
 I_AM_NODE_HOST_P
 I_AM_NODE_ZERO_P
 I_AM_NODE_ONE_P
 I_AM_NODE_LAST_P
 I_AM_NODE_SAME_P(n)
 I_AM_NODE_LESS_P(n)
 I_AM_NODE_MORE_P(n)
 PRINCIPAL_FACE_P(f,t)

These above predicates can be used to conditionally perform a set of codes within a if
condition, for e.g. if you wish to perform a set of codes on the first node process then you
may use the following predicate –

if(I_AM_NODE_ZERO_P)
{
...expressions;
}

In this case, we are performing the file open to write operation only on the first node
process to avoid multiple file writing operations on each of the nodes. As this if condition
is wrapped inside the #if PARALLEL compiler directive, it wont’t be executed when
FLUENT is opened in Serial mode and the file opening operation would be directly

1 Akshay Residency, 50 Anand Park, Aundh, Pune – 411007, India © Copyrights : www.LearnCAx.com
 Web : http://LearnCAx.com
Email : info@learncax.com

executed. Hence, this makes the file opening operation section of this UDF compatible in
both Serial as well as Parallel mode of ANSYS FLUENT.

Modification 4: Type - Predicates: Similar to “Modification 3” but this time we are

calculating all the required values from a surface thru that node process which has the
surface's principal face present in it.

Modification 5: Type - Global Reduction Macros: We are performing node

synochronisation operations here. What it means is that, now since we have multiple node
processes, for every variable declared in all the node processes, there is a chance of this
variable having different values in different node processes. For e.g. lets say, the mesh of
a surface is distributed unevenly over two node processes and if we get the area of that
surface from each of the nodes into a variable, then it is obvious that this variable will
have different values for different nodes. Now since we need the summation of the values
of this variable from all the nodes to get the actual area of the surface, these global
reduction macros come into picture. Various available global reduction macros are as
follows –

Global Summations –

Variables

 PRF_GISUM1(x)
 PRF_GRSUM1(x)

Arrays

 PRF_GISUM(x,N,iwork)
 PRF_GRSUM(x,N,iwork)

Global Maximums and Minimums –

Variables

 PRF_GIHIGH1(x)
 PRF_GRHIGH1(x)
 PRF_GILOW1(x)
 PRF_GRLOW1(x)

Arrays

 PRF_GIHIGH(x,N,iwork)
 PRF_GRHIGH(x,N,iwork)
 PRF_GILOW(x,N,iwork)
 PRF_GRLOW(x,N,iwork)

Global Logicals -

Variables

 PRF_GLOR1(x)
 PRF_GLAND1(x)

Arrays

 PRF_GLOR(x,N,work)

1 Akshay Residency, 50 Anand Park, Aundh, Pune – 411007, India © Copyrights : www.LearnCAx.com
 Web : http://LearnCAx.com
Email : info@learncax.com

 PRF_GLAND(x,N,iwork)

Global Synchronization -

 PRF_GSYNC()

These global reduction macros present in FLUENT parallel mode come in various versions
like most macros have two versions - one for integers & the other for real numbers. Then
we have macros to operate on a variable as well as on an array.

In our case, we are traversing thru each of the surface using their IDs and are trying to get
the lowest & highest temperature value from all the nodes and doing global summation
of the values of temperature * area & area, so that these variables would get
synchronized across all the node processes.

Modification 6: Type - Compiler Directives & Predicates: Same as “Modification 1 & 2”

Modification 7: Type - Compiler Directives & Predicates: Same as “Modification 1 & 2”

This is how you modify your serial UDF into parallel one. If you go thru the above serial vs
parallel UDF comparison multiple times, you will not find it difficult to convert your own
UDFs into parallel mode. I hope this knowledge share proves helpful to someone
somewhere in need of it. If you feel it as an useful resource then kindly be in touch with
us or even better let us know what you would like to hear from us in our upcoming blogs!
Wish you good luck in your FLUENT customization work.

Feel free to contact me in case you have any query regarding any of your UDF work or if
you need more information on any of the discussed topics from this blog, I will try to
resolve your queries to the best of my knowledge.

Reference:

 ANSYS FLUENT User Guide

1 Akshay Residency, 50 Anand Park, Aundh, Pune – 411007, India © Copyrights : www.LearnCAx.com
 Web : http://LearnCAx.com
Email : info@learncax.com

LearnCAx knowledge database

Courses offered by LearnCAx are designed to meet all your needs. It

has range of FREE and PREMIUM courses which is designed to meet
the industry requirements. All the courses are self-contained with
video lectures, quiz, assignments and projects. Every course comes
with FAQs and discussion forums where you can get answers to all
your questions. Each course contains live assistance from LearnCAx
faculty where faculties will guide you through online sessions and
desktop sharing.

Blogs is the place where our coaches share their knowledge through
articles. This includes best practices, advance techniques, and recent
development in respective field. LearnCAx is backed with strong
industrial consultancy team. This team does projects for industries.
As LearnCAx main focus is “from academics to industry”, blogs gives
us an opportunity to share details about our industrial work. It’s not
only about what is done, but also about how the project is done. The
objective is to give student’s more knowledge about industrial
project so that they feel connected to the industry.

No matter what is the form of learning, an interaction with experts is

an inevitable part of every learning process. LearnCAx faculty
conducts webinars to share the knowledge with you. Let it be
knowhow of software, introduction to a particular topics or
discussing fundamentals of a subject, all webinars are targeted
towards sharing the knowledge and getting feedback about what
your training needs are? Webinar is also a place for our consultancy
team to share their work with you. All these live sessions would give
an opportunity to you to talk to the experts in the domain.

Create FREE LearnCAx account to access all the knowledge base

1 Akshay Residency, 50 Anand Park, Aundh, Pune – 411007, India © Copyrights : www.LearnCAx.com