Introduction to OpenMP

Dr M Rajasekhara Babu
Vellore Institute of Technology (VIT)
Vellore-632014, Tamil Nadu, India
 Outline
Session objectives

Introduction, Parallel Programming Models,

OpenMP Architecture

OPenMP Programming Model, Parallel

Construct

Sample Program Demo

Summary

Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT)-Vellore Slide.# 2

 Objectives
To provide knowledge on
OpenMP

To Combine serial and

parallel codes by using
various constructs

To create threaded codes

by using OpenMP
pragmas.
Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT)-Vellore Slide.# 3
 Parallel programming
– Models
Parallel Programming models
 Parallel programming
– Models
Parallel Programming models
– Popular models
 Parallel programming
– Models
What Is OpenMP*?
– Popular models
 OpenMP C$OMP FLUSH #pragma omp critical

C$OMP THREADPRIVATE(/ABC/) CALL OMP_SET_NUM_THREADS(10)

call omp_test_lock(jlok)
C$OMP parallel do shared(a, b, c)

C$OMP MASTER
call OMP_INIT_LOCK (ilok)

C$OMP
http://www.openmp.org
SINGLE PRIVATE(X)
C$OMP ATOMIC

Current spec is OpenMP

setenv 4.0“dynamic”
OMP_SCHEDULE
C$OMP PARALLEL DO ORDERED PRIVATE (A, B, C)
250 Pages C$OMP ORDERED

C$OMP PARALLEL REDUCTION (+: A, B)

(combined C/C++ and C$OMP
Fortran)
SECTIONS

#pragma omp parallel for private(A, B) !$OMP BARRIER

C$OMP PARALLEL COPYIN(/blk/) C$OMP DO lastprivate(XX)

Nthrds = OMP_GET_NUM_PROCS() omp_set_lock(lck)

 Parallel programming
– Models
OpenMP Basic Defs: Solution Stack
– Popular models
 OpenMP End User
– Solution stack
Application

Directives, Environment
OpenMP library
Compiler variables

OpenMP Runtime library

OS/system support for shared memory and threading

Proc1 Proc2 Proc3 ProcN

Shared Address Space

 Parallel programming
– Models OpenMP* Architecture
– Popular models
 OpenMP
• Fork-join model
– Solution stack • Work-sharing constructs
– Architecture
• Data environment constructs
• Synchronization constructs
• Extensive Application Program Interface
(API) for finer control
• Thread implementation within OpenMP is
based on the fork-join model that facilitates
parallel execution of threads.

Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT)-Vellore Slide.# 8

 Parallel programming
– Models Programming Model
– Popular models
• Fork-join parallelism:
 OpenMP
– Solution stack – Master thread spawns a team of threads as needed
– Architecture – Parallelism is added incrementally:
– Programming model • the sequential program evolves into a parallel program
• The application starts executing serially with a
single thread called the master thread.
• The execution of parallel regions of a program
following the fork-join model proceeds through
the following two stages:

Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT)-Vellore Slide.# 9

 Parallel programming
– Models
fork-join model
– Popular models
 OpenMP F J F J

– Solution stack O O O O

R R I
– Architecture I
K N K N
– Programming model
• Fork-Join model
Parallel
Master
Regions
Thread

• Fork
– It creates a team of parallel threads called worker
threads that execute the code in the parallel region
concurrently.
• Join
– At the end of the parallel region, the worker threads
synchronize and terminate, leaving only the master
thread.
 Parallel programming
– Models
Terms for OpenMP
– Popular models
 OpenMP
• Parallel constructs
– Solution stack – Indicate to the compiler to use multiple threads
– Architecture to facilitate parallel execution of code.
– Programming model
• Fork-Join model • Work-sharing constructs
 Terms for OpenMP
– Indicate to the compiler to generate code to
automatically distribute workload across the
team of threads.
• Data environment constructs
– Control data environment during the execution
of parallel constructs by making variables
shared or private.

Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT)-Vellore Slide.# 11

 Parallel programming
– Models
Terms for OpenMP
– Popular models
 OpenMP
• Data environment constructs
– Solution stack – Control data environment during the execution
– Architecture of parallel constructs by making variables
– Programming model
• Fork-Join model shared or private.
 Terms for OpenMP • Extensive API library for finer control
– Contains features that provide implementation-
specific capabilities and run-time control.

Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT)-Vellore Slide.# 12

 Parallel programming
– Models
OpenMP core syntax
– Popular models • Most of the constructs in OpenMP are compiler
 OpenMP directives.
– Solution stack
– Architecture #pragma omp construct [clause [clause]…]
– Programming model • Example
• Fork-Join model
 Terms for OpenMP
#pragma omp parallel
 OpenMP Core Syntax • Function prototypes and types in the file:
#include <omp.h>
• Most OpenMP* constructs apply to a “structured block”.
• Structured block
– a block of one or more statements with one point of entry at the
top and one point of exit at the bottom.
– It’s OK to have an exit() within the structured block.

Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT)-Vellore Slide.# 13

 Parallel programming
– Models
Parallel Region Constructs
– Popular models • The parallel region construct is used to parallelize serial
 OpenMP code.
– Solution stack – A team of worker threads is created to execute the code in
– Architecture parallel.
– Programming model
• Parallel regions
• Fork-Join model
– refer to structured blocks of code that can be executed in parallel
 Terms for OpenMP
by a team of threads.
 OpenMP Core Syntax
• Structured blocks
 Parallel constructs
– refer to the code segments that have one entry point at the top and
one exit point at the bottom.
• Specify a parallel region by using the
– omp parallel pragma.
• The omp parallel pragma operates over a single statement
or block of statements enclosed within curly braces.

Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT)-Vellore Slide.# 14

 Parallel programming Structured Block Vs Unstructured Block
– Models
– Popular models A Structured Block
 OpenMP
– Solution stack #pragma omp parallel
– Architecture {
– Programming model more: res(id) = do_big_job(id);
• Fork-Join model
if (conv(res[id])) goto more;
}
 Terms for OpenMP printf(“All done\n”);
 OpenMP Core Syntax
 Parallel constructs
 Structured block An Unstructured Block

 Unstructured block
#pragma omp parallel
{
more: res(id) = do_big_job(id);
if (conv(res[id])) goto done;
goto more;
}
done: if (!really_done()) goto more;
 Parallel programming
– Models
Flow diagram
– Popular models • Defines parallel
 OpenMP
– Solution stack
region over
#pragma omp parallel
– Architecture structured block of
– Programming model code
• Fork-Join model
 Terms for OpenMP
• Threads are created Thread Thread Thread Thread

as ‘parallel’ pragma 0 1 2 3
 OpenMP Core Syntax
 Parallel constructs is crossed
 Structured block • Threads block at end
 Unstructured block
– Flow diagram
of region
• Data is shared C/C++ :
among threads #pragma omp parallel
{
unless specified block
otherwise }
Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT)-Vellore Slide.# 16
 Parallel programming
– Models
OpenMP first program
– Popular models
 OpenMP
• program that prints “hello world”.
– Solution stack
– Architecture
– Programming model int main()
• Fork-Join model
 Terms for OpenMP {
 OpenMP Core Syntax
 Parallel constructs
 Structured block
 Unstructured block int ID = 0;
– Flow diagram
 OpenMP first program printf(“ hello(%d) ”, ID);
printf(“ world(%d) \n”, ID);
}
Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT)-Vellore Slide.# 17
 Parallel programming
– Models
OpenMP first program
– Popular models
 OpenMP
• Multithreaded program that prints “hello
– Solution stack world”.“omp.h”
#include
– Architecture int main()
– Programming model
• Fork-Join model
{
 Terms for OpenMP
 OpenMP Core Syntax #pragma omp parallel
 Parallel constructs
 Structured block {
 Unstructured block int ID = 0;
– Flow diagram
 OpenMP first program printf(“ hello(%d) ”, ID);
printf(“ world(%d) \n”, ID);
}
}
Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT)-Vellore Slide.# 18
 Parallel programming
– Models
OpenMP first program
– Popular models
• Multithreaded program that prints “hello world”.
 OpenMP OpenMP include
– Solution stack #include “omp.h” file
– Architecture
– Programming model int main()
Parallel region
• Fork-Join model { with default
 Terms for OpenMP number of threads

 OpenMP Core Syntax

#pragma omp parallel
 Parallel constructs
 Structured block
{
 Unstructured block int ID = omp_get_thread_num();
– Flow diagram printf(“ hello(%d) ”, ID);
 OpenMP first program printf(“ world(%d) \n”, ID); Runtime library
 Runtime library function
} function to
return a thread
} End of the Parallel region ID.

Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT)-Vellore Slide.# 19

 Parallel programming
– Models
OpenMP first program
– Popular models
 OpenMP
• Multithreaded program that prints “hello
– Solution stack world”.
#include “omp.h”
OpenMP include
file
– Architecture
– Programming model int main()
Parallel region with
• Fork-Join model { default number of
 Terms for OpenMP threads

 OpenMP Core Syntax

#pragma omp parallel Sample Output:
 Parallel constructs hello(1) hello(0) world(1) world(0)

 Structured block
{ hello (3) hello(2) world(3) world(2)

 Unstructured block int ID = omp_get_thread_num();

– Flow diagram printf(“ hello(%d) ”, ID);
 OpenMP first program printf(“ world(%d) \n”, ID);
 Runtime library function
} Runtime library
function to return
} End of the Parallel region a thread ID.

Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT)-Vellore Slide.# 20

 Parallel programming
– Models
OpenMP program Demo
– Popular models
 OpenMP
– Solution stack
– Architecture
– Programming model
• Fork-Join model
 Terms for OpenMP
 OpenMP Core Syntax
 Parallel constructs
 Structured block
 Unstructured block
– Flow diagram
 OpenMP first program
 Runtime library function

Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT)-Vellore Slide.# 21

 Threading concepts Summary
• Parallel Syntax
programming • Parallel constructs
– Models • Structured block
– Popular models
• Unstructured block
• OpenMP – Flow diagram
– Solution stack
• OpenMP first
– Architecture
program
– Programming model
• Fork-Join model
• Runtime library
function
• Terms for OpenMP
• OpenMP Core
Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT)-Vellore Slide.# 22