Professional Documents
Culture Documents
06 D.openMP SchedulingClauses
06 D.openMP SchedulingClauses
06 D.openMP SchedulingClauses
Vellore-632014,
CSE5006: Tamil
Multicore Architectures; Nadu,
Dr M India
Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Outline
Static Scheduling
Dynamic Scheduling
Guided Scheduling
Summary, References
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Session Plan
Skill and
Time Learning Aid and Typical Student
Content Faculty Approach Competency
(in min) Methodology Activity
Developed
Comprehensi
05 Re-Cap Quiz Questions Answers
on
Guided Scheduling
20 Presentation Explains Observes Synthesis
Comparisons
Conclusion and Listens Comprehensi
05 Summary Facilitates
Summary Participates on
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Objectives & Teaching Learning Material
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
RE-CAP
1. List two routines to estimate Execution Time (E.T) of a program
3. Name the header file to be included for estimating E.T. using Windows
routine
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Analogy
Analogy
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Schedule clauses
Analogy
Schedule Clauses
• A parallel region has atleast one barrier at its end. At the
barrier, the other members of the team must wait for the last
thread to arrive.
• To minimize this wait time shared work should be
distributed, so that all threads arrive at the barrier almost at
the same time.
• If some of the shared work is contained in for construct
then the schedule clause can be used.
• It gives control over which loop iterations are executed by
which thread.
• If each iteration takes the same amount of time to complete
its task, synchronization is not required to distribute the
work among the threads ie; scheduling is not required at all.
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Analogy THREAD 0 THREAD 1 THREAD 2 THREAD 3
Schedule Clauses
END OF THE
TASK
BARRIER POINT
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Schedule clauses
Analogy
Schedule Clauses • The three main scheduling clauses that are
Static
Dynamic
used to assign iterations to threads in the team
Guided are:
– Static
– Dynamic
– Guided
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
The Static Schedule
Analogy
Schedule Clauses
• The syntax for the static scheduling clause is:
Static
schedule(static [,chunk])
Dynamic
Guided • The static scheduling clause divides loop iterations
Static Schedule
Syntax
into pieces of size chunk
• The static scheduling clause divides loop iterations
into pieces of size chunk
• The last chunk to be assigned may have a smaller
number of iterations.
• If the chunk size is not specified, the static schedule
divides the iterations into chunks that are
approximately equal in size
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
The Static Schedule
Analogy
Schedule Clauses
• and distributes iterations among the threads in the
Static team so that no thread has more than one chunk.
Dynamic
Guided • When no chunksize specified
Static Schedule
Syntax
• Iterations are assigned to threads one after the other,
means thread1 gets iteration1,thread2 gets
iteration2……..till all iterations are assigned. The
iterations are assigned cyclically to each thread.
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
The Static Schedule
Analogy
Schedule Clauses
• The number of iterations assigned to each thread
Static when chunk size is not specified is given by
Dynamic
Guided total number of iterations
Static Schedule
Syntax
number of threads
• Suppose there are 16 iterations and 4 threads, then
the number of iterations assigned to each thread is
given by
16/4=4
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
The Static Schedule
Analogy SCHEDULE(STATIC)
Schedule Clauses
T1 T2 T3 T4 T1 T2 T3 T4 T1 T2 T3 T4 T1 T2 T3 T4
Static
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Dynamic T1,T2,T3,T4 are threads
Guided
Static Schedule • When chunksize is specified
Syntax – Iteration space is divided into chunks,
Example
• each of size chunk size specified.
– The chunks are assigned cyclically to each thread.
SCHEDULE(STATIC,2)
T1 T2 T3 T4 T1 T2 T3 T4
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
T1,T2,T3,T4 are threads
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Example – Scheduling Clauses
Analogy #pragma omp parallel for schedule(static,8)
Schedule Clauses for (int i=start; i<=end; i+=2)
Static {
Dynamic if (TestForPrime(i))
Static Schedule gPrimesFound++;
}
Syntax
Guided
Example • In this example, iterations are divided into chunks
of size 8. If the value of start is 3, the first chunk is:
i = {3, 5, 7, 9, 11, 13, 15, 17}
• At runtime, further chunks are created similarly.
These chunks are then assigned to threads in a
round-robin manner.
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
The Dynamic Schedule
Analogy
Schedule Clauses
• The syntax for the dynamic scheduling clause is:
Static
schedule(dynamic [,chunk])
Dynamic
Guided • The dynamic scheduling clause divides loop
Static Schedule
Syntax
iterations into pieces of size chunk and initially
Example assigns one chunk to a thread.
Dynamic Schedule
Syntax • When a thread completes its iterations, it
dynamically grabs the next chunk.
• The default chunk size is one.
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
The Dynamic Schedule
Analogy
Schedule Clauses
• When no chunksize specified
Static – When the chunk size is not specified it takes the default
Dynamic
Guided
value as 1.
Static Schedule – Iterations are assigned to threads one at a time as they
Syntax become available.
Example
Dynamic Schedule – The number of iterations assigned to each thread when
Syntax chunk size is not specified is unpredicted because it
Example depends on the readily available thread at that time.
T0 T3 T3 T1 T2 T2 T2 T0 T0 T3 T2 T0 T1 T1 T3 T2 T3 T1 T3 T0
SCHEDULE(DYNAMIC)
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
The Dynamic Schedule
Analogy
Schedule Clauses
• When chunksize is specified
Static – Divides the iteration space into chunks of size chunksize
Dynamic
Guided
and assigns them to threads on a first-come first-served
Static Schedule basis. i.e. whichever thread finishes a chunk, it is
Syntax assigned the next chunk in the list.
Example
Dynamic Schedule
Syntax SCHEDULE(DYNAMIC,4)
Example
T0 T0 T0 T0 T2 T2 T2 T2 T1 T1 T1 T1 T0 T0 T0 T0 T3 T3 T3 T3
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Application for Dynamic
Analogy
Schedule Clauses • Power of a matrix:
Static
Let A and B are two matrices of same order 1✕10 (say)
Dynamic
Guided
Static Schedule
A=[29 5 3 24 14 6 38 9 16 142 ]
Syntax
Example
Dynamic Schedule
B=[ 32 2 279 648 5 3 785 3 5 931]
Syntax
Example
Application
C=[2932 52 3279 24648 145 63 38785 93 165 142931]
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Code for the Power Matrix Program
Analogy
Schedule Clauses
Static
Dynamic
#pragma omp parallel for schedule(dynamic,2)
Guided
Static Schedule
for(i=0;i<SIZE;i++)
Syntax
Example
{
Dynamic Schedule answerMatrix[i]=pow(baseMatrix[i%SIZE],po
Syntax
Example werMatrix[i%SIZE]);
Application
threadId[i]=omp_get_thread_num();
}
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Output for the dynamic scheduling with chunk size 4.
Analogy
Schedule Clauses
Static
Dynamic
Guided
Static Schedule
Syntax
Example
Dynamic Schedule
Syntax
Example
Application
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
The Guided Schedule
Analogy
Schedule Clauses
• The syntax for the dynamic scheduling clause is:
Static
schedule(guided [,chunk])
Dynamic
Guided • The guided scheduling clause initially assigns
Static Schedule
Syntax
iterations larger than the chunk size.
Example • It starts with large blocks and assigns them to
Dynamic Schedule
Syntax threads dynamically.
Example
Application
• The assigned chunk size is decreased exponentially
Guided Schedule with each succeeding assignment until it reaches the
Syntax chunk size specified.
• The default chunk size is 1
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
The Guided Schedule
Analogy
• In addition to the static, dynamic, and guided scheduling clauses,
Schedule Clauses
there is another scheduling clause called the runtime schedule.
Static
Dynamic • The runtime scheduling clause takes a schedule and chunk from the
Guided string assigned to the environment variable OMP_SCHEDULE at run
Static Schedule time.
Syntax • Like dynamic, the guided schedule guarantees that no thread waits at
Example the barrier for longer time.
Dynamic Schedule
• It is similar to dynamic, but the chunks start off large and get smaller
Syntax
Example
exponentially.
Application • Initially the tasks that require more computation are identified and
Guided Schedule sorted in decreasing order of their computational power.
Syntax • The tasks that requires more computations are assigned with more
number of iterations to compute when compared with the tasks that
require less computations
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Analogy
Schedule Clauses
Static
Dynamic
Guided SCHEDULE(GUIDED)
Static Schedule
Syntax
Example
T1 T1 T1 T1 T1 T3 T3 T3 T3 T2 T2 T2 T3 T3 T0 T0 T1 T1 T3 T3
Dynamic Schedule
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Syntax
Example
Application
T0,T1,T2,T3 are threads
Guided Schedule
Syntax
Example
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
How Guided Actually Works
Analogy
Schedule Clauses • For guided scheduling, partitioning of loop is done
Static based on the following formula with a start value
Dynamic of α = number of loop iterations.
Guided
Static Schedule
Syntax 2 N
Example
• Where N is the number of thread, denotes the
Dynamic Schedule size of Kth chunk and denotes the size of
Syntax
Example
remaining unscheduled loop iteration while
Application
computing the size of Kth chunk.
Guided Schedule • For example =800, N=2 and S=80, where S is
Syntax chunk size.
Example
• The loop partition is 200,150,113,85,80,80,80,12
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Application for Guided
Analogy
Schedule Clauses • Printing Wedding Cards
Static
Dynamic Problem definition:
Guided
Static Schedule The customer gave an order of printing 500 wedding
Syntax cards to the printing press which contains 4 workers A, B,C and D
Example with different level of skills i.e. A works faster than B, B works
Dynamic Schedule faster than C and C works faster than D. Based on the skills of a
Syntax workers the no of cards assigned to them vary, that means Initially
Example A may get 140(say) cards to print, B may get 90 cards, C may get 75
Application
cards and D gets 50 cards. Next time who ever completes their job
Guided Schedule
first they will be assigned with some new task likewise the task is
Syntax
shared among the four workers.
Example
Application A(140) B(90) C(75) D(50) B(60) A(50) C(20) D(15)
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Which Schedule to Use?
Analogy
Schedule Clauses
Static Scheduling Clause When to Use
Dynamic
Guided
Static Predictable and similar work per iteration
Static Schedule
Syntax
Example Dynamic Unpredictable, highly variable work per iteration
Dynamic Schedule
Syntax
Guided Special case of dynamic to reduce scheduling overhead
Example
Application
Guided Schedule
Runtime Scheduling decision to be made at run time
Syntax
Example
Application
When to use
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Summary
• Analogy – Example
• Schedule Clauses – Application
– Static • Guided Schedule
– Dynamic – Syntax
– Guided – Example
• Static Schedule – Application
– Syntax • When to use
– Example
• Dynamic Schedule
– Syntax
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Homework
1. Identify an application for demonstration of each scheduling
clause
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
Summary
T2: Shameem Akhter and Jason Roberts, Multi-Core Programming, Intel
Press,1st Edition2006.
T5: Chapman, Gabriele Jost, Ruud van van der Pas, Using OpenMP:
Portable Shared Memory Parallel Programming (Scientific and
Engineering Computation), The MIT Press, 2007
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.
CSE5006: Multicore Architectures; Dr M Rajasekhara Babu, Vellore Institute of Technology (VIT); 21-Sep-18 12:13 AM Sl.No.