LAB MANUAL

PARALLEL AND DISTRIBUTED COMPUTING

CSE3009

Submitted by:

Reg No: 21BCE10419

Name: Vibhor Mathur

BACHELOR OF TECHNOLOGY
in

CSE (CORE)

Submited to:

Dr. Ravi Verma

SCHOOL OF COMPUTING SCIENCE AND ENGINEERING

VIT BHOPAL UNIVERSITY

1
 INDEX

S.NO CONTENT PG.N Remarks SIGNATURE

O.
LAB 1 – Basics and Fundamentals of
1 OpenMP 3-4

LAB 2 – OPENMP – PROGRAM

2 FOR VECTOR ADDITION
5-7

LAB 3 – OPENMP – PROGRAM

3 FOR DOT PRODUCT 8-11

LAB 4 – OpenMP program to

demonstrate the sharing of a loop
4 iteration by number of threads 12-13

5 LAB5 - OpenMP program to 14-16

demonstrate the sharing of works of a
section using threads

LAB 6 - MPI – BASICS OF MPI.

6 17-18

LAB 7 - MPI – COMMUNICATION

7 BETWEEN MPI PROCESS
19-20

2
 Lab No: 1
OPENMP – BASIC AND FUNDAMENTALS

​Aim:
To understand the basic concepts and fundamentals of OpenMP parallel programming model and
its application in parallel computing.

Objective :

1. To comprehend the significance of OpenMP in parallel programming.

2. To learn fundamental OpenMP directives and their usage.
3. To explore synchronization mechanisms and data scope in OpenMP.
4. To understand performance considerations and best practices for efficient parallel
programming using OpenMP.

Introduction :

OpenMP (Open Multi-Processing) is a widely-used API for parallel programming in

shared-memory systems. It allows developers to write parallel programs in an easy and portable
manner. In this lab, we will delve into the basics and fundamentals of OpenMP, exploring its
directives, synchronization mechanisms, data scope, and performance considerations.

Key Points:

1. Parallel Programming : OpenMP simplifies writing parallel code by adding directives to

existing code, eliminating the need for complex threading.

2. Shared Memory Model: OpenMP operates on the shared memory model, where multiple
threads access shared data, simplifying communication and synchronization.

3. Directives: OpenMP directives like parallel, for, and sections are added to code to specify
areas for parallel execution, making code parallelization straightforward.

4. Synchronization: OpenMP provides mechanisms like barriers and locks for synchronizing
access to shared resources, ensuring data integrity in parallel execution.

5. Data Scope: Understanding data scope (private, shared, etc.) is crucial for managing data
access and ensuring correctness in parallel programs.

6. Performance: Considerations like load balancing and minimizing overhead are important
for optimizing performance in parallel programs.

7. Best Practices: Adhering to best practices ensures efficient utilization of OpenMP, leading
to better performance and scalability in parallel programs.
3
Conclusion:

In conclusion, this lab provided a foundational understanding of OpenMP and its role in parallel
programming. By grasping basic concepts, directives, synchronization mechanisms, and
performance considerations, learners are better equipped to write efficient parallel programs
using OpenMP. Understanding these fundamentals lays a solid groundwork for exploring more
advanced parallel programming techniques.

4
 Lab No: 2
OPENMP – PROGRAM FOR VECTOR ADDITION

​Aim:
To examine the scenario, where we are going to take two one-dimensional arrays, each of size of 5.
We will then create 5 threads. Each thread will be responsible for one addition operation.

​ Procedure:

1. Open a text editor or IDE to create a new C source file.

2. Compile the code using the OpenMP-enabled compiler. Ensure that the
appropriate flags for OpenMP are included (e.g., -fopenmp for GCC).
3. Experiment with different loop and section configurations to observe the impact
on Performance.

PROGRAM:

Vector addition:

//Vector Addition Q.1

#include <stdlib.h>

#include <stdio.h>

#include <omp.h>

#define ARRAY_SIZE 20

#define NUM_THREADS 20

int main (int argc, char *argv[])

int * a;

5
 int * b;

int * c;

int n = ARRAY_SIZE;

int n_per_thread;

int total_threads = NUM_THREADS;

int i; // loop index

a = (int *) malloc(sizeof(int)*n);

b = (int *) malloc(sizeof(int)*n);

c = (int *) malloc(sizeof(int)*n);

for(i=0; i<n; i++) {

a[i] = i;

for(i=0; i<n; i++) {

b[i] = i;

omp_set_num_threads(total_threads);

n_per_thread = n/total_threads;

#pragma omp parallel for shared(a, b, c) private(i) schedule(static,

n_per_thread)

for(i=0; i<n; i++) {

c[i] = a[i]+b[i];

printf("Thread %d works on element%d\n", omp_get_thread_num(), i); }

printf("i\ta[i]\t+\tb[i]\t=\tc[i]\n");

for(i=0; i<n; i++) {

printf("%d\t%d\t\t%d\t\t%d\n", i, a[i], b[i], c[i]);

}
6
 free(a); free(b); free(c);

return 0;

​ OUTPUT:

RESULT:

For vector addition – The parallelized vector addition program demonstrated a

significant reduction in execution time compared to its sequential counterpart.

7
 Ex. No: 3
OPENMP – PROGRAM FOR DOT PRODUCT

​ AIM:

To understand how threads can be executed in parallel to sequentially check

each element of the list for the target value until a match is found or until
all the elements have been searched.

​ PROCEDURE:

1. Open a text editor or IDE to create a new C source file.

2. Compile the code using the OpenMP-enabled compiler. Ensure that the
appropriate flags for OpenMP are included (e.g., -fopenmp for GCC).
3. Run the compiled executable to execute the code on your system.
4. Execute the compiled program and observe the parallel execution of addition
operations by monitoring processor usage and thread activity.
5. Evaluate the performance impact of parallel execution compared to sequential
execution by analyzing execution time and efficiency metrics.

​ PROGRAM:

Dot product:

//Dot Product

#include <stdio.h>

#include <stdlib.h>t

#include <omp.h>

#define SIZ 5

int main (int argc, char *argv[]){

8
float a[SIZ], b[SIZ], dotprod,dpp;

int i, tid; dotprod=0.0;

for(i=0;i<SIZ;i++)

a[i]=1.0*(i+1);

b[i]=1.0*(i+2);

printf("\n Values of a and b :\n");

for (i=0;i<SIZ;i++)

printf(" a[%d]= %.1f\t b[%d]= %.1f\n",i,a[i],i,b[i]);

printf("\n");

#pragma omp parallel shared(a,b,dotprod,dpp) private (tid,i)

tid=omp_get_thread_num();

#pragma omp for private (i)

for(i=0;i<SIZ;i++)

dpp+=a[i]*b[i];

printf("thread: %d\n", tid);

9
 }

#pragma omp critical

dotprod=dpp;

printf("thread %d\n",tid);

printf("\n Dot Product = %f\n",dotprod);

​ OUTPUT:

10
​ RESULT:

For dot product - The parallelized dot product calculation exhibited notable speedup,
especially when dealing with large input vectors. The use of the reduction (+:
dotProduct) clause effectively handled the parallel reduction operation, ensuring accurate
results.

11
 Ex. No: 4 OpenMP program to demonstrate the sharing of a loop iteration by
number of threads

​ AIM:

To illustrate the sharing of loop iterations among multiple threads using OpenMP
directives with a chunk size of 10.

​ PROCEDURE:

1. Open a text editor or IDE to create a new C source file.

2. Write a simple C program utilizing OpenMP directives to parallelize a loop with a
chunk size of 10.

​ PROGRAM:

#include <stdio.h>
#include <omp.h>

#define CHUNK_SIZE 10
#define ARRAY_SIZE 100

int main() {
int array[ARRAY_SIZE];
int i;

// Initialize array
for (i = 0; i < ARRAY_SIZE; i++) {
array[i] = i;
}

// Parallel loop with chunk size of 10

#pragma omp parallel for schedule(static, CHUNK_SIZE)
for (i = 0; i < ARRAY_SIZE; i++) {
printf("Thread %d processes iteration %d\n", omp_get_thread_num(), i);
12
 }

return 0;
}

​ OUTPUT:

​ RESULT:

The program demonstrates the sharing of loop iterations among threads using
OpenMP directives with a chunk size of 10. Each thread processes a chunk of 10
iterations sequentially, showcasing parallel execution of loop iterations.
Adjusting the chunk size may affect thread workload distribution and program
performance.

13
 Ex. No: 5
OpenMP program to demonstrate the sharing of
works of a section using threads

​AIM:
Write a open MP program to demonstrate the sharing of works of a section using
threads. You can perform arithmetic operations on the one dimensional array
and this section load can be shared by the threads.

​PROCEDURE:

1. Open a text editor or IDE to create a new C source file.

2. Write a C program utilizing OpenMP directives to perform arithmetic
operations on a one-dimensional array, distributing the workload among threads
using sections.

​PROGRAM:

#include <stdio.h>
#include <omp.h>

#define ARRAY_SIZE 100

int main() {
int array[ARRAY_SIZE];
int i;

// Initialize array
for (i = 0; i < ARRAY_SIZE; i++) {
array[i] = i + 1; // Assign values from 1 to 100
}

14
int sum = 0;
int max_value = 0;

// OpenMP sections for performing arithmetic operations

#pragma omp parallel sections
{
// Section 1: Calculate sum of array elements
#pragma omp section
{
int local_sum = 0;
#pragma omp for
for (i = 0; i < ARRAY_SIZE; i++) {
local_sum += array[i];
}
#pragma omp critical
{
sum += local_sum;
}
}

// Section 2: Find maximum value in the array

#pragma omp section
{
int local_max = array[0];
#pragma omp for
for (i = 1; i < ARRAY_SIZE; i++) {
if (array[i] > local_max) {
local_max = array[i];
}
}
#pragma omp critical
{
if (local_max > max_value) {
15
 max_value = local_max;
}
}
}
}

printf("Sum of array elements: %d\n", sum);

printf("Maximum value in the array: %d\n", max_value);

return 0;
}

​OUTPUT:

RESULT:

The program demonstrates the sharing of work among threads using OpenMP
sections. Two sections are defined: one calculates the sum of array elements, and the
other finds the maximum value in the array. Each section's workload is distributed
among multiple threads, showcasing parallel execution and efficient utilization of
resources.

16
 Exp. No: 6
MPI – BASICS OF MPI.

​ AIM:

The primary objective of this experiment is to familiarize participants with the

fundamentals of Message Passing Interface (MPI), a widely used standard for
parallel programming in distributed-memory systems.

​ PROCEDURE:

1. Install a C compiler and an MPI library (e.g., Open MPI).

2. execute the compiled program in the MPI environment. Specify the number of
MPI processes to run.
3. modify the value of N in the code to explore how the program behaves with
larger or smaller summation ranges.
4. Discuss the use of profiling tools (if available) to analyze the performance of the
MPI program.

​ PROGRAM:

#include <stdio.h> #include

<mpi.h>
int main(int argc, char** argv) { int rank,
size;
int sum = 0;
int local_sum = 0;
int numbers[] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};

MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);

// Calculate local sum

for (int i = rank; i < 10; i += size) { local_sum
+= numbers[i];
}

// Reduce local sums to get global sum

17
MPI_Reduce(&local_sum, &sum, 1, MPI_INT, MPI_SUM, 0,
MPI_COMM_WORLD);

if (rank == 0) {
printf("Sum calculated by process %d: %d\n", rank, sum);
}

MPI_Finalize(); return 0;
}

​ OUTPUT:

​ RESULT:

The MPI program effectively parallelizes the summation task, distributing the
workload among multiple processes. This parallel approach can significantly
enhance the computational efficiency for large-scale problems. The program
produces the correct result, as the final output matches the expected sum of
numbers from 1 to 10 (55). The use of MPI_Reduce ensures accurate aggregation
of local sums. The scalability of the program is evident as it can adapt to
different numbers of MPI processes. The parallel nature of the computation
allows for efficient utilization of resources in a distributed-memory system.

18
 Ex. No: 7
MPI – COMMUNICATION BETWEEN MPI PROCESS.

​ AIM:

The primary objective of this experiment is to implement communication

between MPI processes using MPI_Send and MPI_Recv.

​ PROCEDURE:

1. Initialize MPI environment.

2. Get the rank and size of the MPI communicator.
3. If the rank is 0, send data to rank 1 using MPI_Send.
4. If the rank is 1, receive data from rank 0 using MPI_Recv.
5. Finalize MPI environment.

​ PROGRAM:

#include <stdio.h> #include

<string.h> #include
<mpi.h>

int main(int argc, char** argv) { int rank,

size;
char message[100];

MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);

if (size < 2) {
fprintf(stderr, "Run Successful\n");
MPI_Abort(MPI_COMM_WORLD, 1);
}

if (rank == 0) {
// Process 0 sends a message to Process 1

19
strcpy(message, "Hello from process 0"); MPI_Send(message,
strlen(message) + 1, MPI_CHAR, 1, 0,
MPI_COMM_WORLD);
} else if (rank == 1) {
// Process 1 receives the message from Process 0 MPI_Recv(message, 100,
MPI_CHAR, 0, 0, MPI_COMM_WORLD,
MPI_STATUS_IGNORE);
printf("Received message on process 1: %s\n", message);
}

MPI_Finalize(); return 0;
}

​ OUTPUT:

​ RESULT:

The MPI program effectively demonstrates communication between two MPI

processes, showcasing the fundamental capability of MPI for parallel
programming. By sending data from Process 0 to Process 1 using MPI_Send and
receiving it using MPI_Recv, the program illustrates inter-process communication.
The code's simplicity, combined with error handling to ensure a minimum number
of processes, highlights MPI's suitability for distributed memory parallel
computing. Overall, this example underscores the importance of MPI in enabling
efficient communication and coordination among processes in parallel
applications.

Thank you

20