PARALLEL COMPUTING- simultaneous processing to solve a comp problem

characteristics:
-

Specific input description

Specific output description

PARALLEL PROGRAMMING- writing of program to solve parallel problems

PARALLEL SYSTEMS- computer system capable of performing parallel computing
MOTIVATION:
-

Process large quantities of data

Speed- system offers fast processing
High performance computing (HPC)

MULTIPROGRAMMING two or more programs being executed at the same time

PARALLEL TASK- can be executed by multiple processors
Characteristics of parallelizable problems
-

Must be a problem that can be decomposed

Problem must allow execution of multiple instruction on it at the same time
High ratio of local memory access to remote memory
The problem can be solved in less time with multiple resources than with a
single computing resource

Forms of parallelism
i.
ii.
iii.
iv.
v.

Embarrassingly parallel- same application runs on different data without

dependency
Almost embarrassingly parallel- minimum coordination between
application is required
Synchronous parallelism- each operation performed on data depends on
result of prior operation
Pipeline the tasks that have to be completed sequentially can be
transformed into partially sequential tasks
Loosely synchronous parallelism- minimal intermittent data sharing

Automatic vs. Manual Parallelism

Automatic- parallelizing identifies opportunities for parallelism and generates a
parallel code. The programmer does need to worry about parallelism.
The compiler is in charge of entire process.
Limited to parallelizing loops i.e. loops are picked by compiler

Manual- programmer is responsible for identifying and implementing parallelism

Scalability of Parallel Systems
Deals with improving service delivery and scope of parallel systems
Factors influencing scalability
1.
2.
3.
4.
5.

Hardware type
Algorithms
Parallel overheads
Application characteristics
Quality of parallel programming