WOLAITA SODO UNIVERSITY SCHOOL OF INFORMATICS

AND COMPUTING
WEEKEND PROGRAM (MSC IN COMPUTER SCIENCE)
PARALLEL PROCESSING ARTICLE REVIEW ON
(The Structure of Parallel Algorithms)
 
Reviewed by
Lewi Thomas
Afework Milkias
Gebre Worku

Submitted to: - Mr. Temesgen (Ass.Professor)

 
June/14/2023
Wolaita Sodo, Ethiopia
June 2023
School of Informatics and Computing Wolaita Sodo University
Wolaita Sodo, Ethiopia

Article Review on the Structure of Parallel Algorithms

H. T. KUNG Lewi Thomas, Afework Milkias, Gebre Worku
Department of Computer Science Wolaita Sodo University School of Informatics and
Carnegie-Mellon University Computing Department of Computer Science
Pittsburgh. Pennsylvania

ABSTRACT
The article provides an It aims to provide a
overview of parallel algorithms comprehensive understanding of
and their classifications based the basic issues and techniques
on three important attributes: in designing parallel algorithms
for different architectures.
 concurrency control
The abstract concludes by
 module granularity and stating that the article aims to be
 communication geometry a useful resource for readers
The article presents various interested in understanding the
examples of parallel algorithms basics of parallel algorithm
and studies them under a design and choosing the
appropriate parallel architecture
uniform framework.
 Overview Of
Introduction
Parallel algorithms have been studied
since the 1960s, but the emergence of
large-scale parallel computers has
increased interest in designing parallel
algorithms.
The article presents examples of parallel
algorithms and aims to be a useful
resource for understanding the basic
issues and techniques in designing
parallel algorithms. A parallel algorithm
is a collection of independent task
modules that can be executed in parallel
and communicate with each other. The
three orthogonal dimensions of the space
of parallel algorithms are concurrency
control, module granularity, and
communication geometry
.
The Article
1.Concurrency Control:
concurrency control ensures the
correctness of concurrent execution in
parallel algorithms.
2. Module Granularity:
Module granularity refers to the amount of
computation a task module can perform
before needing to communicate with other
modules.
3. Communication Geometry
Suppose that task modules of a parallel
algorithm are connected to represent
intermodule communication. Then a
geometric layout of the resulting network
is referred to as the communication
geometry of the algorithm.
Matching Parallel Algorithms with Parallel Architectures
The article discusses the matching
along the dimensions of concurrency
control and module granularity. There
are three architectures and their
matching algorithms.
1.SIMD and MIMD Machines and
Algorithms
The notion of single-instruction
stream multiple-data stream (SIMD)
and multiple-instruction stream
multiple data stream (MIMD) is often
used for classifying parallel
computers in MIMD One stream of
instructions issued by the central
control unit controls all the
processors, each operating upon its
own memory synchronously
With a MIMD machine such as the
processors have independent instruction
counters, and operate asynchronously on
shared memories.SIMD machines correspond
to synchronous lock-step algorithms that
require central controls, whereas MIMD
machines correspond to asynchronous
algorithms with relatively large granularities.
2 Systolic Machines and Algorithms
systolic machine is a network of simple and
primitive processors that circulate data in a
regular fashion. Each processor regularly
pumps data in and out, each time performing
some short computation, so that a regular
flow of data is kept up in the network
A Taxonomy for Parallel Algorithms
The article proposes taxonomy
for parallel algorithms based on
the three dimensions mentioned
earlier: concurrency control,
module granularity, and
communication geometry.
Algorithms for Synchronous
Parallel Computers
This section focuses on parallel
algorithms for synchronous
parallel computers, including
systolic and SIMD machines.
Algorithms will be classified
based on their communication
geometries.
Algorithms Using One Dimensional
Linear Arrays
One-dimensional linear arrays are simple
and fundamental for connecting processors.
Four Examples of algorithms are discussed
in this article.
 Odd-Even Transposition Sort.
 Real-Time Finite Impulse Response
(FIR) Filtering
 Priority Queue
 Recurrence Evaluation (Recursive
Filtering)
Following the discussions of the four
algorithms,we mention the use of linear
pipelines in the implementation of
arithmetic operations.
 Algorithms Using Two-Dimensional Arrays
Two-dimensional
communication geometries that Algorithms Using Tree
are simple and regular are Structures
considered. In the article Tree structures are discussed
Algorithms using two- for chip implementation, and
dimensional hexagonal arrays
their advantages for
are presented, some of this are:
broadcasting, searching, and
 matrix multiplication
fan-in operations are
 LU decomposition
explained. Search algorithms
 transitive closure of a and systolic search trees are
Boolean matrix. highlighted as applications of
tree structures
 Algorithms for Asynchronous
Multiprocessors
Concurrent Data Base Systems
In a concurrent data base system,
a number of on-line transactions
are allowed to run concurrently on
a shared data base.
Algorithms for Specific
Problems
When designing an asynchronous
algorithm for solving a specific
problem, we have control over the
tasks that will be included in the
algorithm.
The major issues are the
correctness and efficiency of an
algorithm in then presence of the
asynchronous behavior of its
processes. For the general data
base environment where only
syntactic information can be
used, the
serialization approach is the
method for ensuring correctness.
Serialization can be achieved by
either locking or transaction
backup.
 Analysis of the Article
Here is a breakdown of the main points
covered in each section:
 Introduction to parallel algorithms and
their historical development. Increasing
interest in parallel algorithms due to the
emergence of large-scale parallel
computers.
 Mention of surveys and references on
parallel architectures and algorithms.
 The impact of large-scale integration
technology on parallel algorithms.
 The Space of Parallel Algorithms:
Taxonomy and Relation to Parallel
Architectures:
 Three important dimensions of parallel
algorithms:concurrency control,module
granularity, and communication
geometry.
 Classification of parallel algorithms
based on these dimensions.
 Correspondence between the
classification of algorithms and parallel
architectures
 Proposal of a taxonomy for parallel
algorithms in the systolic, SIMD,
MIMD communication geometries
space.
 Algorithms for Synchronous Parallel
Computers:
 Introduction to synchronous parallel
computers, including systolic and
SIMD machines.
 Classification of algorithms based on
communication geometries.
 Examples of algorithms using one-
dimensional linear arrays and two-
dimensional arrays (hexagonal arrays
and trees).
 Main Gaps of the Article
 The author not well reported Scalability
issues: Many parallel algorithms suffer
from scalability issues when the size of
data or number of processors increases
beyond certain limits. There is a need for
new algorithmic techniques and
strategies to improve scalability.
 The author not described Load
balancing in which the work load is
distributed evenly across all processors.
This is particularly challenging in
dynamic systems where the workload
may change over time.
 The author not clarified Faults and error
that occur in parallel systems, and it is
important to develop algorithms that are
resilient to such errors and can continue
to function correctly even in the
presence of faults.
 Portability: Parallel algorithms
developed for one architecture may
not be portable to other
architectures due to differences in
hardware and software.
CONCLUSUION
In general the paper presents
taxonomy for parallel algorithms
based on their concurrency control,
module granularity, and
communication geometry. It provides
examples of algorithms for different
communication geometries, including
linear arrays, hexagonal arrays, and
tree structures. The paper serves as a
valuable resource for understanding
the design principles and techniques
of parallel algorithms for various
architectures.
THANK YOU!!