14/10/2021

1-Introduction
•Introduction •Three New Computing
Course No. SE-7123 / CE 7110
•The Platform Evolution Paradigms
•Evolutionary Trend •Computing Paradigm
•High-Performance Distinctions
Cloud Computing Computing •Centralized Computing
(Platform Evolution, HPC, HTC, Computing Paradigm
•High-Throughput •Parallel Computing
Distinctions)
Computing •Distributed Computing
•Cloud Computing
1 2

2-Type of Distributed System Design Goals 3-Architectural Styles, Middleware Organization

•Introduction
•Middleware & Distributed •Types of distributed •Architectures
Systems •Architectures Style
systems. • Layered Architectures
•Design Goals • Cluster Computing. • Object-based Architectures
• Supporting Resource Sharing. • Grid Computing. • Resource-Centered Architectures
• Distribution transparent. • Cloud Computing. • Event-based Architectures
• Openness. •Distributed •Middleware Organization
• Scalability. Information Systems.

3 4

4,5-Architectural Styles, Middleware Organization
• System Architecture
• Centralized Organizations
• Simple Client-Server Architecture
• Multitiered Architecture
• Decentralized Organizations
• Peer-to-Peer Systems
• Structured Peer-to-Peer Systems
• Unstructured Peer-to-Peer Systems
5 6
7-Code Migration
• Code Migration
• Advantage
• Disadvantage
• Models for Code Migration
• Code Segment
• Resource Segment
• Execution Segment
• Sender-Initiated Migration
• Receiver-Initiated Migration
7 8
14/10/2021
6-Process, Thread, Virtualization,
• Processes • Principle of Virtualization
• Decentralized Organizations
• Threads • Virtualization & Distributed
• Structured Peer-to-Peer
Systems • Thread implementation Systems
• Unstructured Peer-to-Peer • Lightweight Processes • Type of Virtualization
Systems
• Hierarchically Organized Peer- • Threads in Distributed • Application of Virtual Machines
to DS
to-Peer Networks Systems
• Hybrid Architectures • Multithreaded Servers • Client
• Edge-Server Systems
• Collaborative Distributed • Virtualization • Client-side software for
Systems distribution transparency
8-Communication
• Client-Server Computing • Introduction • Stub Generation
• Remote Evaluation • Type of Communication • Variations on RPC
• Code-on-Demand • Remote Procedure Call • Asynchronous RPC
• Basic RPC Operations • Multicast RPC
• Mobile Agents
• RPC Steps • DCE RPC
• Weak Mobility
• Parameter Passing • Binding a Client to a Server
• Strong Mobility • Parameter passing in Object- • Message-Oriented
• Migration in based Systems Communication
Heterogeneous Systems • RPC-based Application Support • Advanced Transient
Messaging

9-MPI and Naming
• The Message-Passing
Interface
• Naming
• Flat Naming
• Broadcasting
• Forwarding Pointers
• Home-based Approaches
• Distributed Hash Tables
• Hierarchical Approaches
9 10
11-DME & Election Algorithms
• Distributed Mutual Exclusion
• Single-Computer vs. Distributed
System
• DME: Requirements
• DME: Performance
• DME: A Simple Solution
• DME: LAMPORT’S ALGORITHM
• DME: RICART AGRAWALA
11
• Structured Naming
• Implementation of a
Name Space
• Attribute-based Naming
• Hierarchical
implementations
• DME: A generalized Non-
token based Algorithm
• DME: RAYMOND’S TREE-
BASED
• DME: SUZUKI-KASAMI’S
ALGO
• Election Algorithms
• The Bully Algorithm
12
10-DHT & Coordination
• DHT
• Coordination
• Clock Synchronization
• Logical Clocks
• Lamport’s Logical Clocks
• Vector Logical Clock
• Distributed Mutual Exclusion
12-Distributed Deadlock Detection
• Introduction
• Resource vs.
Communication Deadlocks
• A Graph-Theoretic Model
• Resource Allocation Graph
• Wait-For Graph
• Deadlock handling
strategies in DS
• Deadlock Prevention
14/10/2021
• Single-Computer vs.
Distributed System
• DME: Requirements
• DME: Performance
• DME: A Simple Solution
• DME: Lamport’s Algorithm
• DME: The Ricart-Agrawala
Algorithm
• Deadlock Avoidance
• Deadlock Detection and
Recovery
• Control Organizations
• Centralized Deadlock-Detection
• Distributed Deadlock Detection
Algorithms
• Algorithms Unleashed
Perspectives
 14/10/2021

13-Distributed Deadlock Detection 14-Presentation Session by Students

• Introduction
• Resource vs. Communication
Deadlocks
• A Graph-Theoretic Model
• Resource Allocation Graph
• Wait-For Graph
• Deadlock handling strategies
in DS
• Deadlock Prevention
• Deadlock Avoidance
• Deadlock Detection and
Recovery
• Control Organizations
• Centralized Deadlock-
Detection
• Distributed Deadlock
Detection Algorithms
• Algorithms Unleashed
Perspectives
• Introduction
• Resource vs. Communication
Deadlocks
• A Graph-Theoretic Model
• Resource Allocation Graph
• Wait-For Graph
• Deadlock handling strategies
in DS
• Deadlock Prevention
• Deadlock Avoidance
• Deadlock Detection and
Recovery
• Control Organizations
• Centralized Deadlock-
Detection
• Distributed Deadlock
Detection Algorithms
• Algorithms Unleashed
Perspectives

13 14

Introduction
• In this session we learn about the evolutionary
changes that have occurred in parallel, distributed,
and cloud computing over the past 30 years.
• Its driven by applications with variable workloads
and large data sets.
• We study both High-Performance (HPC) and High-
Throughput Computing (HTC) systems in parallel
computers appearing as Computer Clusters, Service-
Oriented Architecture, Computational Grids, Peer-to-
Peer Networks, Internet Clouds and the IoT.
Introduction
•These systems are distinguished by their
Hardware Architectures, OS Platforms,
Processing Algorithms, Communication
Protocols, and Service Models applied.
•We also identify/learn essential issues on the
Scalability, Performance, Availability, Security,
and Energy Efficiency in distributed systems.

15 16
 14/10/2021

The Platform Evolution Evolutionary Trend

•Computer technology • 1950 to 1970 (Mainframes) • Evolutionary trend
has gone through five • 1960 to 1980 (Lower-Cost toward parallel,
generations of Minicomputers)
distributed, and
development, with each • 1970 to 1990 (Personal Computers)
generation lasting from cloud computing
• 1980 to 2000 (Portable Computers)
10 to 20 years. with clusters, MPPs,
• 1990 to 2010 (???????)
•Successive generations • 2000 to 2020 (????????)
P2P networks,
are overlapped in about grids, clouds, web
10 years. services, and the
Internet of Things.

17 18

High-Performance Computing High-Throughput Computing

•HPC systems emphasize the raw speed
performance. Floating Point Operations Per
Second (FLOPS).
•Gflops in the early 1990s to now Pflops in 2010.
Floating Point Operations Per Second (FLOPS).
•This improvement was driven mainly by the
demands from scientific, engineering, and
manufacturing communities.
• The development of market-oriented high-end
computing systems is undergoing a strategic change
from an HPC paradigm to an HTC paradigm.
• This HTC paradigm pays more attention to High-Flux
computing.
• High-Flux computing is in Internet searches and web
services.
• The performance goal thus shifts to measure high
throughput or the number of tasks completed per unit
of time.

19 20

Three New Computing Paradigms
•Advances in virtualization make it possible
to see the growth of Internet clouds as a
new computing paradigm.
•Service-Oriented Architectures (SOA),
Virtualization, Cloud Computing, IoT with
CPS (Cyber-Physical Systems)
21
Centralized Computing
•This is a computing paradigm by which all
computer resources are centralized in one
physical system.
•All resources (processors, memory, and storage)
are fully shared and tightly coupled within one
integrated OS.
•Many data centers and supercomputers are
centralized systems.
23
14/10/2021
Computing Paradigm Distinctions
• The high-technology community has argued for many
years about the precise definitions of centralized
computing, parallel computing, distributed computing,
and cloud computing.
• Distributed Computing is the opposite of Centralized
Computing.
• Parallel Computing overlaps with Distributed Computing.
• Cloud Computing overlaps with Distributed, Centralized,
and Parallel Computing.
22
Parallel Computing
•In parallel computing, all processors are either
tightly coupled with centralized shared memory
or loosely coupled with distributed memory.
•Interprocessor Communication is accomplished
through shared memory or via message passing.
Parallel Computer, Parallel Programs, Parallel
Programming.
24
 14/10/2021

Distributed Computing Cloud Computing

•Its consists of multiple autonomous computers,
each having its own private memory,
communicating through a computer network.
•Information exchange in a distributed system is
accomplished through message passing.
•Distributed Program, Distributed Programming.
•An Internet cloud of resources can be either a
centralized or a distributed computing system.
•Clouds can be built with physical or virtualized
resources over large data centers that are
centralized or distributed.
•Some authors consider cloud computing to be a
form of utility computing or service computing.

25 26

Query Regarding Lecture Thanks

Email: mnaseem@ssuet.edu.pk Email: mnaseem@ssuet.edu.pk

27 28