PARALLEL

COMPUTING
TM2033 - PLATFORM TECHONOLOGY

CONTENT

• What are Parallel architectures?

• What is cluster computing?
• What is grid computing?
• What are differences between cluster, grid
and cloud?
• Discussion

DR HADI
FTSM, UKM
Introduction

• One of the characteristics of cloud computing is exploiting the current

technologies such as grid computing.
• Is cloud computing is evolution of grid computing?
• How about cluster?

FTSM, UKM
Introduction

All these three technologies, cloud, grid and cluster are under high-perfor
mance computing (HPC) where the main idea is to meet the challenge of
solving difficult problems that require massive amount of computing data
This is because many applications require HPC for example:
1) numeric and scientific application
2) modelling, simulation and complex process system like climate,
3) Business and internet applications like web servers (Google and
Yahoo) and e–commerce (Amazon).

FTSM, UKM
Introduction

• However, dedicated parallel computers or supercomputer are very exp

ensive and not easily extendable.
• Therefore, cost effective solutions are cluster, grid and cloud.
• In this, topic we try to understand the main characteristics of these
three technologies so that we can select which technology is suitable
to solve our computing problems
• These technologies are based on parallel architectures

FTSM, UKM
Single Architecture

Architecture for system with parallel processing capability must have

more than one processing units.
If the system only have one processing unit, so this system doesn’t have
parallel processing capability.
The category for this system based on Flynn taxonomy is SISD (Single
instruction, single data stream – SISD)

FTSM, UKM
Parallel Architecture

Single instruction, multiple data

stream – SIMD
Multiple instructions, multiple data
stream- MIMD
• However, for SIMD, there is
only one single stream of
instruction from a control unit.

loosely-coupled architecture
FTSM, UKM
SMP
• Processors are connected by a bus or
other internal connection
• Memory access time is approximately the
same for each processor
• All processors share access to I/O
• Either through same channels or different
channels giving paths to same devices
• All processors can perform the same
functions (hence symmetric)
• System controlled by integrated operating
system providing interaction between
processors SMP FTSM, UKM
Cluster
The architecture for MIMD can be loosely coupled if there is distributed
memory

Example:
Cluster!

FTSM, UKM
Parallel Architecture

FTSM, UKM
Cluster Computing
Why now cluster computing is one of the hottest technologies?

Data
using HPC
Intensity

Powerful
networking capability
PC

Expandable compare to SMP

FTSM, UKM
Cluster Computing
• collection of parallel or distributed computers
• interconnected among themselves using high-speed networks, such as
gigabit Ethernet.
• They work together in the execution of compute intensive and data inte
nsive tasks that would be not feasible to execute on a single computer.

No Shared Disk Shared Disk

FTSM, UKM
Cluster vs SMP

SMP
•Easier to manage
SMP •Closer to Single Processor
•Less physical
•low power consumption

Clustering
•Superior incremental
Cluster •scalable
•Availability
•redundancy

FTSM, UKM
Grid computing
Grid computing - the collection of computer resources from multiple
locations to reach a common goal.
Like a distributed system with non-interactive workloads that involve a
large number of files.

FTSM, UKM
Grid computing

Grid computing vs HPC systems (cluster com

puting)
• grids tend to be more loosely coupled,
heterogeneous, and geographically
dispersed.
• A single grid can be dedicated to a
particular application
• Commonly a grid is used for a variety of
purposes.
• Grids are often constructed with general-
purpose grid middleware software libraries

FTSM, UKM
 Grid in Malaysia
• National Grid Computing Initiatives (NGCI)
– Launched as KnowledgeGRID Malaysia (KGM) on
20 August 2007.
– Initiative under MOSTI
– To sustain R&D collaboration amongst the Malaysian research
communities through Grid Computing in order to realize the
National IT Service Utility Grid.
• Plan was documented in two documents.
– National Grid Computing Initiative (NGCI) Terms of Reference
(TOR) drafted in December 2005 with several revisions in 2006/7/9
– The National Technology Roadmap for Grid Computing
• One virtual Organization – Academic Grid Malaysia

FTSM, UKM
 Academic Grid Malaysia
• Open access Distributed Computing Infrastructure
• Open to any Malaysian Academia
• Institutions and Organizations that are recognized by respective ministries
to be involved or associated with Distributed, Grid, Cloud and Emerging
Computing Applications/Projects.
• More than 400 core of CPUs in total.

• Currently there are 5 resource contributors:

• UPM - MY-UPM-BIRUNI-01
• UM - MY-UM-CRYSTAL
• UTM MY - UTM-GRID
• UUM - MY-UUM-GRID
• Nuclear Malaysia - MY-NM-GRID
• Secretariat
• UNITEN
• http://academicgrid.my
• secretariat@academicgrid.my

FTSM, UKM
Academic Grid Malaysia

FTSM, UKM
Grid Computation Problem

SPSD • Sequential application

• High CPU and Memory

SPMD
•Sequential Application SPSD - Single Process
•Data can be divided Single Data
• Speed up factor is liner
SPMD - Single Process
Multiple Data

MPSD
• Partial parallel process MPSD - Multiple Process
• Data cannot be divided Single Data
• Speed-up factor varies
MPMD - Multiple Process
Multiple Data

MPMD
• Partial parallel process
• data can be divided
• Speed up factor varies
FTSM, UKM
Case Study: Animation Rendering
• Animation rendering is basically a process to combine
many still image (called frame) as one to became an
animation.
• Example: A rendering project has 100 frames to be
processed.
• Let’s say to process 1 frame it takes 5 minutes wall
time.
• Hence it will takes about 500 minutes to complete
render the frames if we use single cpu core.

FTSM, UKM
Case Study: Animation Rendering
• Now, if we use SPMD model and run it at Grid;
• We split the frames to 10 data chunks
• Each cpu process 1 data chunks (i.e cpu 1 process frame 1-10,
cpu 2 process frame 11-20, etc)
• Hence it only takes 50 minutes (10X faster) to complete the
render process since it runs simultaneously.

FTSM, UKM
 CLOUD
COMPUTING
TM2033 - PLATFORM TECHONOLOGY

CONTENT

• What is the definition of cloud computing?

• What are the deployment models?
• What are the service models?
• What are the essential characteristics of cloud
computing
• What are technologies that can create shared
pool of resources?
• What is virtualization?

DR. ADI
FTSM, UKM
Introduction
Question: Identify keywords from this advertisement:

FTSM, UKM
Introduction

* The keywords from this advertisement are subscription, service,

virtually anywhere and any device.
* This also shows a trend from product-based to service-based.
* Microsoft which is one of the driving force companies for cloud
computing might have thought this question:

*How do we provide services to

customers or end users?
FTSM, UKM
Definition
• Cloud computing is about providing computing services via virtualized
computing resources.
• Office 365 is an example of software based service. Cloud computing
not only involves software, but also hardware, network and platform to
develop and deploy software.
• This lead to a definition by Christian Baun, Marcel Kunze, Jens Nimis,
Stefan Tai in their book Cloud Computing: Web-Based Dynamic IT Services

FTSM, UKM
Cloud Model

• Cloud computing is a model for enabling

ubiquitous, convenient, on demand netwo Deployment
rk access to a shared pool of configurabl
e computing resources.
• (e.g., networks, servers, storage, applicati
ons, and services) that can be rapidly pro Service
visioned and released with minimal mana
gement effort or service provider interacti
on.
• This cloud model is composed of five ess
ential characteristics, three service model Essential Characteristics
s, and four deployment models.
FTSM, UKM
Deployment Model - Public
The infrastructure is available for public use alternatively for large industry
group and is owned by the organization selling cloud services.

Service providers use the internet to make resources,

such as applications (also known as Software-as-a-
service) and storage, available to the general public,
or on a ‘public cloud.
Examples :
Amazon Elastic Compute Cloud (EC2), IBM’s Blue
Cloud, Sun Cloud, Google AppEngine and Windows
Azure Services Platform.
FTSM, UKM
Deployment Model - Private
The infrastructure is operated for the exclusive use of an organization.
The cloud may be managed by that organization or a third party. Private
cloud may be either on- or off-premises.

FTSM, UKM
Deployment Model - Private

• Private clouds are data center architectures owned by a single

company that provides flexibility, scalability, provisioning, automation
and monitoring. The goal of a private cloud is not sell “as-a-service”
offerings to external customers but instead to gain the benefits of
cloud architecture without giving up the control of maintaining your
own data center.
• Private clouds can be expensive with typically modest economies of
scale. This is usually not an option for the average Small-to-Medium
sized business and is most typically put to use by large enterprises.
Private clouds are driven by concerns around security and
compliance, and keeping assets within the firewall.
FTSM, UKM
Deployment Model - Hybrid
Combines multiple clouds (private, community of public) where those
clouds retain their unique identities, but are bound together as a unit

• By using a Hybrid approach, companies can

maintain control of an internally managed
private cloud while relying on the public clou
d as needed.
• For instance during peak periods individual a
pplications, or portions of applications can be
migrated to the Public Cloud.
• This will also be beneficial during predictable
outages: hurricane warnings, scheduled
maintenance windows
FTSM, UKM
Deployment Model - Community
The cloud has been organized to serve a common function or purpose.

FTSM, UKM
Cloud Computing Model

Deployment

Service

Essential Characteristics

FTSM, UKM
Service Model

<something> as a Service

FTSM, UKM
Service Model - IaaS
Provides virtual: machines, storage, infrastructure and other hardware
assets as resources that clients can provision

Examples of IaaS providers:

• Amazon Elastic Compute Cloud (EC2)
• Eucalyptus
• GoGrid
• RackSpace Cloud

FTSM, UKM
IaaS

• a.k.a Utility Computing

• VM in different OS
• Used admin VMs
• Run any applications
• Billing on CPU hour,
storage and network
traffic

FTSM, UKM
Service Model - PaaS
Provides virtual machines, operating systems, applications, services,
development frameworks, transactions and control structures

Examples of PaaS providers:

- Google AppEngine
- Windows Azure Platform
- Force.com

FTSM, UKM
PaaS

FTSM, UKM
Service Model - SaaS
A complete operating environment with applications, management and
user interface.

Examples of SaaS:
Quickbooks : online accounting package
Google Apps

FTSM, UKM
Cloud Computing Model

Deployment

Service

Essential Characteristics

FTSM, UKM
Essential Characteristics
Technologies that can create shared pool of resources
1. Load balancing
- To creating high performance cloud-based
solution
- One characteristic of cloud computing is
virtualized network access to a service.
- No matter where you access the service,
you are directed to the available resources.
- The technology used to distribute service
requests to resources is referred to as load
balancing.
- Load balancing can be implemented in hard http://youtu.be/c7W40_bn3Ss
ware and software.
FTSM, UKM
Essential Characteristics
Technologies that can create shared pool of resources
2. Virtualization
- abstract all the physical resources in order to
create virtual system so that we can see one
big computing resource.
- Therefore, for abstraction, we need a layer call
virtualization layer where we can create a
virtual machine.
- In cloud computing, all the services can be pro
vided via this virtualised platform/ resources.
- creating virtual hardware system
http://youtu.be/7a0kCODjR7s
- Own address space in memory, its own proces
sor resource allocation and its own device I/O
using its own virtual device driver.
FTSM, UKM
Virtualization
In order to enable virtualization, a hypervisor is needed.
Hypervisor is a low level program that c
an provide system resource access to
VM.
This program is also called Virtual
Machine Monitor (VMM).
The operating system loaded into a
virtual machine is referred as guest OS
.

FTSM, UKM
Virtualization
In order to enable virtualization, a hypervisor is needed.
• The host for this VM can be either the
physical hardware or the operating system
installed on the physical hardware.
• If there is no OS installed in the physical
hardware, the VM can be directly access to
the physical hardware via hypervisor. This
is type 1 hypervisor and have full
virtualization.
• But if there is OS installed on hardware, the
VM have to work with this host OS. This is
based on para virtualization and called
type two hypervisor.
FTSM, UKM
Cloud Computing Model

Deployment

Service

Essential Characteristics

FTSM, UKM
 END OF
SYLLABUS
TM2033 - PLATFORM TECHONOLOGY

DR HADI
FTSM, UKM