CLOUD COMPUTING

Grid Architecture and Tools

Dr. Adeel Asghar

Courtesy:
NUML RWP
Dr Gnanasekaran
adeel.asghar@numl.edu.pk Thangavel
 UNIT 2 OVERVIEW

Grid Architecture and Tools:

– The Hourglass model
– Layered Architecture
– Simulation Tools
– Clouds and Grids
– Service Oriented Architecture
Grid Architecture
 The Hourglass Model

• Focus on architecture issues

Applications
– Propose set of core services as basic
Diverse global services
infrastructure
– Used to construct high-level, domain-specific
solutions (diverse)
• Design principles
Core
– Keep participation cost low services
– Enable local control
– Support for adaptation
– “IP hourglass” model

Local OS
• The architecture consists of four layers.

• The lowest fabric layer provides interfaces to

local resources at a specific site. These interfaces
are tailored to allow sharing of resources within a
virtual organization.

• The connectivity layer consists of small, standard

collection of communication protocols for
supporting grid transactions that span the usage
of multiple resources. In addition, the connectivity
layer will contain security protocols to authenticate
users and resources.

• The next layer in the hierarchy is the collective layer. It deals with handling access to multiple
resources and typically consists of services for resource discovery, allocation and scheduling of tasks
onto multiple resources, data replication, and so on.

• The application layer consists of the applications that operate within a virtual organization and which
make use of the grid computing environment.
 Layered Grid Architecture
(By Analogy to Internet Architecture)

Application

Internet Protocol Architecture

“Coordinating multiple resources”: ubiquitous
infrastructure services, app-specifc distributed Collective
services Application
“Sharing single resources”: negotiating access,
controlling use Resource

“Talking to things”: communication (Internet

protocols) & security Connectivity Transport
Internet
“Controlling things locally”: Access to, & control
of, resources Fabric Link
 Example:
Data Grid Architecture
App Discipline-Specifc Data Grid Application

Collective Coherency control, replica selection, task management, virtual data catalog,
(App) virtual data code catalog, …

Collective Replica catalog, replica management, co-allocation, certifcate authorities,

(Generic) metadata catalogs,

Resource Access to data, access to computers, access to network performance data, …

Connect Communication, service discovery (DNS), authentication, authorization,

delegation

Fabric Storage systems, clusters, networks, network caches, …

 Simulation tools

• GridSim – Job scheduling

–http://sourceforge.net/projects/gridsim
• SimGrid – Single client multiserver scheduling
• Bricks – Scheduling
• GangSim- Ganglia VO
• OptoSim – Data Grid Simulations
• G3S – Grid Security Services Simulator – security
services
 Simulation tools
• GridSim is a Java-based toolkit for modeling, and
simulation of distributed resource management and
scheduling for conventional Grid environment.

• GridSim is based on SimJava, a general purpose discrete-

event simulation package implemented in Java.

• All components in GridSim communicate with each other

through message passing operations defined by SimJava.

http://www.gridbus.org/gridsim
http://www.buyya.com/gridsim/
 Salient features of the GridSim
• It allows modeling of heterogeneous types of resources.
• Resources can be modeled operating under space- or time-
shared mode.
• Resource capability can be defined (in the form of MIPS
(Million Instructions Per Second) benchmark.
• Resources can be located in any time zone.
• Weekends and holidays can be mapped depending on
resource’s local time to model non-Grid (local) workload.
• Resources can be booked for advance reservation.
• Applications with different parallel application models can be
simulated.
 Salient features of the GridSim
• Application tasks can be heterogeneous and they can be CPU
or I/O intensive.
• There is no limit on the number of application jobs that can be
submitted to a resource.
• Multiple user entities can submit tasks for execution
simultaneously in the same resource, which may be time-
shared or space-shared. This feature helps in building
schedulers that can use different market-driven economic
models for selecting services competitively.
• Network speed between resources can be specified.
• It supports simulation of both static and dynamic schedulers.
• Statistics of all or selected operations can be recorded and
they can be analyzed using GridSim statistics analysis
methods.
 Cloud: Definition
“A large-scale distributed computing paradigm that is driven by
economies of scale, in which a pool of abstracted, virtualized,
dynamically-scalable, managed computing power, storage,
platforms, and services are delivered on demand to external
customers over the Internet.”

(According to Foster, Zhao, Raicu and Lu, Cloud Computing and Grid
Computing 360-Degree Compared, 2008)
CLOUDS
 Cloud Computing

• Just a new name for Grid?

– Yes…
–…No….
– Nevertheless Yes!!!
Cloud: just a new name for Grid?

• YES:
– Reduce the cost of computing
– Increase reliability
– Increase flexibility (third party)
Cloud: just a new name for Grid?

• NO:
– Great increase demand for computing
(clusters, high speed networks)
– Billions of dollars being spent by Amazon,
Google, Microsoft to create real commercial
large-scale systems with hundreds of
thousands of computers – www.top500.org
shows computers with 100,000+ computers
– Analysis of massive data
Cloud: just a new name for Grid?

• Nevertheless YES:
– Problems are the same in clouds and grids
– Common need to manage large facilities
– Define methods to discover, request and use
resources
– Implement highly parallel computations
Clouds: key points of the definition

• Differences related to traditional

distributed paradigms:
– Massively scalable
– Can be encapsulated as an abstract entity
that delivers different levels of service
– Services can be dynamically configured (via
virtualization or other approaches) and
delivered on demand
 Clouds: reasons for interest

• Rapid decrease in hardware cost, increase

in computing power (multi-cores etc.) and
storage capacity
• Exponentially growing data size
• Widespread adoption of Services
Computing and Web 2.0 apps
 Clouds: yet about definition…
“The interesting thing about Cloud Computing is that
we’ve redefined Cloud Computing to include
everything that we already do. . . . I don’t
understand what we would do differently in the
light of Cloud Computing other than change the
wording of some of our ads.”
Larry Ellison (Oracle CEO), quoted in the Wall Street
Journal, September 26, 2008
 Clouds: yet about definition…
“A lot of people are jumping on the [cloud]
bandwagon, but I have not heard two people say
the same thing about it. There are multiple
definitions out there of “the cloud.””
Andy Isherwood (HP VP of sales), quoted in ZDnet
News, December 11, 2008
 Clouds: yet about definition…
“It’s stupidity. It’s worse than stupidity: it’s a marketing
hype campaign. Somebody is saying this is inevitable
— and whenever you hear somebody saying that, it’s
very likely to be a set of businesses campaigning to
make it true.”
Richard Stallman (known for his advocacy of free
software), quoted in The Guardian, September 29,
2008
 Clouds: yet about definition…
• From a hardware point of view, three aspects are new in
Cloud Computing:
1. The illusion of infinite computing resources available on
demand, thereby eliminating the need for Cloud Computing
users to plan far ahead for provisioning;
2. The elimination of an up-front commitment by Cloud users,
thereby allowing companies to start small and increase
hardware resources only when there is an increase in their
needs; and
3. The ability to pay for use of computing resources on a short-
term basis as needed (e.g., processors by the hour and storage
by the day) and release them as needed, thereby rewarding
conservation by letting machines and storage go when they are
no longer useful.
Clouds: relation with other paradigms
Web 2.0, Clouds, and Internet of Things

HTC: High - Throughput Computing HPC: High - Performance Computing P2P: Peer to Peer
MPP: Massively Parallel Processors
What is a Service Oriented Architecture?
 What is a Service Oriented Architecture (SOA)?

• A method of design, deployment, and management of

both applications and the software infrastructure
where:
– All software is organized into business services that
are network accessible and executable.
– Service interfaces are based on public standards for
interoperability.
 Key Characteristics of SOA
• Quality of service, security and performance are specified.
• Software infrastructure is responsible for managing.
• Services are cataloged and discoverable.
• Data are cataloged and discoverable.
• Protocols use only industry standards.
 What is a “Service”?

• A Service is a reusable component.

• A Service changes business data from one state to
another.
• A Service is the only way how data is accessed.
• If you can describe a component in WSDL, it is a
Service.
 Why Getting SOA Will be Difficult
• Managing for Projects:
– Software: 1 - 4 years
– Hardware: 3 - 5 years;
– Communications: 1 - 3 years;
– Project Managers: 2 - 4 years;
– Reliable funding: 1 - 4 years;
– User turnover: 30%/year;

• Managing for SOA:

– Data: forever.
– Infrastructure: 10+ years.
Why Managing Business Systems is Difficult?

• 40 Million lines of code in Windows XP is unknowable.

• Testing application (3 Million lines) requires >1015 tests.

• Probability correct data entry for a supply item is <65%..

• Output / Office Worker: >30 e-messages /day.

 How to View Organization for SOA

Pr iv a te A p p lic a t io n s a n d File s PERSO N A L LEV EL

Pri v acy an d
I n d i v i d u al
Secu ri ty B arri er VARIETY HERE
G r ap h ic I n fo W in d o w , Pe r so n a l To o ls, I n q u ir y La n gu a ge s C u
sto m iz e d A p p lic a t io n s, Pr o t o t y p in g To o ls, Lo ca l LO C A L LEV EL
A p p lic a tio n s a n d Files
A p p l i cati on s
Secu r i ty B arri er

A p p l ic a t io n s D e ve lo p m e n t & M a in t e n a n c e A PPLI C AT I O N LEV EL

B u si n ess
Secu ri ty B arri er

Ser vice A Service B OS

BU SI N ESS LEV EL
D
P rocess
Secu ri ty B arri er Fu n c t io n a l Pr o c e ss A
Fu n c t io n a l Pr o c e ss B
Fu n c t io n a l Pr o c e ss C PRO C ESS LEV EL
Fu n c t io n a l Pr o c e ss D
C o r p o r a te Po lic y, C o r p o r at e Sta n d a r d s, Refer e n c e M o d e ls, D a
ta M a n a ge m e n t a n d To o ls, I n t e gr a t e d Sy st em s
C o n f igu r a tio n D at a Base , Sh a r e d C o m p u t in g a n d Te EN T ERPRI SE LEV EL
le c o m m u n ic a t io n s
STABILITY HERE
I n d u str y St a n d ar d s, C o m m er c ia l O ff- t h e - Sh elf Pr
o d u c t s an d Ser v ic e s G LO BA L LEV EL
 Organization of Infrastructure Services

Infrastructur
e Services
(Enterprise Information)

Data Security Computing Communication Application

Services Services Services Services Services
 Organization of Data Services

Data
Services

Discovery Management Collaboration Interoperability Semantic

Services Services Services Services Services
 Data Interoperability Policies
• Data is an enterprise resource.

• Single-point entry of unique data.

• Enterprise certification of all data definitions.

• Data stewardship defines data custodians.

• Zero defects at point of entry.

• De-conflict data at source, not at higher levels.

• Data aggregations from sources data, not from reports.

 Organization of Security Services

Security
Services

Transfer Protection Certifca tion Systems Authentication

Services Services Services Assurance Services
 Security Services = Information Assurance
• Conduct Attack/Event Response
• Ensure timely detection and appropriate response to attacks.
• Manage measures required to minimize the network’s
vulnerability.
• Secure Information Exchanges
• Secure information exchanges that occur on the network with a
level of protection that is matched to the risk of compromise.
• Provide Authorization and Non-Repudiation Services
• Identify and confirm a user's authorization to access the network.
 Organization of Computing Services

Computing
Services

Computing Resource Control & Con fguration Financial

Facilities Planning Quality Services Management
 Computing Services
• Provide Adaptable Hosting Environments

– Global facilities for hosting to the “edge”.

– Virtual environments for data centers.

• Distributed Computing Infrastructure

– Data storage, and shared spaces for information sharing.
• Shared Computing Infrastructure Resources
– Access shared resources regardless of physical access device.
 Organization of Communication Services

Communication
Services

Interoperability Spectrum Connectivity Continuity of Resource

Services Management Arrangements Services Management
 Communication Services
• Provide Information Transport

– Transport information, data and services anywhere.

– Ensures transport between end-user devices and servers.
– Expand the infrastructure for on-demand capacity.
 Organization of Application Services

Application
Services

Component Code Binding Maintenance Experimental

Portals
Repository Services Management Services
 Application Services and Tools

• Provide Common End User Interface Tools

–Application generators, test suites, error identification,
application components and standard utilities.
• Common end-user Interface Tools.
–E-mail, collaboration tools, information dashboards, Intranet
portals, etc.
 A Few Key SOA Protocols
• Universal Description, Discovery, and Integration, UDDI defines the
publication and discovery of web service implementations.
• The Web Services Description Language, WSDL, is an XML-based
language that defines Web Services.
• SOAP is the Service Oriented Architecture Protocol. It is a key SOA in
which a network node (the client) sends a request to another node (the
server).
• The Lightweight Directory Access Protocol, or LDAP is protocol for
querying and modifying directory services.
• Extract, Transform, and Load, ETL, is a process of moving data from a
legacy system and loading it into a SOA application.
 References
1. Kai Hwang, Geoffery C. Fox and Jack J. Dongarra, “Distributed and Cloud
Computing: Clusters, Grids, Clouds and the Future of Internet”, First Edition, Morgan
Kaufman Publisher, an Imprint of Elsevier, 2012.
2. Distributed Computing. http://distributedcomputing.info/index.html
3. Jie Wu, Distributed System Design, CRC Press, 1999.
4. Distributed Computing, Wikipedia http://en.wikipedia.org/wiki/Distributed_computing
5. www.psgtech.edu/yrgcc/attach/GridComputing-an%20introduction.ppt
6. www.cse.unr.edu/~mgunes/cpe401/cpe401sp12/lect15_cloud.ppt
7. csnotes.upm.edu.my/kelasmaya/web.nsf/.../$FILE/Distributed%20Computing.ppt
8. www.strassmann.com/pubs/gmu/2007-11-slides.ppt
Assignment #1

• Visit the links for GridSim information on slide 9 to:

• Write a report on GridSim, its main features
• Selected any of the related publications about GridSim
and summarise in your report.
 Thank You

Questions and Comments?