Distributed Systems: Silberschatz, Galvin and Gagne ©2013 Operating System Concepts - 9 Edition

Distributed Systems
Operating System Concepts – 9th Edition Silberschatz, Galvin and Gagne ©2013
Distributed Systems
 Advantages of Distributed Systems
 Types of Network-Based Operating Systems
 Network Structure
 Communication Structure
 Communication Protocols
 An Example: TCP/IP
 Robustness
 Design Issues
Operating System Concepts – 9th Edition 17.2 Silberschatz, Galvin and Gagne ©2013
Overview of Distributed Systems
n Distributed system is collection of loosely coupled processors
interconnected by a communications network
n Processors variously called nodes, computers, machines, hosts
l Site is location of the processor
l Generally a server has a resource a client node at a different site
wants to use
Example
Reasons for Distributed Systems
 Reasons for distributed systems

 Resource sharing -
 Sharing and printing files at remote sites
 Processing information in a distributed database
 Using remote specialized hardware devices
 Computation speedup -
 Job migration: Computations can be divided in smaller sub
computations and they are distributed among various sites
to run concurrently.
 Load sharing: Some of the job from heavily loaded systems
can be moved to lightly loaded system.
 Reliability-
 detect and recover from site failure, function transfer,
reintegrate failed site
Reasons for Distributed Systems
 Communication – message passing

 Various functions can be carried out over long distances. Two
people at geographically distant sites can collaborate on a
project.
 By transferring the files of the project, logging in to each other's
remote systems to run programs, and exchanging mail to
coordinate the work
 Computers can be downsized by moving from large system to

multiple smaller systems performing distributed computing.
Types of Network-based Operating
Systems
 Network Operating Systems
 Distributed Operating Systems
Network-Operating Systems
 Users are aware of multiplicity of machines.
 Access to resources of various machines is done explicitly by:
 Remote logging into the appropriate remote machine (telnet)
 User wishes to compute on "cs.yale.edu," a computer that is located
at Yale University.
telnet cs.yale.edu
 Remote File Transfer: (FTP)
 Suppose that a user on "cs.uvm.edu" wants to copy a Java program
Server. java that resides on "cs.yale.edu." The user must first invoke
the FTP program by executing
ftp cs.yale.edu
 Users must establish a session, give network-based commands
 More difficult for users
Distributed-Operating Systems
Distributed-Operating Systems
 Users not aware of multiplicity of machines

 Access to remote resources similar to access to local
resources
 Data Migration – transfer data by transferring entire file, or
transferring only those portions of the file necessary for the
immediate task
 Computation Migration – transfer the computation, rather
than the data, across the system
 Via remote procedure calls (RPCs)
 or via messaging system
Distributed-Operating Systems (Cont.)
 Process Migration – execute an entire process, or parts of it, at
different sites
 Load balancing – distribute processes across network to
even the workload
 Computation speedup – subprocesses can run concurrently
on different sites
 Hardware preference – process execution may require
specialized processor
 Software preference – required software may be available at
only a particular site
 Data access – run process remotely, rather than transfer all
data locally
 Consider the World Wide Web
Network Structure
 Local-Area Network (LAN) – designed to cover small geographical
area ( Office, Single building, and adjacent buildings)
 All the sites in such systems are close to one another, so the
communication links tend to have a higher speed and lower error
rate than do their counterparts in wide-area networks.
 Speeds from 1Mb per second (Appletalk, bluetooth) to 40 Gbps for
fastest Ethernet over twisted pair copper or optical fibre (10BaseT,
100BaseT).
 Consists of multiple computers (mainframes through mobile

devices), peripherals (printers, storage arrays), routers (specialized
network communication processors) providing access to other
networks
Local-area Network
Network Types (Cont.)
 Wide-Area Network (WAN) –
 The sites in a WAN are physically distributed over a large
geographical area, the communication links are, by default,
relatively slow and unreliable.
 Typical links are telephone lines, leased (dedicated data) lines,
microwave links, and satellite channels.
 This are Implemented via connection processors known as
routers
 Internet WAN enables world wide hosts to communicate
 Hosts differ in all dimensions but WAN allows
communications amongst them
 Connections between networks frequently use a telephone-
system service called T1
 T1 link is 1.544 Megabits per second
 T3 is 28 x T1s = 45 Mbps
Communication Processors in a Wide-Area Network
Network Topology
The sites in a distributed system can be connected physically

in a variety of ways. We can compare the configurations by
using the following criteria:
 Installation cost:
 The cost of physically linking the sites in the system
 Communication cost:
 The cost in time and money to send a message from site A to
site B
 Availability:
 The extent to which data can be accessed despite the failure
of some links or sites
Network Topology
Network Topology
 In a fully connected network, each site is directly connected to every

other site. However, the number of links grows as the square of the number
of sites, resulting in a huge installation cost. Therefore, fully connected
networks are impractical in any large system.
 In a partially connected networks direct links exist between some-but not

all-pairs of sites. Hence, the installation cost of such a configuration is
lower than that of the fully connected network.
 If two sites A and B are not directly connected, messages from one to
the other must be through a sequence of communication links. This
requirement results in a higher communication cost.
Network Topology
 Installation and communication costs are relatively low for a tree-

structured network. However, the failure of a single link in such a network
can result in the network's becoming partitioned.
 In a ring network, at least two links must fail for partition to occur. Thus,
the ring network has a higher degree of availability than does a tree-
structured network. However, the communication cost is high, since a
message may have to cross a large number of links.
 In a star network, the failure of a single link results in a network partition,

but one of the partitions has only a single site. Such a partition can be
treated as a single-site failure. The star network also has a low
communication cost, since each site is at most two links away from every
other site.
Communication Structure
The design of a communication network must address four basic
issues:
 Naming and name resolution - How do two processes
locate each other to communicate?
 Routing strategies - How are messages sent through the
network?
 Connection strategies - How do two processes send a
sequence of messages?
 Contention - The network is a shared resource, so how do
we resolve conflicting demands for its use?
Naming and Name Resolution
 For a process at site A to exchange information with a process at site
B, each must be able to specify the other.
 Within a computer system, each process has a process identifier, and

messages may be addressed with the process identifier.
 Identify processes on remote systems by

<host-name, identifier> pair
 where host name is a name unique within the network and
identifier may be a process identifier or other unique number
within that host.
 A host name is usually an alphanumeric identifier, rather than a

number, to make it easier for users to specify. For instance, site A
might have hosts named homer, marge, bart, and lisa easier to
remember than is 12814831100
 There must be a mechanism to resolve the host name into a host-id
that describes the destination system to the networking hardware.
 First, every host may have a data file containing the names and
addresses of all the other hosts reachable on the network (similar to
binding at compile time).
 The problem with this model is that adding or removing a host from
the network requires updating the data files on all the hosts.
 The alternative is to distribute the information among systems on

the network. (This scheme is like execution-time binding.)
 The network must then use a protocol to distribute and retrieve the
information.
 Domain name system (DNS) – specifies the naming structure of the

hosts, as well as name to address resolution (Internet)
 The systems resolves the addresses by examining the host-name
components in reverse order. Each component has a name-server
(accepts name and returns address) Request by process on Site A to
communicate with bob.cs.brown.edu is resolved as:
1. The kernel of system A issues a request to the name server for
the edu domain, asking for the address of the name server for
brown.edu. The name server for the edu domain must be at a
known address, so that it can be queried.
2. The edu name server returns the address of the host on which the
brown.edu name server resides.
3. The kernel on system A then queries the name server at this
address and asks about cs.brown.edu.
4. An address is returned; and a request to that address for
bob.cs.brown.edu now, finally, returns an host-id for that host
(for example, 128.148.31.100).
Routing Strategies
 When a process at site A wants to communicate with a process at site
B, how is the message sent?
 If there is only one physical path from A to B (a star or tree-
structured network), the message must be sent through that path.
However, if there are multiple physical paths from A to B, then
several routing options exist.
 Fixed routing - A path from A to B is specified in advance; path

changes only if a hardware failure disables it
 Since the shortest path is usually chosen, communication costs are
minimized
 Fixed routing cannot adapt to load changes
 Ensures that messages will be delivered in the order in which they
were sent
Routing Strategies (Cont.)
 Virtual routing- A path from A to B is fixed for the duration of one
session. Different sessions involving messages from A to B may have
different paths
 Partial remedy to adapting to load changes
 Ensures that messages will be delivered in the order in which they
were sent
 Dynamic routing - The path used to send a message form site A to site
B is chosen only when a message is sent
 Because the decision is made dynamically, separate messages may
be assigned different paths.
 Site A will make a decision to send the message to site C; C, in turn,
will decide to send it to site D, and so on. Eventually, a site will deliver
the message to B. Usually, a site sends a message to another site on
whatever link is the least used at that particular time.
 Messages may arrive out of order
 This problem can be remedied by appending a sequence number
to each message
 Most complex to set up
 Tradeoffs mean all methods are used
 UNIX provides ability to mix fixed and dynamic
 Hosts may have fixed routes and gateways connecting
networks together may have dynamic routes
 Router is communications processor responsible for routing

messages
 Must have at least 2 network connections
 Maybe special purpose or just function running on host
 Checks its tables to determine where destination host is, where to
send messages
 Static routing – table only changed manually
 Dynamic routing – table changed via routing protocol
 More recently, routing managed by intelligent software more
intelligently than routing protocols
 OpenFlow is device-independent, allowing developers to
introduce network efficiencies by decoupling data-routing
decisions from underlying network devices
 Messages vary in length – simplified design breaks them into
packets (or frames, or datagrams)
 Connectionless message is just one packet
 Otherwise need a connection to get a multi-packet message
from source to destination
Connection Strategies
 Circuit switching -
 If two processes want to communicate, a permanent physical link is
established between them.
 The is link is allocated for the duration of the communication
session, and no other process can use that link during this period
 This scheme is similar to that used in the telephone system.
 Message switching –
 A temporary link is established for the duration of one message
transfer.
 Physical links are allocated dynamically among correspondents as
needed and are allocated for only short periods.
 This scheme is similar to the post-office mailing system. Each letter
is a message that contains both the destination address and source
(return) address. Many messages can be shipped over the same
link.
Connection Strategies
 Packet switching - Messages of variable length are divided into fixed-
length packets which are sent to the destination
 Each packet may take a different path through the network
 The packets must be reassembled into messages as they arrive
 Circuit switching requires setup time, but incurs less overhead for
shipping each message, and may waste network bandwidth
 Message and packet switching require less setup time, but incur more
overhead per message
Contention
 Depending on the network topology, a link may connect more than two
sites in the computer network, and several of these sites may want to
transmit information over a link simultaneously (Ring).
 Several techniques have been developed to avoid repeated collisions,

including collision detection and token passing.
 CSMA/CD(Carrier Sense with Multiple Access/ Collision Detection):

 A site must listen to determine whether another message is currently
being transmitted over that link.
 If two or more sites begin transmitting at exactly the same time (each
thinking that no other site is using the link), then they will register a
CD.
 One strategy for limiting the number of collisions is to limit the
number of hosts per Ethernet network.
Contention
 Token passing (Ring):
 A site that wants to transmit information must wait until the token
arrives.
 When the site completes its round of message passing, it retransmits

the token.
 If the token gets lost, the system must detect the loss and generate a
new token. It usually does that by declaring an to choose a unique
site where a new token will be generated.
Communication Protocol
The Open System Interconnection (OSI) model defines a networking

framework to implement protocols in seven layers.
 Layers 1-4 are considered as the lower layers (with moving data around)
 Layers 5-7, called the upper layers, contain application-level data.

Networks operate on one basic principle: “pass it on”
 Layer 1: Physical layer – handles the mechanical and electrical details of
the physical transmission of a bit stream. It provides the hardware means
of sending and receiving data on a carrier, including defining cables, cards
and physical aspects.
 Ex: Fast Ethernet, RS232 are protocols with physical layer
components.
 Layer 2: Data-link layer – handles the frames, or fixed-length parts of

packets, including any error detection and recovery that occurred in the
physical layer.
 Media Access Control (MAC) layer: how a computer on the network
gains access to the data and permission to transmit it
 Logical Link Control (LLC) layer: frame synchronization, flow control
and error checking.
Communication Protocol (Cont.)
 Layer 3: Network layer –
 provides connections and routes packets in the communication network,
including handling the address of outgoing packets, decoding the address
of incoming packets, and maintaining routing information for proper
response to changing load levels.
 Layer 4: Transport layer –

 provides transparent transfer of data between end systems, or hosts, and
 It is also responsible for end-to-end error recovery and flow control.
 It ensures complete data transfer.
 Layer 5: Session layer –

 The session layer sets up, coordinates, and terminates conversations,
exchanges, and dialogues between the applications at each end.
Communication Protocol (Cont.)
 Layer 6: Presentation layer –
 The presentation layer works to transform data into the form that
the application layer can accept.
 Resolves the differences in formats among the various sites in
the network, including character conversions, and half
duplex/full duplex (echoing).
 Layer 7: Application layer

 Communication partners are identified, user authentication and
privacy are considered, deals with file transfer, remote-login
protocols and any constraints on data syntax.
Communication Via ISO Network Model
The ISO Protocol Layer
The ISO Network Message
The TCP/IP Protocol Layers
Example: TCP/IP
 The transmission of a network packet between hosts on an
Ethernet network
 Every host has a unique IP address and a corresponding
Ethernet Media Access Control (MAC) address
 Communication requires both addresses
 Domain Name Service (DNS) can be used to acquire IP addresses
 If the hosts are on different networks, the sending host will send
the packet to a router which routes the packet to the destination
network
 If the hosts are on the same network, ARP can be used
Example: TCP/IP
Example: TCP/IP
Robustness
 Failure detection
 The failure of a link,
 The failure of a site, and
 The loss of a message
To ensure that the robustness of the system, these failures must
be detected.
The systems should be reconfigured so that the computation can
continue, and recover when a site or a link is repaired.
 Reconfiguration
Failure Detection
 Detecting these failure is difficult due to no shared memory concept
 To detect a link failure, a heartbeat (Handshaking) protocol can be used
 Assume Site A and Site B have established a link

 At fixed intervals, each site will exchange an I-am-up message
indicating that they are up and running
 If Site A does not receive a message within the predetermined time period,
then it can assume that
 the site B has failed (site failure),
 the link between A and B has failed (link failure),
 the message from B has been lost (message loss).
 Site A can now send an Are-you-up? message to Site B
Failure Detection
 A has two choices:

 It can wait for another time period to receive an I-am-up
message from B, or
 it can send an Are-you-up? message to B.
 If Site A does not receive a reply, it can repeat the message or try
an alternate route to Site B
Failure Detection (Cont.)
 Site A can try to differentiate between link failure and site failure by sending
an Are-you-up? message to B by another route.
 At the time A sends the Are-you-up? message, it specifies a time interval

during which it is willing to wait for the reply from B.
 If Site A does not ultimately receive a reply from Site B, it concludes some
type of failure has occurred.
 Site B is down
 The direct link between A and B is down
 The alternate link from A to B is down
 The message has been lost
 However, Site A cannot determine exactly why the failure has occurred
Reconfiguration
 When Site A determines a failure has occurred, it must reconfigure the
system:
1. If the link from A to B has failed, this must be broadcast to every

site in the system (Routing tables are updated)
2. If a site has failed, every other site must also be notified indicating
that the services offered by the failed site are no longer available.
The failure of a site that serves as a central coordinator for some
activity (such as deadlock detection) requires the election of a new
coordinator.
 When the link or the site becomes available again, this information
must again be broadcast to all other sites
Recovery from Failure
 When the link or the site becomes available again, this information must
again be broadcast to all other sites
 Suppose that a link between A and B has failed. When it is repaired,

both A and B must be notified by continuously repeating the
handshaking procedure.
 Suppose that site B has failed. When it recovers, it must notify all
other sites that it is up again. It may have to receive information from
the other sites to update its local tables; for example, it may need
routing-table information, a list of sites that are down, or undelivered
messages.
Fault Tolerance
 Fault tolerance is the dynamic method that’s used to keep the interconnected
systems together, sustain reliability, and availability in distributed systems.
 A distributed system must tolerate a certain level of failure and continue to

function normally when faced with various types of failures.
 A system failure without redundancy can cause an application or an entire

facility to stop operation.
Types:
 Hardware Fault Tolerance:
 This involves the provision of supplementary backup hardware such as;
CPU, Memory, Hard disks, Power Supply Units, etc.
 Software Fault Tolerance:
 This consists of checkpoints storage and rollback recovery.
 Checkpoints are the snapshot of the entire system in a working state. The
snapshot holds all information to restart the program from the checkpoint
Fault Tolerance
 Replication Based Fault Tolerance Technique

 This technique actually replicate the data on different other system
 A critical process can be simultaneously executed on two nodes so that if
one of the two nodes fails, the execution of the process can be completed
at the other node.
 fault tolerance can be difficult and expensive to implement.

 At the network layer, multiple redundant communication paths and
network devices such as switches and routers are needed to avoid a
communication failure.
 With redundancy techniques additional system overhead is needed to
maintain two or more copies of a replicated resource and to keep all the
copies of a resource consistent.
Fault Tolerance
 In general, the larger is the number of copies kept, the better is the reliability
but the larger is the system overhead involved.
 Therefore, a distributed operating system must be designed to maintain a

proper balance between the degree of reliability and the incurred overhead.
 How much replication is enough?
 The system is said to be k-fault tolerant, if it can continue to function even in

the event of the failure of k components
 Therefore, if the system is to be designed to tolerate k fail-stop failures,

k+1replicas are needed.
Veritas cluster
Design Issues
 Making the multiplicity of processors and storage devices transparent to the

users has been a key challenge to many designers.
 Transparency – the distributed system should appear as a conventional,

centralized system to the user
 Making processors and storage devices transparent to the users is a key
challenge to many designers (should not distinguish between local
and remote resources).
 User Mobility:
 allow users to log into any machine
 bringing over the user's environment (for example, home directory) to
wherever he logs in
 Fault tolerance – the distributed system should continue to function in the

face of failure
Design Issues
 Scalability –
 Systems have bounded resources and can become completely saturated
under increased load.
 as demands increase, the system should easily accept the addition of
new resources to accommodate the increased demand
 adding machines to a distributed system can clog the network and
increase service loads and can call for expensive design modifications.
 In a distributed system, the ability to scale up is of special importance,
since expanding the network by adding new machines or interconnecting
two networks is commonplace.
 Scalability is related to fault tolerance. A heavily loaded component can
become paralyzed and behave like a faulty component.
 Generally, having spare resources is essential for ensuring reliability as
well as for handling peak loads gracefully.
 An inherent advantage of a distributed system is a potential for fault
tolerance and scalability because of the multiplicity of resources.
Design Issues
 Deciding on the process structure of the server is a major problem in the

design of any service.
 Servers are supposed to operate efficiently in peak periods, when hundreds

of active clients need to be served simultaneously.
 A single-process server is certainly not a good choice, since whenever a

request necessitates disk I/0, the whole service will be blocked.
End of Module 1
Operating System Concepts – 9th Edition Silberschatz, Galvin and Gagne ©2013

Distributed Systems: Silberschatz, Galvin and Gagne ©2013 Operating System Concepts - 9 Edition

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Distributed Systems: Silberschatz, Galvin and Gagne ©2013 Operating System Concepts - 9 Edition

Uploaded by

Copyright:

Available Formats

Distributed Systems

 Reasons for distributed systems

 Communication – message passing

 Computers can be downsized by moving from large system to

 Network Operating Systems

 Distributed Operating Systems

 Users not aware of multiplicity of machines

 Consists of multiple computers (mainframes through mobile

The sites in a distributed system can be connected physically

 In a fully connected network, each site is directly connected to every

 In a partially connected networks direct links exist between some-but not

 Installation and communication costs are relatively low for a tree-

 In a star network, the failure of a single link results in a network partition,

 Within a computer system, each process has a process identifier, and

 Identify processes on remote systems by

 A host name is usually an alphanumeric identifier, rather than a

 The alternative is to distribute the information among systems on

 Domain name system (DNS) – specifies the naming structure of the

 Fixed routing - A path from A to B is specified in advance; path

 Router is communications processor responsible for routing

 Several techniques have been developed to avoid repeated collisions,

 CSMA/CD(Carrier Sense with Multiple Access/ Collision Detection):

 When the site completes its round of message passing, it retransmits

The Open System Interconnection (OSI) model defines a networking

 Layers 5-7, called the upper layers, contain application-level data.

 Layer 2: Data-link layer – handles the frames, or fixed-length parts of

 Layer 4: Transport layer –

 Layer 5: Session layer –

 Layer 7: Application layer

 If the hosts are on the same network, ARP can be used

 To detect a link failure, a heartbeat (Handshaking) protocol can be used

 Assume Site A and Site B have established a link

 Site A can now send an Are-you-up? message to Site B

 A has two choices:

 At the time A sends the Are-you-up? message, it specifies a time interval

1. If the link from A to B has failed, this must be broadcast to every

 Suppose that a link between A and B has failed. When it is repaired,

 A distributed system must tolerate a certain level of failure and continue to

 A system failure without redundancy can cause an application or an entire

 Replication Based Fault Tolerance Technique

 fault tolerance can be difficult and expensive to implement.

 Therefore, a distributed operating system must be designed to maintain a

 How much replication is enough?

 The system is said to be k-fault tolerant, if it can continue to function even in

 Therefore, if the system is to be designed to tolerate k fail-stop failures,

 Making the multiplicity of processors and storage devices transparent to the

 Transparency – the distributed system should appear as a conventional,

 Fault tolerance – the distributed system should continue to function in the

 Deciding on the process structure of the server is a major problem in the

 Servers are supposed to operate efficiently in peak periods, when hundreds

 A single-process server is certainly not a good choice, since whenever a

You might also like