2.

4 Message Oriented Communication

 RPCs and RMIs are not adequate for all distributed system
applications
 the provision of access transparency may be good but
they have semantics that is not adequate for all
applications
 example problems
 they assume that the receiving side is running at the
time of communication
 a client is blocked until its request has been processed

1
2.4.1 Persistence and Synchronicity in Communication
 assume the communication system is organized as a
computer network shown below

general organization of a communication system in which hosts are connected

through a network
2
 communication can be
 persistent or transient
 asynchronous or synchronous
 persistent: a message that has been submitted for
transmission is stored by the communication system as long
as it takes to deliver it to the receiver
 e.g., email delivery, snail mail delivery

persistent communication of letters back in the days of the Pony Express

3
 transient: a message that has been submitted for transmission
is stored by the communication system only as long as the
sending and receiving applications are executing
 asynchronous: a sender continues immediately after it has
submitted its message for transmission
 synchronous: the sender is blocked until its message is
stored in a local buffer at the receiving host or delivered to the
receiver
 the different types of communication can be combined
 persistent asynchronous: e.g., email
 transient asynchronous: e.g., UDP, asynchronous RPC
 in general there are six possibilities

Persistent Transient

Asynchronous  

Synchronous  message-oriented; three forms

4
persistent asynchronous communication persistent synchronous communication

5
transient asynchronous communication receipt-based transient synchronous
communication

 weakest form; the sender is

blocked until the message is
stored in a local buffer at the
receiving host

6
delivery-based transient synchronous response-based transient synchronous
communication at message delivery communication
 the sender is blocked until the  strongest form; the sender is
message is delivered to the blocked until it receives a reply
receiver for further processing; message from the receiver
e.g., asynchronous RPC

7
2.4.2 Message-Oriented Transient Communication
 many applications are built on top of the simple message-
oriented model offered by the transport layer
 standardizing the interface of the transport layer by
providing a set of primitives allows programmers to use
messaging protocols
 they also allow porting applications

1. Berkley Sockets
 an example is the socket interface as used in Berkley
UNIX
 a socket is a communication endpoint to which an
application can write data that are to be sent over the
network, and from which incoming data can be read

8
Primitive Meaning Executed by
Socket Create a new communication endpoint; also
reserve resources to send and receive messages both
Bind Attach a local address to a socket; e.g., IP
address with a known port number
Listen Announce willingness to accept connections; for
connection-oriented communication
Accept Block caller until a connection request arrives
servers
Connect Actively attempt to establish a connection; the
client is blocked until connection is set up
Send Send some data over the connection
Receive Receive some data over the connection
Close Release the connection
socket primitives for TCP/IP

9
connection-oriented communication pattern using sockets

10
2. The Message-Passing Interface (MPI)
 sockets were designed to communicate across networks
using general-purpose protocol stacks such as TCP/IP
 they were not designed for proprietary protocols
developed for high-speed interconnection networks; of
course portability will suffer
 MPI is designed for parallel applications and tailored for
transient communication
 MPI assumes communication takes place within a known
group of processes, where each group is assigned an
identifier (groupID)
 each process within a group is also assigned an identifier
(processID)
 a (groupID, processID) identifies the source or destination
of a message, and is used instead of a transport-level
address

11
Primitive Meaning

MPI_bsend Append outgoing message to a local send buffer; to support

transient asynchronous communication
Send a message and wait until copied to local or remote
MPI_send buffer (to support receipt-based transient synchronous
communication)

MPI_ssend Send a message and wait until receipt starts (to support
delivery-based transient synchronous communication)

MPI_sendrecv Send a message and wait for reply (to support response-
based transient synchronous communication)

MPI_isend Pass reference to outgoing message, and continue (a

variant of MPI_send)

MPI_issend Pass reference to outgoing message, and wait until receipt

starts (a variant of MPI_ssend)
MPI_recv Receive a message; block if there are none
MPI_irecv Check if there is an incoming message, but do not block
some of the most intuitive message-passing primitives of MPI

12
2.4.3 Message-Oriented Persistent Communication
 there are message-oriented middleware services, called
message-queuing systems or Message-Oriented Middleware
(MOM)
 they support persistent asynchronous communication
 they have intermediate-term storage capacity for messages,
without requiring the sender or the receiver to be active
during message transmission
 unlike Berkley sockets and MPI, message transfer may take
minutes instead of seconds or milliseconds
Message-Queuing Model
 applications communicate by inserting messages in
specific queues
 it permits loosely-coupled communication
 the sender may or may not be running; similarly the
receiver may or may not be running, giving four possible
combinations
13
four combinations for loosely-coupled communications using queues
14
Primitive Meaning

Put Append a message to a specified queue; by the sender

and is nonblocking

Get Block until the specified queue is nonempty, and remove

the first message

Poll Check a specified queue for messages, and remove the

first. Never block

Notify Install a handler to be called when a message is put into

the specified queue; usually a daemon
basic interface to a queue in a message-queuing system

15
 General Architecture of a Message-Queuing System
 messages can be put only into queues that are local to the
sender (same machine or on a nearby machine on a LAN)
 such a queue is called the source queue
 messages can also be read only from local queues
 a message put into a local queue must contain the specification
of the destination queue; hence a message-queuing system
must maintain a mapping of queues to network locations; like
in DNS

the relationship between queue-level addressing and network-level addressing 16

 messages are managed by queue managers
 they generally interact with the application that sends and
receives messages
 some also serve as routers or relays, i.e., they forward
incoming messages to other queue managers
 however, each queue manager needs a copy of the queue-
to-location mapping, leading to network management
problems for large-scale queuing systems
 the solution is to use a few routers that know about the
network topology

17
the general organization of a message-queuing system with routers
18
 Message Brokers
 how can applications understand the messages they receive
 each receiver can not be made to understand message formats
of new applications
 hence, in a message-queuing system conversations are
handled by message brokers
 a message broker converts incoming messages to a format
that can be understood by the destination application based on
a set of rules

the general organization of a message broker in a message-queuing system 19

 2.4 Stream Oriented Communication
 until now, we focused on exchanging independent and
complete units of information
 time has no effect on correctness; a system can be slow or fast
 however, there are communications where time has a critical
role
 Multimedia
 media
 storage, transmission, interchange, presentation,
representation and perception of different data types:
 text, graphics, images, voice, audio, video, animation, ...
 movie: video + audio + …
 multimedia: handling of a variety of representation media
 end user pull
 information overload and starvation
 technology push
 emerging technology to integrate media 20
 The Challenge
 new applications
 multimedia will be pervasive in few years (as graphics)
 storage and transmission
 e.g., 2 hours uncompressed HDTV (1920×1080) movie:
1.02 TB (1920×1080x3x25x60x60x2)
 videos are extremely large, even compressed
 continuous delivery
 e.g., 30 frames/s (NTSC), 25 frames/s (PAL) for video
 guaranteed Quality of Service
 admission control
 search
 can we look at 100… videos to find the proper one?

21
 Types of Media
 two types
 discrete media: text, executable code, graphics, images;
temporal relationships between data items are not
fundamental to correctly interpret the data
 continuous media: video, audio, animation; temporal
relationships between data items are fundamental to
correctly interpret the data
 a data stream is a sequence of data units and can be applied
to discrete as well as continuous media
 stream-oriented communication provides facilities for the
exchange of time-dependent information (continuous media)
such as audio and video streams

22
 timing in transmission modes
 asynchronous transmission mode: data items are
transmitted one after the other, but no timing constraints;
e.g. text transfer
 synchronous transmission mode: a maximum end-to-end
delay defined for each data unit; it is possible that data can
be transmitted faster than the maximum delay, but not slower
 isochronous transmission mode: maximum and minimum
end-to-end delay are defined; also called bounded delay
jitter; applicable for distributed multimedia systems
 a continuous data stream can be simple or complex
 simple stream: consists of a single sequence of data; e.g.,
mono audio, video only
 complex stream: consists of several related simple streams
that must be synchronized; e.g., stereo audio, video
consisting of audio and video (may also contain subtitles,
translation to other languages, ...)
23
movie as a set of simple streams

24
 a stream can be considered as a virtual connection between a
source and a sink
 the source or the sink could be a process or a device

setting up a stream between two processes across a network

setting up a stream directly between two devices 25

 the data stream can also be multicasted to several receivers
 if devices and the underlying networks have different
capabilities, the stream may be filtered, generally called
adaptation (filtering?, transcoding?)

an example of multicasting a stream to several receivers

26
 Quality of Service (QoS)
 QoS requirements describe what is needed from the
underlying distributed system and network to ensure
acceptable delivery; e.g. viewing experience of a user
 for continuous data, the concerns are
 timeliness: data must be delivered in time
 volume: the required throughput must be met
 reliability: a given level of loss of data must not be
exceeded
 quality of perception; highly subjective

27
 QoS requirements can be specified using flow specification
containing bandwidth requirements, transmission rates,
delays, ...
 e.g. by Partridge (1992)
 it uses the token bucket algorithm which specifies how the
stream will shape its network traffic (in fact the leaky
bucket, as used in networking)
 the idea is to shape bursty traffic into fixed-rate traffic by
averaging the data rate
 packets may be dropped if the bucket is full
 the input rate may vary, but the output rate remains
constant

28
the principle of a token bucket algorithm

29
  problem in flow specification
 an application may not know its requirements
 how can a user (human) specify quality using the various
parameters? usually very difficult
 may be provide defaults for various streams as high,
medium, low quality
 Setting up a Stream
 resources such as bandwidth, buffers, processing power
must be reserved once a flow specification is made
 on such protocol is RSVP - Resource reSerVation Protocol
 it is a transport-level protocol for enabling resource
reservation in network routers

30
the basic organization of RSVP for resource reservation in a distributed system

31
 Stream Synchronization
 how to maintain temporal relations between streams, e.g., lip
synchronization
 two approaches
1. explicitly by operating on the data units of simple
streams; the responsibility of the application

the principle of explicit synchronization on the level of data units 32

2. through a multimedia middleware that offers a collection of
interfaces for controlling audio and video streams as well as
devices such as monitors, cameras, microphones, ...

the principle of synchronization as supported by high-level interfaces

33