Real-Time Protocols

RTP/RTCP
Overview
History of streaming media
Streaming performance requirements
Protocol stack for multimedia services
Real-time transport protocol(RTP)
RTP control protocol (RTCP)

Brief history of streaming media

Real-time multimedia streaming

Real-time multimedia applications
Video teleconferencing
Internet Telephony (VoIP)
Internet audio, video streaming
(A-PDUs)

Streaming performance requirements

Sequencing
to report PDU loss
to report PDU reordering
to perform out-of-order decoding
Time stamping and Buffering
for play out
for jitter and delay calculation
Payload type identification
for media interpretation
Error concealment covers up errors from lost PDU by using
redundancy in most-adjacent-frame
Quality of Service (QoS) feedback from receiver to sender for
operation adjustment
Rate control sender reduces sending rate adaptively to network
congestion

Ideal Timing no jitter

30 seconds

00.00.00
00.00.10
00.00.20
00.00.30

First RTP-PDU

Second RTP-PDU

application
00.00.11
00.00.21

Third RTP-PDU

00.00.31

Send time
Play time

Reality jitter

delay

00.00.00
00.00.10

First RTP-PDU

00.00.20

Second RTP-PDU

00.00.21

00.00.30
00.00.40

Third RTP
-PDU
Fourth
RTP-P
DU

Send time

00.00.11

00.00.25
00.00.35
00.00.37
00.00.41
00.00.47
00.00.51

Play time

Jitter

(contd.)

00.00.00
00.00.10

First RTP-PDU(0)

00.00.20

Second RTP-PDU(1
0)

00.00.21

00.00.30
00.00.40

Third RTP
-PDU(20)
Fourth
RTP-P
DU

Send time

00.00.11

(30)

00.00.25
00.00.35
00.00.37
00.00.41
00.00.47

00.00.18

00.00.28

00.00.38
00.00.48

00.00.51

Play time

00.00.58

Jitter

(contd.)

Playback buffer
At time 00:00:18

At time 00:00:28

At time 00:00:38

Audio silence example:

Not send anything

Why might this cause problems?

sender

Consider audio data

What should the sender do during silence?

Seq no
.1, Tmp
st 1 0 0
Seq no
.2, Tmp
st 2 0 0
Seq no
.3, Tmp
st 3 0 0

silence

Receiver cannot distinguish between loss

and silence

Solution:

Seq no
.4, Tmp
st 6 0 0
Seq no
.5, Tmp
st 700

After receiving no PDUs for a while, next PDU received at

the receiver will reflect a big jump in timestamp, but
have the correct next seq. no. Thus, receiver knows
what happened.

receiver

How does Sequence number and Timestamp help ?

Streaming performance requirements

Sequencing
to report PDU loss
to report PDU reordering
to perform out-of-order decoding
Time stamping and Buffering
for play out
for jitter and delay calculation
Payload type identification
for media interpretation
Error concealment covers up errors from lost PDU by using
redundancy in most-adjacent-frame
Quality of Service (QoS) feedback from receiver to sender for
operation adjustment
Rate control sender reduces sending rate adaptively to network
congestion

Support from transport layers

TCP is not used because:

TCP does retransmissions unbounded delays

No provision for time stamping
TCP does not support multicast
TCP congestion control (slow-start) unsuitable for real-time transport

RTP + UDP usually used for multimedia services

Protocol stack for multimedia services

RTSP

RTP

RTCP
TCP
(till now)

RTP: Introduction

Provides end-to-end transport functions for real-time

applications
Supports different payload types

All RTP and RTCP PDUs are sent to same multicast group
(by all participants)
All RTP PDUs sent to an even-numbered UDP port, 2p
All RTCP PDUs sent to UDP port 2p+1
Transport
layer

Does NOT provide timely delivery or other QoS guarantees

Relies on other protocols like RTCP and lower layers

Application
RTP RTCP
UDP
IP
Data Link

Does NOT assume the underlying network is reliable and

delivers PDUs in sequence
Uses sequence number

Physical

RTP Session

RTP session is sending and receiving of RTP data by a group

of participants
For each participant, a session is a pair of transport
addresses used to communicate with the group
If multiple media types are communicated by the group, the
transmission of each medium constitutes a session.

RTP Synchronization Source

synchronization source - each source of RTP PDUs

Identified by a unique,randomly chosen 32-bit ID (the SSRC)
A host generating multiple streams within a single RTP must
use a different SSRC per stream

RTP Basics of Data Transmission

RTP PDUs

RTP PDU Header

Sampling instant of first data octet
multiple PDUs can have same timestamp
not necessarily monotonic
used to synchronize different
Payload type
media streams

Incremented by one for

each RTP PDU:
PDU loss detection
Restore PDU sequence

Identifies synchronization source

Identifies contributing sources

(used by mixers)

Mixer
RTP mixer - an intermediate system that receives & combines
RTP PDUs of one or more RTP sessions into a new RTP PDU

Stream may be transcoded, special effects may be performed.

A mixer will typically have to define synchronization relationships between
streams.Thus
Sources that are mixed together become contributing sources (CSRC)
Mixer itself appears as a new source having a new SSRC

Translator
An intermediate system that
Connects two or more networks
Multicasting through a firewall
Modifies stream encoding, changing the streams timing
Transparent to participants
SSRCs remain intact

end system 1
from ES1: SSRC=6
from ES2: SSRC=23

end system 2

transl.1

from ES1: SSRC=6

from ES2: SSRC=23

authorized tunnel
firewall

transl.2
from ES1: SSRC=6
from ES2: SSRC=23

RTP Control Protocol (RTCP)

RTCP specifies report PDUs exchanged between sources and

destinations of multimedia information

receiver reception report
sender report
source description report
Reports contain statistics such as the number of RTP-PDUs sent,

number of RTP-PDUs lost, inter-arrival jitter

Used by application to modify sender transmission rates and for

diagnostics purposes

RTCP message types

Typically, several RTCP PDUs of different types are transmitted

in a single UDP PDU

Sender/Receiver report PDUs

V

RC

PT=200/201 SR/RR

Length (16 bits)

SSRC of Sender

Header

NTP Timestamp, most significant word

NTP Timestamp, least significant word
RTP Timestamp
Senders PDU Count

Sender
Info

Senders Octet Count

SSRC_1 (SSRC of the 1st Source)
Fraction Lost

Cumulative Number of PDU Lost

Extended Highest sequence Number Received

Interarrival Jitter

Report
Block 1

Last SR (LSR)
Delay Since Last SR (DLSR)
SSRC_2 (SSRC of the 2nd Source)

Profile-Specific Extensions

Report
Block 2

Ethereal capture for RTP-PDU

Basic header

Ethereal capture for RTCP-PDU

header of SR report
sender info

receiver report block

SDES items

Synchronization of streams using RTCP

RTP audio
RTCP audio
RTP video
RTP video

Internetwork

Timestamps in RTP PDUs are tied to the individual video and audio sampling clocks
timestamps are not tied to the wall-clock time, or each other!

Each RTCP sender-report PDU contains (for most recently generated PDU in
associated RTP stream):
The timestamp of RTP PDU
The wall-clock time for when PDU was created
Receivers can use this association to synchronize the playout of audio and video

RTCP bandwidth scaling

Problem
What happens when there is
one sender and many receivers?
RTCP reports scale linearly with
the number of participants and
would match or exceed the
amount of RTP data! More
overhead than useful data!
Solution
RTCP attempts to limit its
traffic to 5% of the session
bandwidth to ensure it can
scale!
RTCP gives 75% of this rate
to the receivers; and the
remaining 25% to the
sender.

Example
Suppose one sender, sending
video at a rate of 2 Mbps. Then
RTCP attempts to limit its traffic
to 100 Kbps.
The 75 kbps is equally shared
among receivers:
With R receivers, each
receiver gets to send RTCP
traffic at 75/R kbps.
Sender gets to send RTCP traffic
at 25 kbps.

Real-Time Streaming Protocol (RTSP)

Application layer protocol (default port 554)

Usually runs on RTP for stream & TCP for control
Provides the control channel
Uses out-of-band signaling
Usable for Live broadcasts / multicast

Also known as Network remote control for multi-media

servers.

RTSP Overview

Web Server

web
browser

HTTP
presentation descriptor

Presentation
descriptor

media
player

Web Server/Media server

RTSP
pres. desc,streaming commands

RTP/RTCP
audio/video content

RTSP Methods
OPTIONS

CS
CS

determine capabilities of server/client

DESCRIBE

CS

get description of media stream

ANNOUNCE

CS

announce new session description

SETUP

CS

create media session

RECORD

CS

start media recording

PLAY

CS

start media delivery

PAUSE

CS

pause media delivery

REDIRECT

CS

redirection to another server

TEARDOWN

CS

immediate teardown

SET_PARAMETER

CS

change server/client parameter

GET_PARAMETER

CS

read server/client parameter

RTSP Session

Default port
554

RTSP SETUP
RTSP OK

RTSP
server

RTSP PLAY
RTSP OK
RTSP TEARDOWN
RTSP OK

TCP

RTSP
client

get UDP port

data
source

RTP VIDEO
RTP AUDIO
RTCP

media server

choose
UDP port

UDP

AV
subsyste
m

media player

Example:Media on demand (Unicast)

Media server A
audio.example.com
Media server V
video.example.com

Client C

Web server W
-holds the media descriptors

RTSP Message sequence

C -> W : GET/Twister.sdp HTTP/1.1
Host: www.example.com
Accept: application/sdp
W-> C : HTTP/1.0 200 OK
Content-Type: application/sdp

C-> A : SETUP rtsp://audio.example.com/twister/audio.en RTSP/1.0

Cseq:1
Transport : RTP/AVP/UDP;unicast;client_port=3056-3057
A-> C : RTSP/1.0 200 OK
Cseq:1
Session: 12345678
Transport : RTP/AVP/UDP;unicast;client_port=3056-3057
server_port=5000-5001
C->V : SETUP rtsp://video.example.com/twister/video.en RTSP/1.0
Cseq:1
Transport : RTP/AVP/UDP;unicast;client_port=3058-3059
A-> C : RTSP/1.0 200 OK
Cseq:1
Session: 23456789
Transport : RTP/AVP/UDP;unicast;client_port=3058-3059
server_port=5002-5003

W
V
C

RTSP Message sequence

(contd.)

C->V: PLAY rtsp://video.example.com/twister/video RTSP/1.0

Cseq: 2
Session: 23456789

W
V->C: RTSP/1.0 200 OK
Cseq: 2
Session: 23456789
RTP-Info: url=rtsp://video.example.com/twister/video;
seq=12312232;

C->A: PLAY rtsp://audio.example.com/twister/audio.en RTSP/1.0

Cseq: 2
Session: 12345678
A->C: RTSP/1.0 200 OK
Cseq: 2
Session: 12345678
RTP-Info: url=rtsp://audio.example.com/twister/audio.en;
seq=876655;

V
C

RTSP Message sequence

(contd.)

C->A: TEARDOWN rtsp://audio.example.com/twister/audio.en RTSP/1.0

Cseq: 3
Session: 12345678

A->C: RTSP/1.0 200 OK

Cseq: 3

V
C->V: TEARDOWN rtsp://video.example.com/twister/video RTSP/1.0
Cseq: 3
Session: 23456789
V->C: RTSP/1.0 200 OK
Cseq: 3

References
[1] B. A. Forouzan, TCP/IP Protocol Suite,
Third edition,
[2] H. Schulzrinne, S. Casner, R. Frederick and V.
Jacobson, "RTP: a transport protocol for real-time
applications", RFC 3550, July 2003.
[3] H. Schulzrinne, A. Rao and R. Lanphier, "Real Time
Streaming Protocol (RTSP)", RFC 2326, April 1998.

RTCP compound PDU

receiver
report
source 3

SDES

compound PDU
(single UDP datagram)

SSRC

receiver
report
source 2

RTCP PDU 2
SSRC

sender
report

SSRC

SR

SSRC

RTCP PDU 1

CNAME PHONE

Example
source 1 reports, there are 2 other sources

receiver
report
source 2

receiver
report
source 3

SSRC

sender
report

SSRC

SR

SSRC

RTCP PDU

RTCP processing in Translators

SR sender information : Does not generate their own sender
information(most of the times), but forwards the SR PDUs
received from one side to other
RR reception report blocks : Does not generate their own
RR reports (most of the times), but forwards RR reports
received from one side to another. SSRC are left intact
SDES : Forwards without changing the SDES info. but may
filter non CNAME SDES, if bandwidth is limited
BYE : Forwards BYE PDU unchanged. A translator about to
cease forwarding, send a BYE PDU to each connected nodes

RTCP processing in Mixers

SR sender information : Generates its own SR info. Because
the characteristics of source stream is lost in the mix. The SR
info is sent in same direction as the mixed stream
RR reception report blocks : Generates its own reports for
sources in each cloud and sends them only to same cloud
SDES : Forwards without changing the SDES info. but may
filter non CNAME SDES, if bandwidth is limited
BYE : Forwards BYE PDU unchanged. A mixer about to
cease forwarding, send a BYE PDU to each connected nodes

Source description PDUs

May contain:

a CNAME item (canonical identifier/name)

a NAME item (real user name)
an EMAIL item
a PHONE item
a LOC item (geographic location)
a TOOL item (application name)
a NOTE item (transient msg, e.g. for status)
a PRIV item (private extension)

CNAME=1

length

user and domain name

Value
1
2
3
4
5
6
7
8