Professional Documents
Culture Documents
TESSARES - White Paper Hybrid Access Networks FINAL
TESSARES - White Paper Hybrid Access Networks FINAL
ABSTRACT
To
cope
with
growing
user
appetite
for
bandwidth,
telecommunication
network
operators
are
exploring
various
solutions
to
combine
different
access
network
technologies
together.
This
will
pave
the
way
for
hybrid
access
networks,
i.e.
access
networks
that
combine
very
different
technologies
such
as
DSL
and
LTE.
In
this
white
paper,
we
compare
different
technical
solutions
that
can
be
used
to
create
such
hybrid
access
networks.
In
particular,
we
compare
the
benefits
of
the
Multipath
TCP-based
solutions
to
network-layer
solutions.
Furthermore,
our
measurements
done
on
three
widely
deployed
DSL
access
routers
show
that
a
software-based
solution
using
Multipath
TCP
can
achieve
high
bandwidth
on
existing
devices.
TESSARES
WHITE
PAPER
July
2015
Introduction
Network
operators
have
invested
a
lot
to
deploy
their
broadband
and
mobile
access
networks
during
the
last
decade.
Although
the
performance
of
these
networks
has
continued
to
improve,
the
user
and
application
needs
have
also
continued
to
grow.
This
has
lead
to
users
demanding
higher
bandwidth
while
in
several
countries
governments
are
pushing
to
ensure
that
a
minimum
broadband
performance
is
available
for
all
users
throughout
the
entire
country.
In
cities
and
densely
populated
areas,
deploying
high
bandwidth
networks
is
both
feasible
and
cost
effective.
However,
in
rural
areas
and
less
densely
populated
regions,
the
cost
of
high
bandwidth
networks
may
be
much
higher
than
the
revenue
the
network
operators
can
collect
from
the
users.
To
cope
with
this
problem,
many
network
operators
are
investigating
Hybrid
networks
to
provide
higher
bandwidth
in
some
parts
of
their
footprint
by
combining
two
very
different
network
access
technologies:
fixed
network
(typically
xDSL)
and
cellular
network
(typically
3G
or
4G
wireless
network
technologies).
Hybrid
access
networks
seem
rather
straightforward
at
first
glance
simply
equip
the
home
DSL
router
with
a
cellular
antenna
to
enable
it
to
connect
to
both
networks
at
the
same
time.
This
is
illustrated
in
the
figure
below.
Fixed&
Fixed&
network
network
Cellular&
Mobile&
network
Unfortunately,
in
reality
such
a
Hybrid
access
network
is
not
only
a
hardware
challenge,
i.e.
equipping
a
DSL
router
with
an
antenna.
It
is
mainly
a
software
challenge
because
the
DSL
router
(also
called
HCPE
Hybrid
CPE)
must
include
software
that
enables
it
to
efficiently
send
and
receive
information
over
any
of
the
connected
networks.
There
are
several
possible
solutions
to
solving
this
important
problem.
To
understand
the
pros
and
cons
of
these
solutions
it
is
useful
to
go
back
to
networking
basics.
For
this
we
refer
to
the
five
layer
reference
model
shown
in
most
networking
textbooks.
We
will
use
it
as
our
guideline
to
analyse
in
which
layer
a
Hybrid
access
network
solution
would
best
fit.
layers"
2" Data"Link"
Fixed:"DSL,"Cable,"Fiber,"etc"
Cellular:"WiFi,"WiMAX,"3G,"4G/LTE,"etc"
1" Physical"
The
physical
layer
The
bottom
layer
is
the
physical
layer.
It
deals
with
the
physical
transmission
of
data
and
delivers
a
stream
of
bits.
There
has
been
a
lot
of
innovation
in
this
layer
over
the
years
enabling
high
bandwidth
over
DSL
and
wireless
links.
If
the
two
links
to
be
combined
use
the
same
link
layer
technology,
e.g.
VDSL2,
then
a
pure
physical
layer
solution
is
feasible
and
some
products
already
support
this.
However,
if
the
two
links
use
different
physical
layer
technologies,
then
it
becomes
impossible
to
combine
them
using
a
pure
physical
layer
solution
due
to
the
incompatibilities
between
the
physical
layers
used
by,
for
example,
DSL
and
a
cellular
network
such
as
4G/LTE.
For
the
hybrid
access
network,
the
drawback
with
these
datalink
layer
solutions
is
that
the
most
frequent
candidates
for
hybrid
access
networks,
namely
LTE
and
DSL,
do
not
use
the
same
link
layer
technology.
In
theory,
it
could
be
possible
to
create
a
(Multilink)
PPP
session
over
the
DSL
link
and
a
(Multilink)
PPP
session
over
the
LTE
network
and
terminate
them
in
the
core
network.
Unfortunately,
this
is
difficult
to
implement
because
PPP
was
mainly
targeted
at
low-speed
networks
and
has
not
been
optimised
for
higher
bandwidth
links.
Furthermore,
terminating
the
PPP
session
in
the
wireless
network
would
likely
require
running
the
PPP
sessions
over
a
tunnel
such
as
GRE
over
IP,
which
would
lead
to
a
large
overhead.
Given
the
scarcity
of
IPv4
addresses,
operators
deploy
IPv4
by
allocating
at
most
one
address
per
DSL
user
and
then
rely
on
Network
Address
Translation
(NAT)
on
the
DSL
router
to
enable
all
hosts
connected
in
the
home
to
share
the
same
IP
address.
Each
host
in
the
home
uses
a
private
address,
which
is
translated
by
the
NAT.
With
IPv6,
many
operators
opt
for
a
different
deployment.
Since
an
operator
can
easily
obtain
a
large
block
of
public
IPv6
addresses,
it
can
delegate
a
small
block
of
addresses
to
each
home
router.
The
benefit
of
IPv6
from
an
architectural
viewpoint
is
that
each
home
user
has
a
unique
IPv6
address
and
the
home
router
does
not
need
to
include
any
NAT
function.
Two
families
of
hybrid
access
network
solutions
are
possible
in
the
network
layer:
The
stand-alone
Hybrid
CPE
is
simply
a
router
equipped
with
two
different
access
links
(WAN)
as
shown
in
the
figure
below.
LAN$ WAN$
WiFi$ LTE$
host$ HCPE$ @$
Ethernet$ DSL$
In
practice,
this
simple
scenario
does
not
allow
a
host
to
efficiently
use
the
available
networks.
The
main
problem
is
the
IP
address
that
is
used
by
the
hosts
in
the
home
(LAN).
With
IPv4,
they
use
a
private
IP
address
with
one
address
allocated
to
the
LTE
link
and
one
address
to
the
DSL
link.
When
the
Hybrid
CPE
receives
a
packet
from
a
host
in
the
home
network,
it
must
translate
its
private
IP
address
into
either
the
address
of
the
DSL
link
or
the
address
of
the
LTE
link.
If
the
router
wants
to
send
packets
over
both
the
LTE
and
the
DSL
link,
it
must
translate
their
addresses
intelligently.
It
cannot
simply
send
the
even
packets
over
the
DSL
link
and
the
odd
packets
over
the
LTE
interface.
This
would
break
many
TCP
connections.
Instead,
the
Hybrid
CPE
must
maintain
per-flow
state
and
send
all
the
packets
that
belong
to
a
given
flow
(e.g.
a
TCP
connection/session
or
a
UDP
flow)
over
the
same
interface.
This
implies
that
one
flow
can
only
use
one
interface.
Furthermore,
for
some
applications,
maintaining
per-flow
state
is
not
sufficient.
For
example,
an
online
banking
application
often
opens
several
TCP
connections.
If
these
connections
originate
from
different
addresses
(e.g.
one
from
the
DSL
link
and
one
form
the
LTE
link),
many
online
banks
will
suspect
a
security
problem
and
will
block
the
communication.
For
these
applications,
it
is
important
to
send
all
flows
over
the
same
interface.
Unfortunately,
it
is
not
easy
to
know
in
advance
which
applications
perform
such
security
checks.
In
addition,
if
one
interface
breaks,
it
is
impossible
with
this
solution
to
continue
the
existing
flows
since
they
are
bound
to
one
address
that
is
now
unreachable.
When
a
link
fails,
all
established
flows
have
to
be
re-established
by
the
applications.
This
could
affect
applications
like
some
SSL
VPNs
that
use
long-lived
connections.
When
IPv6
is
used,
the
situation
is
different.
The
host
will
have
one
IPv6
address
assigned
by
the
Hybrid
CPE
that
acts
as
a
regular
router.
The
Hybrid
CPE
receives
this
prefix
from
either
the
DSL
or
the
LTE
network.
Lets
assume
for
simplicity
that
the
prefix
is
assigned
through
the
DSL
network.
The
Hybrid
CPE
will
receive
another
IPv6
address
from
the
LTE
network.
When
sending
a
packet
from
the
host,
the
router
will
forward
the
packet
over
the
DSL
interface
because
security
reasons
prevent
it
from
sending
a
packet
over
the
LTE
network
whose
source
address
was
not
assigned
by
the
LTE
network.
This
implies
that
it
will
be
difficult
to
use
the
LTE
interface.
An
alternative
would
be
to
assign
an
IPv6
prefix
to
the
DSL
router
and
advertise,
e.g.
through
routing
protocols,
this
prefix
over
both
the
LTE
and
the
DSL
network.
This
solution
would
work
in
a
lab
setting
and
would
allow
packets
to
be
sent
over
both
the
LTE
and
the
DSL
network,
but
injecting
one
route
for
each
customer
in
both
the
LTE
and
the
DSL
networks
is
not
a
scalable
solution
that
can
support
a
large
number
of
users.
The
second
family
of
network-layer
solutions
combines
a
Hybrid
CPE
(HCPE)
with
a
Hybrid
Access
Gateway
(HAG)
installed
in
the
operators
network.
Their
main
objective
is
to
hide
to
the
DSL
and
LTE
networks
the
IP
addresses
used
by
the
endhosts.
The
reference
network
is
described
in
the
figure
below.
LAN$ WAN$
WiFi$ LTE$
host$ HCPE$ HAG$ @$
Ethernet$ DSL$
Several
scenarios
are
possible
with
this
deployment.
A
simple
but
realistic
scenario
is
to
use
tunnels
(e.g.
GRE)
on
the
HCPE.
Let
us
assume
that
this
HCPE
has
two
addresses:
A1
and
A2.
A1
is
the
IP
address
assigned
on
the
DSL
link
while
A2
is
the
address
assigned
on
the
LTE
interface.
Let
us
now
analyse
what
happens
when
host
H
sends
many
packets
towards
destination
S.
The
first
packet
is
received
by
the
HCPE
which
decides
to
forward
it
over
the
DSL
interface.
For
this,
the
packet
must
be
translated
to
use
the
A1
source
address.
Upon
reception
of
this
packet,
the
destination
associates
the
flow
to
source
address
A1.
Now,
let
us
consider
how
the
second
packet
sent
by
the
host
will
be
forwarded
by
the
HCPE.
There
are
three
possibilities:
1. The
HCPE
sends
the
packet
with
source
address
A1
over
the
DSL
interface.
The
packet
reaches
the
destination,
but
only
one
path
is
used
for
this
flow.
2. The
HCPE
sends
the
packet
with
source
address
A1
over
the
LTE
interface.
This
is
the
simplistic
approach
that
many
users
would
expect.
Unfortunately,
it
does
not
work
because
the
HCPE
cannot
send
a
packet
over
the
LTE
interface
whose
source
address
does
not
match
the
address
allocated
to
this
interface.
This
restriction
is
a
best
current
practice
of
source
address
filtering
that
is
required
for
obvious
security
reasons.
3. The
HCPE
takes
the
packet
sent
by
the
host,
translates
the
source
address
into
address
A1
(the
same
as
for
the
initial
packet)
and
sends
the
packet
over
the
LTE
interface
inside
a
packet
whose
source
address
is
A2
and
destination
address
is
the
HAG.
The
HAG
knows
that
the
HCPE
has
two
different
addresses
and
when
it
receives
the
encapsulated
packet,
it
removes
the
outer
encapsulation
and
forwards
the
packet
(with
source
address
A1)
towards
the
final
destination.
The
main
benefit
of
the
third
approach
is
that
the
final
destination
does
not
know
that
the
HCPE
and
the
HAG
are
using
encapsulation
to
use
two
different
links.
All
the
packets
received
by
the
final
destination
appear
to
have
originated
from
address
A1.
Thanks
to
encapsulation,
the
HCPE
can
use
the
two
available
interfaces.
However,
this
is
usually
not
sufficient
to
achieve
a
good
user
experience.
The
DSL
and
LTE
links
have
different
delays
and
different
bandwidth.
Furthermore,
the
LTE
network
is
a
shared
network
used
by
a
varying
number
of
users.
This
implies
that
the
bandwidth
of
the
LTE
link
varies
dynamically
over
time.
In
this
context,
let
us
now
consider
a
sequence
of
packets
that
are
sent
over
the
two
interfaces.
Given
the
variability
of
the
bandwidth
on
the
LTE
network,
it
is
difficult
for
the
HCPE/HAG
to
estimate
accurately
the
fraction
of
the
traffic
that
can
be
sent
over
the
LTE
network.
In
particular,
for
many
applications
most
of
the
traffic
is
sent
from
the
HAG
towards
the
HCPE,
and
while
the
HCPE
may
have
some
information
about
its
current
allocated
bandwidth,
it
is
difficult
for
the
HAG
to
track
the
bandwidth
allocated
to
the
thousands
of
HCPEs
that
it
serves.
Given
the
delay
difference,
which
also
varies
over
time
since
it
depends
on
the
load
on
the
LTE
network,
a
sequence
of
packets
sent
over
the
two
interfaces
will
not
reach
the
HAG
or
the
HCPE
in
the
same
order.
This
is
a
major
problem
for
TCP
because
many
TCP
implementations
assume
that
out-of-order
packets
indicate
losses
or
congestion
and
react
to
such
events
by
slowing
down
the
transmission.
If
TCP
is
used
through
a
HAG
that
reorders
packets,
it
will
have
difficulty
in
using
all
the
available
bandwidth.
The
only
solution
to
cope
with
this
problem
is
to
introduce
buffers
in
the
HAG
and
the
HCPE
to
equalise
the
delays
over
the
different
paths,
but
this
will
require
techniques
to
estimate
the
varying
delays
and
to
reorder
the
packets.
This
obviously
increases
the
complexity
of
the
solution.
From
a
performance
viewpoint,
hiding
the
existence
of
the
two
paths
to
protocols
such
as
TCP
that
carry
more
than
90%
of
the
total
Internet
traffic
is
not
a
good
approach.
TCP
includes
a
congestion
control
mechanism
that
continuously
monitors
the
quality
of
the
available
path
and
adapts
its
transmission
rate
accordingly.
But
if
packets
from
a
single
connection
are
sent
over
different
paths,
and
thus
experience
different
delays
and
different
losses,
TCP
may
over-react
since
it
cannot
distinguish
between
the
two
paths
and
degrade
performance.
As
explained
in
the
network
layer
section,
one
of
the
main
difficulties
with
network
layer
solutions
is
that
they
hide
the
hybrid
access
networks
with
different
characteristics
to
TCP
and
thus
can
cause
interferences
that
will
lower
the
performance
of
TCP.
However,
in
contrast,
a
recently
standardised
TCP
extension
called
Multipath
TCP
is
an
ideal
protocol
in
such
a
network
since
it
is
capable
of
efficiently
using
different
paths.
Though
Multipath
TCP
is
not
implemented
as
standard
on
regular
hosts
or
Internet
servers,
it
is
a
possible
to
deploy
it
without
waiting
for
these
hosts
and
servers
to
be
upgraded
by
adding
Multipath
TCP
functionalities
on
both
the
HCPE
and
the
HAG.
LAN$ WAN$
WiFi$ LTE$
bonding$
host$
HCPE$
(MPTCP)$
HAG$
(MPTCP)$ @$
Ethernet$ DSL$
Let
us
consider
in
detail
how
a
Multipath
TCP-based
solution
can
resolve
the
Hybrid
access
network
bonding
problem.
As
before,
assume
that
the
HCPE
has
two
addresses:
A1
and
A2
respectively
assigned
to
the
DSL
and
the
LTE
interfaces.
When
a
host
creates
a
TCP
connection,
it
sends
a
packet
to
the
HCPE.
The
HCPE
decides
over
which
interface
the
first
packet
of
the
TCP
connection
will
be
transported,
e.g.
the
DSL
interface.
The
packet
header
is
translated
by
the
HCPE
and
appears
as
originating
from
address
A1.
In
addition
to
translating
the
source
address,
the
HCPE
also
adds
Multipath
TCP
options
inside
the
packet.
The
HAG
captures
the
packet
before
it
reaches
its
final
destination,
notices
that
this
is
a
new
Multipath
TCP
connection,
removes
the
Multipath
TCP
option
and
sends
it
to
the
final
destination.
The
final
destination
replies
and
the
reply
goes
through
the
HAG
which
inserts
the
Multipath
TCP
option
before
forwarding
it
to
the
HCPE.
The
reply
confirms
the
establishment
of
the
connection
to
the
HCPE.
Data
can
then
flow
over
the
DSL
link.
To
use
the
LTE
link
to
also
forward
data,
the
HCPE
(respectively
the
HAG)
creates
a
sub
flow
over
the
LTE
interface.
This
is
done
by
creating
a
TCP
connection
that
is
terminated
on
the
HAG
(respectively
the
HCPE).
At
this
point,
data
can
flow
over
both
the
DSL
and
the
LTE
interfaces.
The
Multipath
TCP
congestion
control
scheme
automatically
measures
the
performance
of
the
two
interfaces
and
adjusts
the
distribution
of
their
load
based
on
performance
criteria
or
by
enforcing
the
operators
defined
policies.
If
one
interface
fails,
the
connection
continues
via
the
other
without
breaking
the
data
transfer.
Compared
with
network
layer
solutions,
Multipath
TCP
brings
two
important
benefits
from
a
performance
viewpoint.
First,
the
overhead
of
Multipath
TCP
(20
bytes
per
packet)
is
lower
than
the
overhead
of
network
layer
solutions
which
rely
on
encapsulation
(e.g.
36
bytes
for
GRE
for
IPv4).
More
importantly,
thanks
to
its
subflows,
Multipath
TCP
knows
over
which
access
link
each
data
item
is
transmitted.
This
enables
both
the
HCPE
and
the
HAG
to
continuously
measure
the
performance
of
these
access
links
and
distribute
the
data
between
them
based
on
their
real
time
performance.
In
contrast,
a
network
layer
solution
hides
the
existence
of
the
two
access
links
to
the
transport
layer
such
that
it
cannot
then
efficiently
adjust
its
transmission
rate.
In
the
lab,
this
difference
is
not
important
because
the
bonded
links
have
stable
capacity.
In
real
networks,
capacity
fluctuates
and
the
ability
to
efficiently
adjust
the
transmission
rate
is
key
to
maximising
performance
and
perceived
quality
of
experience.
Multipath
TCP
has
other
benefits
from
a
deployment
viewpoint.
Like
other
transport
protocols,
it
is
software
based.
This
implies
that
the
required
Multipath
TCP
functions
can
be
added
as
a
software
upgrade
to
the
CPEs
that
are
already
deployed
in
the
field.
In
regard
to
the
HCPE,
operators
may
fear
that
Multipath
TCP
will
not
be
able
to
achieve
a
good
throughput
due
to
the
low-end
CPUs
used
on
existing
CPEs
and
for
this
reason
prefer
network
layer
solutions
that
are
easier
to
accelerate
with
existing
hardware.
However,
measurements
performed
with
the
Tessares
optimised
Multipath
TCP
stack
on
three
different
HCPEs
based
on
the
Broadcom
BCM63168
chipset
have
demonstrated
that
Multipath
TCP
can
achieve
high
performance
on
widely
deployed
CPEs.
The
figure
below
shows
the
results
of
detailed
experiments
conducted
while
bonding
different
links:
DSL
and
LTE
with
different
bandwidths
and
different
delays.
It
shows
that
Multipath
TCP
bonds
the
two
links
very
efficiently
without
saturating
the
HCPEs
CPU.
Millions
of
such
routers
are
deployed
in
existing
networks.
These
measurements
show
that
operators
could
provide
new
services
to
their
current
users
with
just
a
software
upgrade.
Theore;cal"Aggregated"Downstream" Measured"Aggregated"Downstream"
120"
100"
Applica;on"goodput"[Mbps]"
Combined"
80" throughput"(TP)"
always">"90%"of"
the"theore;cal"
60" TP"(sum"of"DSL"TP"
and"LTE"TP)""
40"
Always">"95%"of""
20" the"sum"of"TCP"
goodputs"
0"
10Mbps"(DSL)"+"25Mbps"(LTE)" 30Mbps"(DSL)"+"25Mbps"(LTE)" 70Mbps"(DSL)"+"40Mbps"(LTE)"
Measurements
with
next
generation
HCPEs
that
are
not
yet
deployed
in
the
field
show
that
they
can
achieve
even
higher
throughput
(several
hundred
Mbps)
without
saturating
their
CPUs.
For
web
traffic,
a
first
solution
is
to
install
a
web
proxy
on
the
HCPE
that
uses
Multipath
TCP
to
interact
with
another
proxy
installed
in
the
operators
network.
This
solution
allows
the
use
of
two
available
links,
but
has
two
important
drawbacks.
First,
it
only
supports
HTTP
traffic.
While
HTTP
is
an
important
application
nowadays,
this
is
not
the
only
application
layer
protocol
that
is
used
by
end-users.
Therefore,
restricting
a
solution
to
support
only
one
application
is
not
a
generic
approach.
Second,
it
is
difficult
to
achieve
high
performance
on
existing
DSL
routers
that
usually
have
limited
CPU
and
memory
to
efficiently
support
a
high
performance
web
proxy.
A
second
solution
is
to
rely
on
applications
such
as
OpenVPN
that
allow
tunnelling
any
type
of
traffic
over
TCP.
OpenVPN
is
a
popular
VPN
application
that
was
initially
designed
to
operate
over
UDP.
It
can
also
operate
over
TCP,
e.g.
when
firewalls
block
UDP.
Some
users
have
experimented
with
OpenVPN
over
Multipath
TCP.
This
approach
works
in
the
laboratory,
but
when
deployed
in
the
field,
packet
losses
cause
performance
problems
that
are
an
inherent
consequence
of
the
utilisation
of
TCP
as
a
tunnelling
protocol.
To
understand
the
problem,
let
us
consider
a
very
simple
scenario
where
one
host
transfers
data
over
a
TCP
connection
through
a
TCP-based
tunnel
between
two
routers.
When
a
packet
loss
occurs
between
the
two
routers,
the
TCP
tunnel
will
detect
the
loss
and
retransmit
the
data.
The
end
hosts
will
observe
an
increase
in
delay
but
no
loss.
For
TCP,
the
losses
are
the
primary
indication
of
congestion.
If
the
end
hosts
do
not
observe
losses,
they
will
try
to
transmit
data
faster,
which
would
cause
long
delays
in
the
routers
that
cannot
absorb
the
data
as
quickly
as
required.
Conclusion
In
this
document,
we
have
qualitatively
compared
the
different
techniques
that
can
be
used
by
network
operators
to
deploy
hybrid
networks.
We
have
explained
the
limitations
of
network-layer
solutions
and
have
shown
the
benefits
that
the
Multipath
TCP
protocol
brings
in
such
environments.
Being
a
software
solution,
Multipath
TCP
can
be
deployed
on
existing
CPEs
without
requiring
a
costly
hardware
upgrade.
About Tessares
Tessares
is
an
innovative
startup
created
by
several
of
the
key
developers
of
the
Multipath
TCP
protocol
and
of
its
reference
implementation
in
the
Linux
kernel.
Tessares
leverages
this
technical
expertise
to
design,
develop
and
deploy
software-based
solutions
to
enable
network
operators
to
improve
network
performance
and
reliability
by
combining
different
access
network
technologies
for
the
benefit
of
their
customers.
Tessares
is
a
privately-owned
company
that
received
significant
first
round
funding
from
both
Proximus
(previously
called
Belgacom)
and
VIVES
(vivesfund.com).