UTT- 1

belo licd a

Awwwmicas

woiy cdal.
puobalitta bekauir

mwuuicalio
Esusy-kue
DisCRErE MEMoRYAESS CHANNEL

puobattaa ke 0,!, . ..k-)

Re 1
kz0
 euttd by tue

ybol ittid at
-Meula
hee gaiu
CCeeue

(s)o
2. Op<1
I(y>I()
4.

daust,

ke0,1, . ..K-!

1(4) K-|
K-|

H (s) z keo
(puot)
a
e 2
weswe te

PeoPERTLES Dr EnROPY

e adix (wo slus)

wkave k
all his doww bod
puobakiktas

i H)-logak
boudl
Puoot:
o H(S) >0 .

H(s)
boune! we

lo
0
2- 1

+1f
 k-l
K-l

K-I
k-l
keo
keo
(G-)
K-I

ke0

We th hae the
K-l

ke0

k 0, !,
k
8 wi the cqui bsababe
oliek copouds
 chanaigae qual
K-1 k 6

k.o

K-l

distbutiom

boumd

HO) lag.k
<

e
hold

MEMORYLESS SouRCE
BiNARY
ENTROH
Bowee ou
bina
0 wist
with Po

uek

(-p) oq:
 the

Similan to

(3) o
2 wheu P, 1 , H
Huax
3

wheu
puobabla

We Po

(-Po)
H()

DiscRErE MEMORVLESs SouRCE

ExTEN SION OE A L Sowe
has k -p
alslabet 8 that
with
evtudd Souue
k te
dinbnet blocks okse

Jouee

putulan &owne

coutlay n H()
H(3")
 Couidn a

H(3) =

bik

3 bih

2 H(s

Seroul-e extueisn o< a

3,32

S, S,
 keo eucodeu
as
syubol auigud .
with
buuay the ke...k-l
t o,!, k
wth
k has
tNcoDING SoURCE
Sowe
Binan bx
Sou
2:
eod te by
wst ed Bownu
tode) Moue (ex
awgued ode shout
Bowu
THEDREM CobINE SouReE
 5

dauote he miiun pescble va

Sownu
HO),
Gtu
boudud
L H(s)

wit H)
HC:)
 DASA CMPRESS 1ON)

Pecex CoDNG: woud tae

LoLth wo wd
toda

So
00
S
0 125

lodab But e
A helesaly
Coue

has

-uoedlagt
Pr-t
ua the ioee
K-l , kuousu
, k.o,),2,
. .

Augt wtan
as ahouu
ula iu eayaiy
- Me
k-l

MeMiUa
aboe kuot-
wiolai
he a

bot e bud oudy

 ke0
H(S)
k-l
Cod-wud
Lode
tocle that
a
tue
touut auu we
a keo
ko
(Budi
euttd
lby eq^ bond hawd -
L< H(s)<
1H(s)+
boumdad uolih
iu
coda au with uut o be
H), out di yiuena
We e

Code,Ln
hau

as

- H()

exlanld
by wakig
Souee

oleind
 (+
HurFHAN CoDING

. Tke owe
loat puobalt,
tuoo bounu Sybob a

a d .
a 0
baing couis
sugaudada
Bowe Jyubo
2. Thee with pao b
weuBoee
The
t w

dowe sltrt
uia dit
by ooukig
The vo,

0-2

D-21

+ o1(s) 22
02(2) to-13)
I 0.42)+ D2(2) +
2 12193 Lit
H(s)
 keo

NoTE:
doiw as

Disc RETE MEMORYLESS GANNELS

uit

whee
 Bnauilio
k
p
we
kuous ha,

we

Disoute Meuyu

P
P(+l)plal
haulwatuix
Cal, chamnd tubut

k-1
Alao,
 ply) P(x: i)
Stauel Rv y
Fou a caf ifp vauiaL x

dauatuy
oliibtuliu P(AlB) P(BAA) P(B)
ply0 P ( )

The waugn
obtel.
PlA) Z RANB)
shouu by

P(lylx ) P(*4)
keo,l, k-)

kuowu as
 keo
H(<)
k-|
) The
a which
(N.H(*
hat
X
JNFORMATION MUTUAL
o
PoPreewe, PeM a
P(yi|z-o)
Ply-ole.a)
:1
o,3-
- 4) (-7, uohuu ke
2 fe
syudo
 H(* y)u caled a
toudeioal

hau hee
about te

Mataad iafoumatien
I(y)H(x)- H(x|4)
T(«y)
H(4) - H(4|a)
I (4;
te chamnd outpt a
H4)