Giáo Trình Lý Thuyết Thông Tin

gio trnh l thuyt thng tin
Bin tp bi:
duongvanhieu
gio trnh l thuyt thng tin

Bin tp bi:
duongvanhieu
Cc tc gi:
phantantai
lequyetthang
duongvanhieu
Phin bn trc tuyn:

http://voer.edu.vn/c/f89a66f8
MC LC
1. gii thiu tng quan gio trnh l thuyt thng tin
2. yu cu
3. ni dung ct li
4. kin thc tin quyt
5. phng php hc tp
6. gii thiu
7. m hnh l thuyt thng tin theo quan im Shannon
8. nh l c s ca k thut truyn tin
9. khi nim v dung lng knh truyn
10. o lng tin
11. cc tnh cht ca entropy
12. entropy ca nhiu bin
13. minh ha cc entropy
14. O LNG TIN (MESURE OF INFORMATION)
15. SINH M TCH C (Decypherable Coding)
16. quan h gia m tch c v di m
17. tnh ti u ca di m
18. knh truyn ri rc khng nh
19. cc dng knh truyn
20. lc gii m
21. nguyn l khong cch nh nht hamming
22. B v t sa li v cn hamming
23. m kim tra chn l
24. nhm cng tnh v b t m chn l
25. lc sa li ti u
26. m hamming
27. thanh ghi li tng bc
28. m xoay vng
29. a thc c trng ca thanh ghi
30. phng php sinh m xoay vng
31. bi tp tng hp
Tham gia ng gp
1/135
gii thiu tng quan gio trnh l thuyt

thng tin
GII THIU TNG QUAN
GIO TRNH L THUYT THNG TIN
MC CH
Gio trnh ny s cung cp cho ngi c nhng khi kin thc c bn ca l thuyt
thng tin nh: do lng tin (Measure of Information), Sinh m tch c
(Decypherable Coding), Knh truyn tin ri rc khng nh (Discrete Memoryless
Channel) v Sa li trn knh truyn (Error Correcting Codings).
Lin quan n o lng tin, gio trnh s trnh by cc khi nim c bn v thng
tin, entropy, mt s cng thc, tnh cht, cc nh l quan trng ca entropy v cch tnh
lng tin.
V Sinh m tch c, gio trnh s gii thiu n ngi hc cc vn v yu cu ca
bi ton sinh m, gii m duy nht, cng nh m tc thi v gii thut kim tra m tch
c. Cc nh l quan trng c cp trong ni dung ny l: nh l Kraft (1949),
nh l Shannon (1948) v nh l sinh m Huffman.
V knh truyn tin ri rc khng nh, gio trnh s gii thiu m hnh knh truyn
theo 2 kha cnh vt l v ton hc. Cc khi nim v dung lng knh truyn, phn lp
knh truyn, nh l v dung lng knh truyn, cng nh cc khi nim trong k thut
truyn tin v phng php xy dng lc gii m ti u cng c trnh by trong
mn hc ny.
Vn Sa li (hay x l m sai) trn knh truyn l mt vn rt quan trng v
c quan tm nhiu trong mn hc ny. Cc ni dung c gii thiu n cc bn s
l Nguyn l Khong cch Hamming, cc nh l v Cn Hamming, phng php kim
tra chn l, cc lc sa li, Bng m Hamming v Bng m xoay vng.
Hn na, hu ht cc vn nu trn u c a vo ni dung ging dy cc bc
i hc ca mt s ngnh trong c ngnh Cng ngh thng tin. Do , c mt ti
liu phc v cng tc ging dy ca gio vin cng nh vic hc tp v nghin cu ca
sinh vin, chng ti mnh dn bin son gio trnh ny nhm gip cho sinh vin c mt
ti liu t hc v nghin cu mt cch hiu qu.
2/135
yu cu
YU CU
Sau khi hc xong mn ny, sinh vin phi c c nhng kh nng sau:
Hiu cc khi nim v v thng tin, Entropy, Entropy ca mt phn phi, Entropy ca
nhiu phn phi, Entropy c iu kin, o lng tin. Vn dng gii quyt cc bi
ton v xc nh lng tin.
Bit khi nim v m tch c, m khng tch c, bng m ti u. Hiu nh l
Kraft (1949), nh l Shannon (1948), nh l sinh m Huffman v phng php sinh
m Huffman. Vn dng sinh bng m tch c ti u, nhn bit c bng m nh
th no l bng m ti u v c th vn dng vit cc chng trnh sinh m, gii m
(hay vit chng trnh nn v gii nn). T y, cc sinh vin c th t nghin cu cc
loi bng m khc vn dng cho vic m ha v bo mt thng tin mt cch hiu qu.
Bit cc khi nim v knh truyn tin ri rc khng nh, dung lng knh truyn v
phn lp knh truyn. Hiu nh l v dung lng knh truyn, phng php xy dng
lc gii m ti u v cch tnh xc sut truyn sai trn knh truyn.
Bit cc khi nim v khong cch Hamming, nguyn l khong cch Hamming, cc
nh l v Cn Hamming, phng php kim tra chn l, cc lc sa li, Bng m
Hamming v Bng m xoay vng.
Vn dng cc kin thc hc c thit k mt h thng truyn nhn d liu vi quy
trnh c bn: m ha, gii m v bo mt thng tin.
L thuyt thng tin cng l mt trong cc mn hc kh ca ngnh Cng ngh thng tin
v n i hi ngi hc phi c kin thc c bn v ton v xc sut thng k. Do ,
i hi ngi hc phi t b sung cc kin thc c bn v ton v xc sut thng k cho
mnh (nu thiu), tham gia lp hc y v lm cc bi tp theo yu cu ca mn hc
th mi tip thu kin thc mn hc mt cch hiu qu.
3/135
ni dung ct li
NI DUNG CT LI
Gio trnh gm 5 chng c trnh by trong 45 tit ging cho sinh vin chuyn ngnh
Cng ngh thng tin, trong c khong 30 tit l thuyt v 15 tit bi tp m gio vin
s hng dn cho sinh vin trn lp.
Chng 1: Gii thiu. Chng ny trnh by cc ni dung c tnh tng quan v mn hc
bao gm: cc i tng nghin cu, m hnh l thuyt thng tin theo quan im ca nh
ton hc Shannon, khi nim v lng tin bit v cha bit, nh l c bn ca k thut
truyn tin.
Chng 2: o lng tin. Chng ny trnh by cc vn c bn v entropy, cc
tnh cht ca entropy, entropy ca nhiu bin, entropy c iu kin, cc nh l v quan
h gia cc entropy v lng tin ca mt s kin.
Chng 3: Sinh m tch c. Ni dung chnh ca chng ny bao gm cc khi nim
v m tch c, quan h gia m tch c v di m, tnh ti u ca di m.
Chng 4: Knh truyn. Cc ni dung c trnh by trong chng ny bao gm khi
nim v knh truyn tin ri rc khng nh, cc m hnh truyn tin kha cnh vt l
v ton hc, dung lng trn knh truyn, phn lp cc knh truyn. Phng php xy
dng lc gii m ti u v cch tnh xc sut truyn sai cng c gii thiu trong
chng ny.
Chng 5: Sa li. Chng ny trnh by cc ni dung ct li sau: khi nim v khong
cch Hamming, nguyn l khong cch nh nht Hamming, b v t sa li v nh
l Cn Hamming. Chng ny cng gii thiu v b m kim tra chn l, phng php
kim tra chn l, lc sa li ti u, m Hamming v m xoay vng.
4/135
kin thc tin quyt

KIN THC TIN QUYT
hc tt mn hc ny, i hi sinh vin phi nm vng cc mn hc c lin quan nh:
xc sut thng k, i s boole (php ton Modulo 2 v a thc nh phn). Cc mn hc
c lin quan v c th tham kho thm nh k thut s, h iu hnh, mng my tnh.
TI LIU THAM KHO
David J.C. Mackey, Information Theory, Infernce, and Learning Algorithms,
CamBridge University Express-2003.
G.J.ChaiTin, Algorithmic Information Theory, CamBridge University Express-1992.
Sanford Goldman, Information Theory.
http://www.inference.phy.cam.ac.uk/mackay/info-theory/course.html.
http://en.wikipedia.org/wiki/Information_theory.
http://www-2.cs.cmu.edu/~dst/Tutorials/Info-Theory/.
http://cscs.umich.edu/~crshalizi/notebooks/information-theory.html.
http://www.lecb.ncifcrf.gov/~toms/paper/primer/primer.pdf.
http://www.cs.ucl.ac.uk/staff/S.Bhatti/D51-notes/node27.html.
http://guest.engelschall.com/~sb/hamming/.
http://www2.rad.com/networks/1994/err_con/hamming.htm
5/135
phng php hc tp
PHNG PHP HC TP
phc v cho mc tiu nng cao kh nng t hc tp v t nghin cu ca sinh vin,
gio trnh ny c bin son cng vi cc gio trnh khc thuc chuyn ngnh Cng
ngh thng tin ca Khoa Cng ngh thng tin v Truyn thng i Hc Cn Th theo
d n ASVIET002CNTT Tng cng hiu qu o to v nng lc o to ca
sinh vin khoa Cng ngh Thng tin-i hc Cn Th. Chng ti c gng trnh
by gio trnh ny mt cch c h thng cc ni dung theo b cc cc chng ng vi
cc khi kin thc nu trn, mi chng c c trnh by theo b cc ca cc bi
hc v mi bi hc gii thiu n ngi hc mt vn no trong s cc vn
ca mt khi kin thc tng ng vi mt chng. Khi hc xong cc bi hc ca mt
chng, ngi hc s c mt khi kin thc cn thit tng ng cho mn hc. Ni dung
ca cc bi hc u c a vo cc v d ngi hc d hiu, ty theo tng vn
m ngi hc cn phi hc v nghin cu trong thi lng t 1 n 2 tit t hc cho
mt bi hc trong mt chng. Nh vy, hc tt mn hc ny, trc ht sinh vin
cn phi:
Hc y cc mn hc tin quyt, b sung nhng kin thc c bn v ton v xc sut
thng k (nu thiu).
Hc v nghin cu k tng chng theo trnh t cc chng c trnh by trong gio
trnh ny. Trong tng chng, hc cc bi theo th t c trnh by, sau mi bi phi
lm bi tp y (nu c).
Tham gia lp y , tho lun cc vn tn ti cha hiu trong qu trnh t hc.
Sau mi chng hc, phi nm vng cc khi nim, cc nh ngha, cc cng thc tnh
ton v vn dng gii cc bi ton c tnh cht tng hp c gii thiu cui chng.
Vn dng kin thc c c sau khi hc xong cc chng gii mt s bi tp tng
hp cui gio trnh, t gip cho ngi hc hiu su hn v mn hc v c th gii
quyt cc vn tng t trong thc t.
Vic cho ra i mt gio trnh vi nhng mc ch nh trn l khng n gin khi kh
nng v kinh nghim ca ngi son cn c hn, nhiu khi nim, thut ng dng trong
gio trnh cha c nh ngha mt cch chnh thng. V vy gio trnh ny chc khng
trnh khi nhng khim khuyt, rt mong nhn c s gp ca cc ng nghip v
ngi c.
6/135
gii thiu
GII THIU
Mc tiu
Sau khi hon tt bi hc ny bn c th bit:
i tng nghin cu,
M hnh l thuyt thng tin theo quan im Shannon,
Cc khi nim v Lng tin bit v lng tin cha bit,
nh l c s ca k thut truyn tin,
Khi nim chung v dung lng knh truyn,
Vn sinh m v gii m.
i tng nghin cu
L thuyt thng k v thng tin c xy dng trn hai hng khc nhau bi hai nh
ton hc Shannon (1948) v Wiener (1949). L thuyt thng tin nghin cu qu trnh x
l tn hiu nh sau:
u vo (input): nhn tn hiu t mt lnh vc c th, tc l tn hiu xut hin theo cc
k hiu (symbol) t mt tp hp cho trc v theo phn phi xc sut bit.
Tn hiu c truyn i trn knh truyn (channel) v c th b nhiu cng theo mt
phn phi xc sut no . Knh truyn c th c hiu di hai ngha:
Di ngha vt l: knh truyn l mt h thng truyn tn hiu (dy dn, mch, sng, ...)
v gy nhiu ty thao cht lng ca h thng.
Di ngha ton hc: knh truyn l cc phn phi xc sut xc nh trn lp cc tn
hiu ang xt u nhn tn hiu (output).
u ra (output): dng li tn hiu chn tht nht c th c so vi tn hiu u vo.
Shannon xy dng m hnh l thuyt thng tin trn c s gii quyt bi ton: sinh m
di ti u khi nhn tn hiu u vo. Tn ti u c xt trn 3 yu t sau:
7/135
Phn phi xc sut ca s xut hin ca cc tn hiu.

Tnh duy nht ca m v cho php t iu chnh m sai nu c vi chnh xc cao
nht. Gii m ng thi t ng iu chnh m hoc xc nh on m truyn sai.
Trong kh , Wiener li nghin cu phng php x l tn hiu u ra: c lng ti
u chui tn hiu so vi chnh n khi nhn u vo khng qua qu trnh sinh m. Nh
vy phng php Wiener c p dng trong nhng trng hp con ngi khng kim
sot c qu trnh truyn tn hiu. Mn x l tn hiu cp n vn ny.
8/135
m hnh l thuyt thng tin theo quan im

Shannon
M hnh l thuyt thng tin theo quan im Shannon
L thuyt thng tin c xt y theo quan im ca Shannon. i tng nghin cu
l mt h thng lin lc truyn tin (communication system) nh s di y:
Din gii:
Ngun (source) thng tin cn gi l thng bo cn c truyn u vo
(Input).
M ha (encode) l b sinh m. ng vi mt thng bo, b sinh m s gn cho
mt i tng (object) ph hp vi k thut truyn tin. i tng c th l:
Dy s ngh phn (Digital) dng: 01010101, cng ging nh m my tnh.
Sng lin tc (Analog) cng ging nh truyn radio.
Knh (channel) l phng tin truyn m ca thng tin.
Nhiu (noise) c sinh ra do knh truyn tin. Ty vo cht lng ca knh
truyn m nhiu nhiu hay t.
Gii m (decode) u ra (output) a dy m tr v dng thng bo ban u
vi xc sut cao nht. Sau thng bo s c chuyn cho ni nhn. Trong s
trn, chng ta quan tm n 2 khi m ha v gii m trong ton b mn
hc.
Lng tin bit v cha bit
Mt bin ngu nhin (BNN) X lun mang mt lng tin no . Nu X cha xy ra (hay
ta cha bit c th thng tin v X) th lng tin ca n l cha bit, trong trng hp
ny X c mt lng tin cha bit. Ngc li nu X xy ra (hay ta bit c th thng
9/135
tin v X) th lng tin v bin ngu nhin X coi nh bit hon ton, trong trng hp
ny X c mt lng tin bit.
Nu bit thng tin ca mt BNN X thng qua BNN Y xy ra th ta c th ni: chng
ta ch bit mt phn lng thng tin ca X trn c s bit Y.
V d v lng tin bit v cha bit
Ta xt v d v mt ngi t chc tr chi may ri khch quan vi vic tung mt ng
tin c u hnh khng c u hnh. Nu ngi chi chn mt khng c u hnh
th thng khi kt qu tung ng tin l khng c u hnh, nguc li th thua. Tuy nhin
ngi t chc chi c th n gian bng cch s dng 2 ng tin Tht- Gi khc
nhau sau:
+ ng tin loi 1 (hay ng tin tht): ng cht c 1 mt c u hnh.
+ ng tin loi 2 (hay ng tin gi ): ng cht, mi mt u c 1 u hnh.
Mc d ngi t chc chi c th n gian nhng qu trnh trao i 2 ng tin cho
nhau l ngu nhiu, vy liu ngi t chc chi c th n gian hon ton c khng?
Hay lng tin bit v cha bit ca s kin ly mt ng tin t 2 ng tin ni trn
c hiu nh th no?
Ta th xt mt trng hp sau: nu ngi chi ly ngu nhin 1 ng tin v sau
thc hin vic tung ng tin ly c 2 ln. Qua 2 ln tung ng tin, ta m c s
u hnh xut hin. Da vo s u hnh xut hin, ta c th phn on c ngi t
chc chi ly c ng tin no.
Chng hn: Nu s u hnh m c sau 2 ln tng l 1 th ng tin ly c l
ng tin tht. Ngc li nu s u hnh m c l 2 th ng tin ly c c th
l tht hay cng c th l gi. Nh vy, ta nhn c mt phn thng tin v loi ng
tin qua s u hnh m c sau 2 ln tung. Ta c th tnh c lng tin bng bao
nhiu? (Vic tnh lng tin ny s c tho lun sau). Di y l mt s bng phn
phi ca bi ton trn:
Gi BNN X v loi ng tin (X=1 nu ly c ng tin loi 1 v X=1 nu ly c
ng tin loi 2 c ly).
Khi phn phi ca X c dng:
10/135
t BNN Y l BNN v s u hnh m c sau 2 ln tung. Khi ta c th xc nh

c phn phi ca Y vi iu kin xy ra ca X trong 2 trng hp sau.
Phn phi ca Y khi bit X=1 c dng:
Phn phi ca Y khi bit X=2 c dng:
11/135
nh l c s ca k thut truyn tin

nh l c s ca k thut truyn tin
Trong New Basic of Information Theory (1954) Feinstein a ra nh l sau:rn
mt knh truyn c nhiu, ngi ta lun c th thc hin mt phng php truyn sao
cho t c sai s nh hn sai s cho php (nh bt k) cho trc i vi knh truyn.
Chng ta s khng chng minh nh l, thay vo , chng ta s tham kho n cc
minh ha gim nhiu trong cc ni dung tip theo ca bi hc.
M t trng thi truyn tin c nhiu
Gi s, mt thng bo c truyn i trn mt knh truyn nh phn ri rc. Thng bo
cn truyn c m ha thnh dy s nh phn (0,1) v c di c tnh theo n v
bit. Gi s 1 bit truyn trn knh nhiu vi xc sut 1/4 (hay tnh trung bnh c truyn 4
bit th c th nhiu 1 bit).
Ta c s trng thi truyn tin sau:
Minh ha k thut gim nhiu

Trong k thut truyn tin, ngi ta c th lm gim sai lm khi nhn tin bng cch truyn
lp li 1 bit vi s l ln.
V d: truyn lp li 3 cho 1 bit cn truyn (xc sut nhiu 1 bit bng 1/4). Khi nhn 3
bit lin nhau cui knh c xem nh l 1 bit. Gi tr ca bit ny c hiu l 0 (hay
1) nu bit 0 (bit 1) c s ln xut hin nhiu hn trong dy 3 bit nhn c lin nhau
(hay gii m theo nguyn tc a s). Ta cn chng minh vi phng php truyn ny th
xc sut truyn sai tht s < 1/4 (xc sut nhiu cho trc ca knh truyn).
S truyn tin:
12/135
Tht vy:
Gi s Xi xc nh gi tr ng hay sai ca bit th i nhn c cui knh truyn vi
Xi =1 nu bit th i nhn c l sai v Xi =0 nu bit th i nhn c l ng. Theo gi
thit ban u ca knh truyn th phn phi xc sut ca Xi c dng Bernoulli b(1/4):
Gi Y ={X1 + X2 + X3 } l tng s bit nhn sai sau 3 ln truyn lp cho 1 bit. Trong
trng hp ny Y tun theo phn phi Nh thc B(p,n), vi p=1/4 (xc sut truyn sai
mt bit) v q =3/4 (xc sut truyn ng 1 bit):
Y ~ B(i,n) hay
Trong :
Vy truyn sai khi Y thuc {2, 3} c xc xut l:
13/135
Hay
(pcm).
Chi ph phi tr cho k thut gim nhiu
Theo cch thc lp li nh trn, ta c th gim sai lm bao nhiu cng c (lp cng
nhiu th sai cng t), nhng thi gian truyn cng tng ln v chi ph truyn cng s
tng theo.
Hay ta c th hiu nh sau:
Lp cng nhiu ln 1 bit=>thi gian truyn cng nhiu=>chi ph cng tng.
14/135
khi nim v dung lng knh truyn

Khi nim v dung lng knh truyn
V d trn cho chng ta thy cn phi xc nh mt thng s cho truyn tin m bo
sai s chp nhn c v ng thi tc truyn cng khng qu chm.
Khi nim dung lng knh truyn l khi nim rt c bn ca l thuyt truyn tin
v l mt i lng vt l ng thi cng l i lng ton hc (c n v l bit). N
cho php xc nh tc truyn ti a ca mi knh truyn. Do , da vo dung lng
knh truyn, ngi ta c th ch ra tc truyn tin ng thi vi mt phng php
truyn c sai s cho php.
Vn sinh m
T k thut truyn tin trn cho ta thy qu trnh sinh m v gii m c m t nh sau:
mt n v thng tin nhn c u vo s c gn cho mt k hiu trong b k hiu
sinh m. Mt k hiu m c gn n ln lp li (da vo dung lng ca knh truyn,
ta c th xc nh c n). Thit b sinh m (Coding device/ Encoder) s thc hin qu
trnh sinh m.
Nh vy, mt n v thng tin t ngun pht tin s c thit b sinh m gn cho mt
dy n k hiu m. Dy k hiu m ca 1 n v thng tin c gi l mt t m (Code
word). Trong trng hp tng qut, ngi ta c th gn mt khi k t m cho mt khi
thng tin no v c gi l mt t m.
Vn gii m
cui knh truyn, mt thit b gii m (Decoding device/ Decoder) s thc hin qu
trnh ngc li nh sau: kim tra dy k hiu m quyt nh gii m v mt t m v
a n v dng khi tin ban u.
V d:
Khi tin ban u : 01010101
Khi k hiu m u truyn (lp 3 ln): 000111000111000111000111.
Khi k hiu m u nhn : 001110100111011001000111
Khi tin nhn c cui cng : 01011001 (sai 2 bit so vi khi tin ban u)
15/135
Do lm sao ua khi tin nhn c v khi tin ban u 01010101, y chnh l

cng vic ca b gii m (Decoder).
Mt vn quan trng cn lu l phi ng b gia tc np thng tin (pht tn hiu)
vi tc truyn tin. Nu tc np thng tin bng hoc ln hn so vi tc truyn
tin ca knh, th cn phi gim tc np thng tin sao cho nh hn tc truyn tin.
16/135
o lng tin
O LNG TIN
Mc tiu: trnh by cc khi nim v o lng tin cha bit v bit v mt bin
ngu nhin X. Tnh ton cc lng tin ny thng qua nh ngha v cc tnh cht ca
Entropy t mt hay nhiu bin ngu nhin.
ENTROPY
Mc tiu
Sau khi hon tt bi hc ny bn c th:
Hiu c cc khi nim Entropy,

Bit Entropy ca mt s kin v Entropy ca mt phn phi,
Hiu nh l dng gii tch ca Entropy,
Bit Bi ton v cy tm kim nh phn v
Lm kin thc c s hiu v hc tt cc bi hc tip theo.
V d v entropy
Trc ht, ta cn tm hiu mt v d v khi nim do ca mt lng tin da vo cc
s kin hay cc phn phi xc sut ngu nhin nh sau:
Xt 2 BNN X v Y c phn phi sau:
X={1, 2, 3, 4, 5} c phn phi u hay p(X=i) = 1/5.
Y={1, 2} cng c phn phi u hay p(Y=i) = 1/2.
Bn thn X v Y u mang mt lng tin v thng tin v X v Y cha bit do chng l
ngu nhin. Do , X hay Y u c mt lng tin khng chc chn v lng tin chc
chn, tng ca 2 lng tin ny l khng i v thc t n bng bao nhiu th ta cha th
bit. Lng tin khng chc chn ca X (hay Y) c gi l Entropy.
Tuy nhin, nu X v Y c tng quan nhau th X cng c mt phn lng tin khng
chc chn thng qua lng tin bit ca Y (hay thng tin v Y c bit). Trong
trng hp ny, mt phn lng tin khng chc chn ca thng qua lng tin bit
ca Y c gi l Entropy c iu kin.
Nhn xt v o lng tin
17/135
R rng, ta cn phi xy dng mt i lng ton hc rt c th c th o c lng

tin cha bit t mt bin ngu nhin. Mt cch trc quan, lng tin phi th hin
c cc vn sau:
Mt s kin c xc sut cng nh th s kin t xy ra, cng c ngha l tnh khng
chc chn cng ln. Nu o lng tin ca n th n cho mt lng tin khng bit cng
ln.
Mt tp hp cc s kin ngu nhin (hay Bin ngu nhin) cng nhiu s kin c phn
phi cng u th tnh khng chc chn cng ln. Nu o lng tin ca n th s c
lng tin khng bit cng ln. Hay lng tin chc chn cng nh.
Mt phn phi xc sut cng lch nhiu (c xc xut rt nh v rt ln) th tnh khng
chc chn cng t v do s c mt lng tin cha bit cng nh so vi phn phi xc
sut u hay lng tin chc chn ca n cng cao.
Khi nim entropy

Trong ting vit ta cha c t tng ng vi t Entropy, tuy nhin chng ta c th tm
hiu hiu thong qua trc khi i vo nh ngha chc ch v mt ton hc ca Entropy
nh sau:
Entropy l mt i lng ton hc dng o lng tin khng chc (hay lng ngu
nhin) ca mt s kin hay ca phn phi ngu nhin cho trc. Hay mt s ti liu
ting anh gi l Uncertainty Measure.
Entropy ca mt s kin
Gi s c mt s kin A c xc sut xut hin l p. Khi , ta ni A c mt lng khng
chc chn c o bi hm s h(p) vi p [0,1]. Hm h(p) c gi l Entropy nu n
tho 2 tiu ton hc sau:
Tin 01: h(p) l hm lin tc khng m v n iu gim.
Tin 02: nu A v B l hai s kin c lp nhau, c xc sut xut hin ln lt l pA
v pB. Khi , p(A,B) = pA.pB nhng h(A,B) = h(pA) + h(pB).
Entropy ca mt phn phi
Xt bin ngu nhin X c phn phi:
18/135
Nu gi Ai l s kin X=xi, (i=1,2,3,..) th Entropy ca Ai l: h(Ai)= h(pi)

Gi Y=h(X) l hm ngu nhin ca X v nhn cc gi tr l dy cc Entropy ca cc s
kin X=xi, tc l Y=h(X)={h(p1), h(p2), , h(pn)}.
Vy, Entropy ca X chnh l k vng ton hc ca Y=h(X) c dng:
H(X)=H(p1, p2, p3, ,pn) = p1h(p1)+ p2h(p2)++pnh(pn).
Tng qut:
nh l dng gii tch ca Entropy

nh l: Hm H(X) = H(p1, p2,...,pM)
C = const >0
C s logarithm l bt k.
B : h(p)=-Clog(p).
Trng hp C=1 v c s logarithm = 2 th n v tnh l bit.
Khi : h(p)=-log2(p) (vt: bit) v
Qui c trong cch vit: log(pi)= log2(pi)

V d minh ha
Nu s kin A c xc sut xut hin l 1/2 th h(A)=h(1/2)= -log(1/2) = 1 (bit)
19/135
Xt BNN X c phn phi sau:
H(X) = H(1/2, 1/4, 1/4) = -(1/2log(1/2)+1/4log(1/4)+1/4log(1/4)) =3/2 (bit)

Bi ton v cy tm kim nh phn-t vn
Gi s, tm 1 trong 5 ngi c tn bit trc s xut hin theo phn phi sau:
Trong : x1, x5 ln lt l tn ca 5 ngi m ta cn nhn ra vi cch xc nh tn

bng cu hi ng sai (yes/no).
S di y minh ha cch xc nh tn ca mt ngi:
Bi ton v cy tm kim nh phn - Din gii

Theo s trn:
tm x1, x2, x3 vixc sut tng ng l
0.2, 0.3, 0.2 ta ch cn tn 2 cu hi.
20/135
tm x4, x5 vi xc sut tng ng 0.15, 0.15 th ta cn 3 cu hi.

Vy:
S cu hi trung bnh l: 2 x (0,2+0,3+0,2) + 3 x (0,15+0,15) = 2.3
Mt khc: Entropy ca X: H(X)= H(0.2, 0.3, 0.2, 0.15, 0.15)=2.27.
Ta lun c s cu hi trung bnh lun H(X) (theo nh l Shannon s trnh by sau).
V s cu hi trung bnh trong trng hp ny xp s H(X) nn y l s cu hi trung
bnh ti u tm ra tn chnh xc ca mt ngi. Do , s tm kim trn l s
ti u.
Sinh vin t cho thm 1 hay 2 s tm kim khc v t din gii tng t - xem nh
bi tp.
Bi tp
Tnh H(X) vi phn phi sau:
Tnh H(Y) vi phn phi sau:
21/135
cc tnh cht ca entropy

CC TNH CHT CA ENTROPY
Mc tiu:
Hiu cc tnh cht c bn ca Entropy,
Hiu nh l cc i ca Entropy,
Vn dng gii mt s bi ton v Entropy,
Lm c s vn dng gii quyt cc bi ton tnh dung lng knh truyn.
Cc tnh cht c bn ca Entropy
Xt bin ngu nhin X = {x1, x2, , xM}. Entropy ca bin ngu nhin X c cc tnh
cht:
Hm s f(M) = H(1/M,1/Mn iu tng.
Hm s f(ML) = f(M)+f(L).
H(p1, p2, , pM) = H(p1 + p2 ++pr, pr+1+ pr+2++ pM)
H(p, 1-p) l hm lin tc theo P.

Minh ha tnh cht 1 v 2
Minh ha tnh cht 1:
Trong trng hp bin ngu nhin X c phn phi u
Entropy ca X nh sau :
22/135
=> H(X)
l hm n iu tng
Minh ha tnh cht 2:
Trong trng hp 2 bin ngu nhin X, Y c lp c phn phi u vi BNN X c M
s kin v BNN Y c L s kin.
Gi f(M), f(L) ln lt l Entropy ca X, ca Y. Theo tnh cht 2 ca Entropy ta c
f(ML)=f(M)+f(L)
Minh ha tnh cht 3 v 4
Minh ha tnh cht 3:
Xt con xc sc c 6 mt vi xc sut xut hin cc mt c cho trong bng sau:
Ta c th gom cc s kin x1, x2, x3 li thnh mt s kin mi l x123 c xc sut xut

hin l 55%, gom s kin x5 v x6 li thnh s kin x56 c xc sut 20%.
Ta c mt nhin ngu nhin mi X* c phn phi sau:
n y cc bn c th p dng cng thc tnh, so snh cc Entropy v nhn xt tnh

cht 3. Phn ny xem nh bi tp cho sinh vin.
Minh ha tnh cht 4:
hiu tnh cht th 4, ta xt dng th ca hm s H(p, 1-p ):
23/135
R rng H(p, 1-p) l mt hm lin tc theo p.

nh l cc i ca entropy
nh l: H(p1, p2, ,pM) log(M)
Trong : ng thc xy ra khi v ch khi p1== pM= 1/M
B :cho 2 b {p1, p2, ,pM} v {q1, q2,,qM} l cc b s dng bt k v
Khi , ta c H(p1, p2, ,pM)=
(*)
ng thc xy ra khi pi=qi vi i=1,..,M.
Bi tp
Bi 1: Cho 2 bin ngu nhin X, Y c lp nhau c phn phi sau:
Tnh H(X), H(Y).

Kim tra li kt qu ca ca bi 1 bng tnh cht 2.
24/135
Cho bin ngu nhin X c phn phi sau:
Ta c th gom cc s kin x1, x2, x3 li thnh mt s kin mi l x123 c xc sut xut

hin l 55%, gom s kin x5 v x6 li thnh s kin x56 c xc sut 20%.
Ta c mt nhin ngu nhin mi X* c phn phi sau:
- Tnh entropy ca X, X* v kim tra li tnh cht 3.

- Kim tra li nh l cc i t d liu cho trn.
25/135
entropy ca nhiu bin

ENTROPY CA NHIU BIN
Mc tiu
Hiu bit cc nh ngha Entropy ca nhiu bin v Entropy c iu kin,

Hiu mi quan h gia H(X,Y) vi H(X) v H(Y) khi X, Y c lp,
Hiu mi quan h gia H(X,Y) vi H(X) v H(Y) khi X, Y tng quan,
Vn dng mi quan h ga cc Entropy tnh cc Entropy mt cch hiu qu,
Vn dng Entropy c iu kin lm c s tnh lng tin trong bi hc k
tip
nh ngha Entropy ca nhiu bin

Gi s: X v Y l 2 bin ngu nhin cho trc vi pj = p(X=xi,Y=yj) ( i=1,..,M v
j=1,,L).
Khi , Entropy H(X,Y) c dng:
Hay
Mt cch tng qut:
V d Entropy ca nhiu bin

Cho 2 BNN X v Y c lp nhau v c cc phn phi:
26/135
Tnh H(X,Y).
- Lp phn phi ca P(X,Y)
- H(X,Y) =H(0.125, 0.25, 0.125, 0.125, 0.25, 0.125)=2.5 (Bit)

nh ngha Entropy c iu kin
Entropy ca Y vi iu kin X=xi (i=1,..,M) c nh ngha l:
Entropy ca Y vi iu kin X xy ra c nh ngha l:
V d Entropy c iu kin
Xt bin ngu nhin X v bin ngu nhin Y c tng quan nhau. Cc phn phi nh
sau:
Phn phi ca Y c iu kin X:
27/135
Entropy ca Y/X=1 v Y/X=2 nh sau :

H(Y/X=1)=H(0.25, 0.5 , 0.25)= -0.25 log0.25 0.5 log0.5-0.25 log0.25
=0.5 + 0.5 + 0.5= 1.5 (Bit)
H(Y/X=2)= H(0; 0; 1)= 0 (Bit)
Entropy ca Y khi X xy ra:
H(Y/X)=P(X=1) H(Y/X=1)+ P(X=2) H(Y/X=2)=(0.5x1.5) + ((0.5x0)=0.75 (Bit).
Quan h gia H(X,Y) vi H(X) v H(Y) khi X, Y tng quan
nh l 2: H(X,Y)=H(X)+H(Y/X)=H(Y)+H(X/Y).
nh l 3: H(Y/X) H(Y) v Du ng thc xy ra khi v ch khi X v Y c lp nhau.
Chng minh nh l 2:
Tng t ta c: H(X,Y)=H(Y)+H(X/Y)
Chng minh nh l 3:
T nh l 1 v nh l v quan h gia cc Entropy, ta c:
H(X,Y)=H(X)+H(Y/X) H(X)+ H(Y) => H(Y/X) H(Y).
28/135
Sinh vin t chng minh

Bi tp
Xt BNN X v BNN Y c tng quan nhau. Cc phn phi nh sau:
Phn phi ca Y c iu kin X:
Tnh cc Entropy sau: H(X), H(Y).

Tnh cc Entropy c iu kin sau: H(X/Y), H(Y/X).
Tnh cc Entropy sau: H(X,Y).
T kt qu cu 1,2 v 3 hy minh ha cc nh l 1, 2 v 3 cho bi hc.
29/135
minh ha cc entropy
MINH HA CC ENTROPY
Mc tiu
Bit c Yu cu ca bi ton,
Bit cch xc nh cc phn phi ngu nhin ca bi ton,
Vn dng cc bi hc trc tnh cc Entropy H(X), H(Y) v H(X,Y),
Vn dng cc bi hc trc tnh cc Entropy c iu kin H(X/Y) v H(Y/
X),
Nhn xt v so snh quan h gia cc Entropy
Ngoi ra cn gip bn n tp v hiu r hn cc cng thc tnh Entropy.
Yu cu ca bi ton
nhau sau:
Mc d ngi t chc chi c th n gian nhng qu trnh trao i 2 ng tin cho
nhau l ngu nhiu, vy liu ngi t chc chi c th n gian hon ton c khng?
Hay lng tin bit v cha bit ca s kin ly mt ng tin t 2 ng tin ni trn
c hiu nh th no?
Ta th xt mt trng hp sau: nu ngi t chc chi ly ngu nhin 1 ng tin v
sau thc hin vic tung ng tin ly c 2 ln. Qua 2 ln tung ng tin, ta m
c s u hnh xut hin. Da vo s u hnh xut hin, ta c th phn on c
ngi t chc chi ly c ng tin no.
Chng hn: Nu s u hnh m c sau 2 ln tng l 1 th ng tin ly c l
ng tin tht, ngc li nu s u hnh m c l 2 th ng tin ly c c th
l tht hay cng c th l gi. Nh vy, ta nhn c mt phn thng tin v loi ng
30/135
tin qua s u hnh m c sau 2 ln tung. Ta c th tnh c lng tin bng bao
nhiu? (Vic tnh lng tin ny s c tho lun sau).
Xc nh cc phn phi ngu nhin ca bi ton
t X l bin ngu nhin v loi ng tin.
Phn phi ca X:
t bin ngu nhin Y l s u hnh m c sau 2 ln tung:

Phn phi ca Y khi nhn c ng tin c 1 mt c u hnh (Y/X=1)
Phn phi ca Y khi nhn c ng tin c 2 mt u c u hnh (Y/X=2)
Tm phn phi ca Y:
P(Y=0) = p(X=1)p(Y=0/X=1)+p(X=2)p(Y=0/X=2) = 0,5 x 0,25 +0,5 x 0 =0.125
P(Y=1) = p(X=1)p(Y=1/X=1)+p(X=2)p(Y=1/X=2) = 0,5 x 0,5 +0,5 x 0 =0.250
P(Y=2) = p(X=1)p(Y=2/X=1)+p(X=2)p(Y=2/X=2) = 0,5 x 0,25 + 0,5 x 1=0.625
Minh ha Entropy H(X), H(Y) v H(X,Y)

Entropy ca X:
H(X) = H(0.5, 05)
31/135
= -(0.5)log(0.5) -(0.5)log(0.5) = 1 (bit)

Entropy ca Y:
H(X) = H(0.125, 0.25, 0.625)
= -(0.125)log(0.125) + (0.25)log(0.25) + (0.625)log(0.625) = 1.2988 (bit)
Entropy ca X v Y: H(X,Y)
Xem nh bi tp dnh cho cc bn sinh vin
Entropy ca Y/X l trung bnh ca cc entropy Y/X=x i .
Vy, Entropy ca Y c iu kin X
Tng t: H(Y,Z/X), H(Z/X,Y)

Minh ha Entropy H(X/Y) v H(Y/X)
Tnh Entropy ca Y khi bit X: H(Y/X)
H(Y/X=1) = H(0.25, 0.5 , 0.25)
= -(0.25log0.25 + 0.5log0.5 + 0.25log0.25)= 1.5 (bit)
H(Y/X=2)= H(0, 0, 1)= 0
H(Y/X)= p(X=1)H(Y/X=1)+ p(X=2)H(Y/X=2)= 0.5 x 1.5 + 0.5 x 0= 0.75 (bit)
Tnh Entropy ca X khi bit Y: H(X/Y)
Xem nh bi tp dnh cho cc bn sinh vin (Gp : bn nn lp cc phn phi cho cc
trng hp (X/Y=0), (X/Y=1) v (X/Y=2).
Minh ha quan h gia cc Entropy
Xem nh bi tp dnh cho cc bn sinh vin.
Gi : sau khi bn tnh H(X,Y) v H(X/Y), bn da vo cc nh l 1,2 v 3 cng vi
cc kt qu tnh c so snh v minh ha.
32/135
O LNG TIN (MESURE OF

INFORMATION)
O LNG TIN (MESURE OF INFORMATION)
Mc tiu
Bit bi ton tnh lng tin,

Hiu nh ngha lng tin,
Bit cch tnh lng tin,
C th vn dng tnh lng tin cho cc bi ton tng t.
t vn bi ton
nhau sau:
Mc d ngi t chi c th n gian nhng qu trnh trao i 2 ng tin cho nhau l
ngu nhiu, vy liu ngi t chc chi c th n gian hon ton c khng? Ta th
xt mt trng hp sau: nu ngi chi ly ngu nhin 1 ng tin v sau thc hin
vic tung ng tin ly c 2 ln. Qua 2 ln tung ng tin, ta m c s u hnh
xut hin. Da vo s u hnh xut hin, hy tnh lng tin v loi ng tin ly c
l bao nhiu?
Xc nh cc phn phi ca bi ton
t bin ngu nhin X l loi ng tin, khi phn phi ca X c dng :
33/135
t bin ngu nhin Y l s u hnh m c sau 2 ln tung. Khi ta c th xc nh

c phn phi ca Y trong 2 trng hp sau.
Trng hp 1: Phn phi ca Y khi bit ng tin l tht (X=1) c dng:
Trng hp 2: Phn phi ca Y khi bit ng tin l gi (X=2) c dng:
Ta c th tnh d dng phn phi ca Y nh sau:
Nhn xt da theo entropy

T cc bng phn phi trn, ta c:
Entropy ca Y:
H(Y) = H(0.125, 0.25, 0.625) = 1.3 (bit)
Entropy ca Y khi bit X
H(Y/X=1) = H(0.25, 0.5 , 0.25)= 1.5 (bit)
H(Y/X=2)= H(0, 0, 1)= 0
H(Y/X)= p(X=1)H(Y/X=1)+ p(X=2)H(Y/X=2) = 0.75 (bit)
Vy, H(Y) > H(Y/X)
nh ngha lng tin
T nhn xt v quan h gia cc entropy trn, ta c th nh ngha lng tin nh sau:
34/135
nh ngha: Lng tin (hay thng lng) ca X khi Y xy ra l lng chnh lch gia
lng khng chc chn ca X v lng khng chc chn ca X khi Y xy ra c quan h
vi X.
Ta c th hiu khi nim ny nh sau: X v Y l 2 bin ngu nhin nn chng c 2 lng
tin khng chc chn. Nu X v Y c lp, th X xy ra khng nh hng ti Y nn ta
vn khng bit g thm v X v X gi nguyn lng khng chc chn ca n. Trong
trng hp ny lng tin v X khi Y xy ra l bng 0. Nu Y c tng quan vi X th
khi Y xy ra ta bit hon ton v Y v mt phn thng tin v X. Phn thng tin chnh
l lng tin bit v X nhng vn cha bit ht v X. Bi ton y l tnh lng tin
bit v X khi Y xy ra.
K hiu: I(X/Y) = H(X)-H(X/Y) l lng tin bit v X khi Y xy ra.
Ch : ta lun c I(X/Y) = I(Y/X)
V d: xt li v d trn, ta c lng tin v X khi bit Y l
I(X/Y)= I(Y/X)= H(Y) H(Y/X) = 1.3 0.75=0.55 (bit).
Bi tp
Thc hin mt php th con xc sc ng cht ng thi vi mt ng tin cng ng
cht. Trong , con xc sc c cc mt im t 1 n 6, ng tin mt mt c u hnh
v mt kia khng c u hnh. Trc tin th con xc sc, nu s im <_ 4 th tung
ng tin mt ln, ngc li th tung ng tin hai ln. Tnh lng tin v s im con
xc sc khi bit thng tin v s u hnh m c.
Ngi ta thc hin mt kho st trn cc sinh vin i hc v mi quan h gia kh nng
hc tp vi s hu phng tin i li v tinh thn i hu. Kt qu cho thy:
Trong tng s sinh vin c 3/4 sinh vin hon thnh chng trnh hc v 1/4 khng hon
thnh. Trong s sinh vin hon thnh chng trnh hc, 10% c xe con. Ngc li, trong
s sinh vin khng hon thnh chng trnh hc c ti 50% c xe con.
Tt c sinh vin c xe con u tham gia hi i hu sinh vin. Trong s sinh vin khng
c xe con (k c hon thnh hay khng hon thnh kha hc) th 40% sinh vin tham
gia hi i hu sinh vin.
Tm thng tin v trng thi hc tp ca sinh vin khi bit iu kin v phng tin i li
ca h.
Tm thng tin v tnh trng hc tp ca sinh vin khi bit tinh thn i hu ca h.
35/135
Nhng ngi dn ca mt lng c chia lm 2 nhm A v B. Mt na nhm A chuyn

ni tht, 3/10 ni di v 2/10 t tri tr li. Trong nhm B: 3/10 ni tht, 1/2 ni di v
2/10 t tri tr li. Gi s p l xc sut chn 1 ngi thuc nhm A v I(p) = I(Y/X) l
lng tin v ngi ni tht sau khi chn nhm, tnh I(p), tm p* sao I(p*) = Max(I(p)
v tnh I(p*).
36/135
SINH M TCH C (Decypherable

Coding)
SINH M TCH C (Decypherable Coding)
Mc tiu:
Phn ny cp n bi ton m ha (coding) cc gi tr ca mt bin X. Khi m cc gi
tr ca X ngi ta phi s dng bng k t m (Coding Character Table) hay bng ch
ci (Code Alphabet). Nh vy, mt gi tr x ca X s c m thnh mt t m (Code
Word) w di dng mt dy cc k t m vi di l n k t. Trong truyn tin, mt
dy cc gi tr ca X c pht sinh v c m thnh mt dy lin tc cc t m hay
mt dy cc k t m ly t bng k t m. Vn cn gii quyt l:
Khi nhn mt dy k t m lin tc th ta c th gii m thnh mt dy cc gi tr duy
nht ca X hay khng ? Ni cch khc, dy k t m ny c tch c thnh cc t m
mt cch duy nht hay khng ?
Ch ra phng php xy dng m tch c ti u.
KHI NIM V M TCH C
Mc tiu
Bit yu cu ca bi ton sinh m,

Hiu khi nim v bng m tch c v bng m khng tch c,
Hiu khi nim v bng m tc thi,
Hiu gii thut kim tra tnh tch c ca mt bng m,
Vn dng gii thut kim tra tnh tch c ca mt bng m kim tra xem
mt bng m c phi l bng m tch c hay khng.
t vn bi ton sinh m
Gi s ngun tin X xut hin v c ghi li thng qua mt thit b c bit. Chng hn
nh nh c ghi li bng my nh, m thanh c ghi li bng my ghi m, Qua
knh truyn, nhng thng tin ny cn phi c m ha cho ph hp. c th m ha
ngi ta cn mt bng ch ci gm cc ch ci quy nh trc (chng hn bng ch ci
la tinh, bng m nh phn, ). Mi gi tr ca X sau c m di dng mt dy
37/135
hu hn cc ch ci v ta gi dy hu hn cc ch ci gn cho mt gi tr ca x l mt
t m.
Ta xt BNN X={x1, x2, ,xn} c phn phi {p1, p2, , pn} c quan st lin tc v
c lp. Dy cc gi tr nhn c gi l thng bo (Message) c dng xi1xi2xin. Tp
hp A={a1, a2, , an} l tp hp k t m (Code Characters) hay l bng ch ci (Code
Alphabet) dng sinh m. Mt gi tr xi thuc X c gn bi mt dy hu hn cc k
t m c gi l t m (Code word). Tp hp gm tt c cc t m gn cho tt c cc
gi tr ca X c gi l b m hay bng m (Code). Cc t m phi khc nhau tng i
mt.
B m c gi l tch c nu nh t mt dy cc k t m nhn c lin tc (c
m ha t b m ny), ta lun lun gii m c vi kt qu duy nht l dy cc gi tr
gc ca X.
Shannon (1948) ln u tin a ra nh l c s v sinh m tch c. Mc Millan
(1956) chng minh nh l v iu kin cn v ca bng m tch c. Nhng vn
sinh m tch c ch c xt mt cch chun mc bi Feinstein (1958), Abramson
(1963) v Fano (1961). Sardinas(1960) v Patterson (1963) a ra nh l v gii
thut kim tra tnh tch c ca mt bng m. Abramson (1963) a ra khi nim
bng m tc thi.
Trong phm vi bi ging ny, bi ton sinh m ti u c t ra y l tm ra mt
phng php sinh m sao cho di trung bnh ca cc t m trong b m l nh nht.
Ngha l, nu gi tr xi c gn bi t m c di ni th bi ton sinh m phi tha:
Huffman (1950) a ra qui trnh xy dng mt bng m ti u tha yu cu ny.

Khi nim v bng m khng tch c
Bng m khng tch c l bng m m khi m ha thng bo Msg ta s nhn c
mt dy cc t m ws, v khi gii m dy cc t m ws th ta c th nhn c nhiu
thng bo Msg khc nhau.
V d: Xt bin ngu nhin X={x1, x2,x3, x4} c bng m W={w1=0, w2=1, w3=01,
w4=10}.
Gi s thng bo ngun c ni dung: x1x2x3x4x3x2x1. Khi dy m tng ng vit t
W c dng: 0101100110.
38/135
Nu gii m tun t t tri qua phi ta nhn kt qu: x1x2x1x2x2x1x1x2x2x1. Nhng nu

bng phng php khc ta c th nhn c kt qu: x3x3x4x3x4 v nhiu thng bo
khc na.
Nhn xt:Bng m gii m khng tch c l bng m m trong tn ti t nht mt
t m ny l m kha ca mt hay nhiu t m khc trong b m (v d t m w1=0 hay
w2=1 l m kha ca w3).
Bng m tch c
Bng m tch c l bng m m khi m ha thng bo Msg ta s nhn c dy cc
t m ws, v khi gii m dy cc t m ws th ta ch nhn c mt thng bo duy nht
l Msg ban u.
V d: Xt bin ngu nhin X={x1, x2} c bng m tng ng W={w1=0, w2=01}.
Phng php gii m c s dng nh sau: ch gii m khi no nhn c on m
vi di bng di ca t m di nht.
Gi s dy m nhn c (cn gii m) l: 0010000101001.
S dng phng php gii m trn ta nhn c duy nht dy thng bo gc:
x1x2x1x1x1x2x2x1x2.
C th chi tit ha cc bc gii m dy t m trn nh sau:
Nhn c on 00 -> Gii ra x1 , cn li 0.
Nhn tip 1 ->01 -> Gii ra x2.
Nhn tip 00 -> Gii ra x1, cn li 0.
Nhn tip 0 -> 00 -> Gii ra x1, cn li 0.
Nhn tip 0 -> 00 -> Gii ra x1, cn li 0.
Nhn tip 1 -> 01 -> Gii ra x2.
Nhn tip 01 -> Gii ra x2.
Nhn tip 00 -> Gii ra x1, cn li 0.
39/135
Nhn tip 1 -> 01 -> Gii ra x2.

Kt qu dy thng bo l: x1x2x1x1x1x2x2x1x2.
Kt lun:Bng m tch c l bng m m trong khng tn li t m ny l m
kha t m khc, tuy nhin vn c th tn ti t m ny l tin t (phn u) ca t m
kia.
Khi nim bng m tc thi
Bng m tc thi l bng m m khi m ha thng bo Msg ta s nhn c dy cc t
m ws, v khi gii m dy cc t m ws th ta ch nhn c mt thng bo duy nht
l Msg ban u. Abramson chng minh c kt qu sau: Bng m tc thi l
bng m khng tn ti t m ny l tin t ca t m khc.
V d 1: Bng m W={w1=10; w2=101; w3=100} khng phi bng m tc thi v w1 l
tin t ca w2 v w3.
V d 2: Bng m W={w1=0, w2=100, w3=101, w4=11} l bng m tc thi v khng
tn ti t m ny l tin t ca t m khc.
Gii thut kim tra tnh tch c ca bng m
Th tc sau y do Sardinas (1960), Patterson (1963) v Abramson (1963) a ra nhm
kim tra xem mt bng m no c phi l bng m tch c (bng m cho php gii
m duy nht) hay khng.
Input: Bng m W
Output: Kt lun bng m tch c hay khng tch c.
Gii thut:
Bc khi to: Gn tp hp S 0 =W.
xc nh tp hp S1 t S0:
- Khi to S1={}
- Vi wi, wj thuc S0, ta xt: nu wi=wjA (wj l tin t ca wi) hoc wj=wi A (wi l
tin t ca wj) th thm A (phn hu t) vo S1.
Bc k: xc nh tp hp Sk (k2) t tp hp S0 v Sk-1:
40/135
- Khi to: Sk={}

- Vi withuc S0 v vj thucSk-1, ta xt: nu wi=vjA (vj l tin t ca wi) hoc
vj=wi A (wi l tin t ca vj) th thm A (phn hu t) vo Sk.
iu kin dng vng lp:
Nu Sk={} th dng v kt lun bng m tch c (k1).
Nu tn ti t m wi trong Sk hay Sk giao S0 khc rng th dng v kt lun bng m
khng tch c.
Nu Sk=St<k th dng v kt lun bng m tch c (k1).
Bi ton 1- yu cu
Bi ton: Kim tra xem bng m W={a, c, ad, abb, bad, deb, bbcde} c phi l bng m
tch c hay khng?
p dng Gii thut kim tra tnh tch c ca mt bng m:
Bc khi to: S0={a, c, ad, abb, bad, deb, bbcde}
Bc 1: Tnh S1
Khi to S1={}
V a l tin t ca ad nn a phn hu t d vo S1 => S1={d}.
V a l tin t ca abb nn a phn hu t bb vo S1 => S1={d, bb}.
Kim tra iu kin dng: khng tha -> qua bc 2.
Bc 2: Tnh S2 t S0 v S1.
Khi to S2={}.
V d thuc S1 l tin t ca deb thuc S0 nn a phn hu t eb vo S2
=> S2={eb}
V bbthuc S1 l tin t ca bbcde thuc S0 nn a phn hu t cde vo S2
=> S2={eb, cde}
41/135

Bi ton 1 - p dng gii thut
Khi to S3={}.
V cthuc S0 l tin t ca cde thuc S2 nn a phn hu t de vo S3
=> S3={de}
Khi to S4={}.
V dethuc S3 l tin t ca deb thuc S0 nn a phn hu t b vo S4
=> S4={b}
+ khi to S5={}.
+ V bthuc S4 l tin t ca bad thuc S0 nn a phn hu t ad vo S5 => S5={ad}
+ V bthuc S4 l tin t ca bbcde thuc S0 nn a bcde vo S5
=> S5={ad, bcde}
Kim tra iu kin dng: V S5 c cha t m ad nn dng li v kt lun y l bng
m khng tch c.
Bi ton 2
Bi ton: Kim tra xem bng m W={010, 0001, 0110, 1100, 00011, 00110, 11110,
101011} c phi l bng m tch c khng?
p dng Gii thut kim tra tnh tch c ca mt bng m:
42/135
Bc khi to v bc 1
- Tp hp S0 ={010, 0001, 0110, 1100, 00011, 00110, 11110, 101011}
- Tp hp S1 ={1}
Dnh cho sinh vin t lm cc buc tip theo.
Kt qu gi :
Tp hp S2 ={100, 1110, 01011}
Tp hp S3={11}
Tp hp S4={00, 110}
Tp hp S5={01, 0, 011, 110}
Tp hp S6={0, 10, 001, 110, 0011, 0110}
Tp hp S6 cha t m 0110 nn bng m ny khng phi l bng m tch c.
Bi tp
Hy cho bit bng m sau c phi l bng m tch c hay khng?
W={w1=00, w2=01, w3=0010, w4=0111, w5=0110}
Hy ly v d mt bng m tch c, v chng minh n l bng m tch c.
43/135
quan h gia m tch c v di m

QUAN H GIA M TCH C V DI M
Mc tiu
Sau khi hon tt bi hc ny bn c th hiu:
nh l Kraft (1949),
nh ngha cy bc D c K,
Vn sinh m cho cy bc D c K,
Vn dng nh l Kraff kim tra s tn ti bng m tch c v sinh bng
m tch c.
nh l Kraftn(1949).
Gi X={x1, x2,, xM} l bin ngu nhin cha cc gi tr cn truyn c phn phi l
P={p1, p2, , pM}.
A={a1, a2,,aD} l b k t sinh m c D ch ci (D c gi l c s sinh m).
Gi tr xi c m ha thnh t m wi c di l ni.
t N={n1, n2,,nM} l tp hp di cc t m.
nh l (Kraft- 1949):
iu kin cn v tn ti bng m tc thi vi di N={n1,n2,,nM} l
V d 1: B m W={w1, w2, w3} vi M=3; n1=1; n2=2; n3=3; D=2
=> Tn ti bng m tc thi.

V d 2: B m W={w1, w2, w3} vi M=3; n1=n2=1; n3=2; D=2
44/135
=> Khng tn ti bng m tc thi.

ngh: sinh vin tm hiu ni dung tip theo v tr li gii thch 2 v d trn.
nh ngha cy bc D c k.
nh ngha: Cy bc D c k l cy c h thng nt, cnh tha iu kin:
T 1 nt c s cnh i ra khng vt qu D hay mt nt c khng qu D nt
con.
Nt cui cng (Nt l) cch nt gc khng vt qu k cnh.
V d: cy bc D=2 v c k=3
Vn sinh m cho cy bc D c k
Sinh m cho cc nt ca cy bc D c K (tr nt gc):
n gin ha: mi nt (tr nt gc) c k hiu bi dy k hiu ca nt cha lm tin
t + mt k t b sung ly t tp hp {0, 1, 2, , D-1} thay cho bng ch ci A={a1,
a2, , aD}.
V d 1: Cy bc D=2 c k=3 V d 2: Cy bc D=3 c k=2.
45/135
Tnh cht:
+ Cc nt (tr nt gc) ca cy u c m ha t bng ch ci {0, 1, 2,, D-1}
+ Mi nt ( m ha) c m ca nt k trc l tin t.
+ Tng s cc nt l bng Dk = tng s cc m tc thi c th c.
Chng minh nh l Kraft (iu kin cn)
Gi s, cho trc bng m tc thi W={w1, w2,, wM} vi N={n1<_ n2 <_ <_ nM}.
Ta cn c/m:
Xy dng cy bc D c nM v sinh m cho cc nt tr nt gc vi cc k t m ly t

bng ch ci A = {0, 1, 2,, D-1}. M ti mi nt (tr nt gc) u c kh nng c
chn l t m.
Nh vy, ta tin hnh chn cc t m cho bng m tc thi vi qui tc l: mt nt no
c chn gn mt t m th tt c cc nt k sau nt gn t m phi c xa.
C th nh sau:
Chn mt nt c m vi di m l n1 gn cho n mt t m w1.
46/135
Chng minh nh l Kraft (iu kin )

Gi s:
,
cn chng minh tn ti bng m tc thi vi N={n1, n2, , nM}, ta ch cn ch ra th
tc xy dng bng m tc thi nh sau:
Th tc to m tc thi:
Xt N={n1, n2, ,nM} v c s sinh m l D:
Bc 1: Ta xp th t n1<_ n2 <_ <_ nM, xy dng cy bc D c k=nM v sinh m
cho cc nt.
Bc 2: Chn nt bt k trn cy c di n1 gn cho t m w1 v xa tt c cc nt k
sau n.
Bc 3: Lp li cc bc 2 i vi vic chn cc t m cn li w2, , wM ng vi n2,
, nM.
=> Bng m W={w1, w2, , wM} l bng m tc thi.
47/135
V d minh ha nh l Kraft
V d 1: Xt bng m tha M=3, D=2, n1=1, n2=2, n3=3. Vy ta kim tra xem c to
c bng m tc thi hay khng?
Ta c
=> W= {w1, w2, w3} l bng m tc thi

Ta Xy dng bng m nh sau:
Ch : ngoi bng m tc thi chn c trn, ta cn c th sinh c nhiu bng m

tc thi khc. ngh sinh vin a ra bng m tc thi khc.
Bi tp
Tm 1 bng m tch c tha tnh cht D = 2, k = 4?
Tm tt c cc bng m tch c tha tnh cht D=2, k=3?
Hy ch ra bng m sau y l bng m khng tch c:
W={w1=00, w2=1, w3=100, w4=110, w5=111}
Hy tm mt bng m nh phn tch c c t nht 5 t m tha iu kin
48/135
49/135
tnh ti u ca di m
TNH TI U CA DI M
Mc tiu
Hiu nh l Shannon (1948),
Bit c cc tiu chun nh gi bng m ti u tuyt i v bng m ti u tng i,
iu kin nhn bit mt bng m ti u,
Hiu nh l Huffman,
Bit Phng php sinh m Huffman,
Vn dng phng php sinh m Huffman sinh m Huffman cho mt thng bo,
Vn dng phng php sinh m Huffman vit chng trnh nn.
nh l Shannon (1948)
Pht biu nh l:
t
l di trung bnh ca bng m.

Khi
Du ng thc xy ra khi v ch khi

Din gii: i vi m tch c di trung bnh ca m s c cn di
lH(x)/Log2(d). Nu m khng tch c di trung bnh ca n c th nh hn cn
50/135
di. Nu m tch c khng ti u th di ca n s ln hn nhiu so vi cn di,

cn nu m tch c ti u th di trung bnh ca n gn vi cn di.
Bi ton t ra s l tm phng php xy dng bng m tch c ti u.
Ch :
l entropy ca X vi c s D.
Bng m ti u tuyt i
nh l: Bng m c gi l ti u tuyt i khi
V d: xt bin ngu nhin X={x1, x2, x3, x4}

C phn phi: P={1/2, 1/4, 1/8, 1/8}
C bng m W={w1= 0, w2=10, w3=110, w4=111}
Ta tnh c di trung bnh t m:
Tnh Entropy ca X: H(X)= H(0.5, 0.25, 0.125, 0.125) = 0.5 +0.5 + 0.375 + 0.375 =1.75
Log2D=1.
51/135
Bng m ti u tng i
nh l: Bng m c gi l ti u tng i khi:
iu kin nhn bit mt bng m ti u

nh l (vi D=2):
Xc sut pi cng ln th di ni ca t m wi cng nh.
Gi s p1_> p2 _> 1_> pM. Nu pi_> pi+1 _> pi+r th ni <_ni+1 <_ni+r th 2 t
m tng ng vi 2 gi tr c xc sut nh nht c di m bng nhau nM-1
=nM.
Trong cc t m c di bng nhau v cng bng nM (di nht) th tn ti t
nht 2 t m wM-1 v wM c M-1 k t u ging nhau v k t th M khc
nhau.
V d: xt bng m W={w1=0, w2=100, w3=101, w4=1101, w5=1110}
Bng m trn khng phi l bng m ti u v 2 t m w4, w5 c di ln nht =4 k
t nhng 3 k t u khc nhau.
Ghi ch: D > 2 c xt tng t.
nh l Huffman
nh l: Gi s X c phn phi xc sut vi th t gim dn sau:
52/135
Gi s bng m ca X l W={w1, w2, , wM-1, wM}.

t xM-1,M={xM-1, xM} c xc sut l pM-1,M=pM-1+pM.
v X* = { x1, x2,, xM-1,M} c phn phi sau:
Gi s W* ={w1, w2, , wM-2, x*M-1,M} l bng m ti u ca X*. Khi :

- wM-1=w*M-1,M + 0.
- wM =w*M-1,M + 1.
Phng php sinh m Huffman
Gi s X c phn phi xc sut vi th t gim dn sau:
Th tc li (D=2):
Khi to: t M0=M
Bc 1:
- t
c xc sut
- Sp xp li theo t t gim dn ca xc sut ta nhn c dy phn phi mi c M0-1

phn t nh sau:
53/135
Bc 2: Lp li bc 1 vi s lu vt
Gim M0: M0=M0-1, vng lp kt thc khi M0=2

(Ch : trong trng hp tng qut, vong lp kt thc khi M0 <_ D.)
Th tc tin:
i ngc li so vi th tc li ng thi xc nh t m mi bc t s lu vt th
tc li.
Minh ha phng php sinh m Huffman
V d 1: sinh bng m nh phn Huffman cho X c phn phi sau:
54/135
Th tc li: di trung bnh ca t m:

Vecto n=(0.3 x 2)+ (0.25 x 2)+ (0.2 x 2) + (0.1 x 3) +(0.1 x 4) + (0.05 x 4) = 2.4
Entropy ca X: H(X) = H(0.3, 0.25; 0.2, 0.1,0.1, 0.05)
= 2.4
Nhn xt: Do D = 2 v log2D=1, Ta c vecto n = H(X) nn bng m trn ti u tuyt
i.
Bi tp
Cho bin ngu nhin X c phn phi sau:
Cho bin ngu nhin Y c phn phi sau:

55/135
Cho on vn bn thoi the thi thoi thi the thoi thi the. Tm bng m nh phn Huffman
dng m ha on vn bn trn.
Thay tng k t trong on vn bn trn thnh mt t m, ct tng on 8 bits i thnh
s thp phn. Cho bit dy s thp phn kt qu.
56/135
knh truyn ri rc khng nh

KNH TRUYN
Mc tiu:
Trnh by m hnh truyn tin ri rc tng k t m c lp ln nhau (ph hp vi c
im ca knh). M hnh ny cn gi l knh truyn ri rc khng nh (Memoryless
Discret Channel). T m hnh ny ngi ta c th xy dng cch tnh dung lng knh
truyn v phng php phn loi u nhn c th gii m tt nht.
KNH TRUYN RI RC KHNG NH
Mc tiu
Bit m hnh knh truyn tin ri rc khng nh 2 kha cnh vt l v ton
hc.
Khi nim v lng tin trn knh truyn
nh ngha dung lng knh truyn
Gii thiu
Trc ht, ta c th hiu khi nim knh truyn ri rc v khng nh bi hc ny nh
sau: khi nim truyn ri rc y l truyn tun t cc k t c lp nhau (hay truyn
tng k t mt), cn khi nim khng nh y l ch xt mi quan h gia k t truyn
v k t nhn c tng ng, khng xt n mi quan h gia k t nhn c vi k
t nhn c trc .
Khi nim v mt knh truyn ri rc da vo phn b xc sut ca tn hiu ra ph thuc
vo tn hiu vo v trng thi ca knh truyn c chun ha bi Feinstein (1958)
v Wolfowitz (1961). Dung lng knh (Channel Capacity) c xc nh chnh xc
nh Muroya (1953) v Fano (1961). Gii thut v chng trnh tnh dung lng knh
c vit bi Eisenberg (1963).
nh l c bn v truyn tin ch ra rng vi dung lng knh cho trc lun c
th tm ra mt phng php truyn tin vi lng tin nh hn dung lng knh v
t sai s nh hn sai s cho php bt k. nh l c bn v truyn tin c
Feinstein (1954, 1958) kho st. Cc nh khoa hc Blackwell, Breinan (1958, 1959) v
Thomasian (1961) ln lt chnh l t chun tt hn. Trong cc ni dung tip
theo ca bi hc, cc bn s tm hiu v m hnh knh truyn tin ri rc khng nh
57/135
khia cnh vt l v ton hc. c bit m hnh ton hc s ch ra cch xc nh phn

phi u ra da vo phn phi u vo.
M hnh vt l
Mt thng bo c cu to t cc k hiu ca mt bng ch ci u truyn (input)
v c truyn trn knh. Thng bo c nhn cui knh (hay u nhn-output) v
c gii m theo bng ch ci u truyn. Mt khc, tng k t u nhn c th
quan h vi cc k t u nhn trc , cc k t u truyn v trng thi ca knh
truyn. n gin, y chng ta ch xt m hnh vt l nh sau:
Xt tng k t u nhn ch ph thuc vo k t u truyn tng ng vi n, nu
knh truyn c nhiu th mt k t u truyn c th c din gii (nhiu) ra nhiu
k t khc nhau u nhn v do to ra mt phn phi xc sut c iu kin cho k
t u nhn. M hnh truyn tin ri rc khng nh l m hnh truyn tin ch xt mi
quan h gia k t truyn v k t nhn c tng ng, khng xt mi quan h gia k
t nhn c v k t nhn c trc .
M hnh:
M hnh ton hc
Ta gi:
58/135
Tx={x1, x2, , xM} l b k t sinh m u truyn (input).

Ty={y1, y2,,yL} l b k t gii m u nhn (output).
Bin ngu nhin X ly gi tr ( m ha) trn Tx v c phn phi
p(X=xi)=p(xi) vi i=1,..,M.
Bin ngu nhin Y ly gi tr (gii m) trn Ty v c phn phi xc sut c iu
kin: P(Y=yj/X=xi)=p(yj/xi)=pij vi j=1,..,L.
Gi A=||pij|| l ma trn truyn tin hay m hnh truyn tin ca knh truyn ri rc khng
nh.
Vi v pij = p(Y=yj/X=xi) = p(yj/xi) l xc sut nhn c gi tr yj khi truyn gi tr
xi.
Tnh phn phi u nhn:
Ta c:
Vy p(yj)= PX .Aj(1)
Mt cc tng qut: PY = PX.A (2)
Trong :
Ajl ct th j ca A
PX = [p(x1), p(x2),., p(xM)].
PY = [p(y1), p(y2),., p(yM)].
V d xc nh phn phi u nhn
Cho ma trn truyn tin nh sau:
59/135
Xc sut truyn: p(x1)=0.5 v p(x2)=p(x3)= 0.25.

Ta tm phn phi ca Y :
Ta c: PX =(0.5, 0.25, 0.25)
p dng cng thc (1) trn ta c:
p(y1) = Px .A1 = 0.375
p(y2) = Px .A2 = 0.3
p(y3) = Px .A3 = 0.325
=> PY =(0.375, 0.3, 0.325)
Lng tin trn knh truyn
V d: cho ma trn truyn tin nh sau:
Xc sut truyn: p(x1)=0.5 v p(x2)=p(x3)= 0.25.

X = {x1,x2,x3} c xem nh tp cc k t truyn v Y ={y1, y2, y3} l tp cc k t
nhn.
Tnh lng tin trn knh truyn:
Ta tm phn phi ca Y :
Ta c: PX =(0.5, 0.25, 0.25)
p dng cng thc (1) trn ta c:
p(y1) = Px .A1 = 0.375
p(y2) = Px .A2 = 0.3
60/135
p(y3) = Px .A3 = 0.325

=> PY =(0.375, 0.3, 0.325)
Tnh cc Entropy:
H(Y) = H(0.375, 0.3, 0.325) = 1.58 (bit)
H(Y/X=x1) = H(0.5, 0.2, 0.3)= 1.49 (bit)
H(Y/X=x2) = H(0.3, 0.5, 0.2)= 1.49 (bit)
H(Y/X=x1) = H(0.2, 0.3, 0.5)= 1.49 (bit)
H(Y/X)= p(x1).H(Y/X=x1) + p(x2).H(Y/X=x2) + p(x3).H(Y/X=x3) = 1.49 (bit)
Lng thng tin truyn trn knh: I (X/Y)= I (Y/X)= H(Y) - H(Y/X) = 0,09 (bit)
nh ngha dung lng knh truyn
Da vo ma trn truyn tin A, ta c th d dng tnh lng tin trn knh truyn.
I(X/Y)= H(X)-H(Y/X)
= H(Y)-H(X/Y)
= I(Y/X)
Ta c I(X/Y)= H(Y)-H(Y/X), trong :
H(Y)= H(PX .A) ph thuc vo PX.
H(Y/X) ph thuc vo PX
Vy: I(Y/X) ph thuc hon ton vo PX v do I(Y/X) c th t Max vi PX xc
nh no .
Ta nh ngha:
l dung lng ca knh truyn (VT: bit).
61/135
cc dng knh truyn

CC DNG KNH TRUYN
Mc tiu
Bit knh truyn khng mt tin,
Bit knh truyn xc nh,
Bit knh truyn khng nhiu,
Bit knh truyn khng s dng c,
Hiu knh truyn i xng,
Hiu nh l v dung lng knh truyn,Knh truyn khng mt tin

M hnh: t tp hp cc gi tr c th nhn c u nhn Y={y1, y2, , yL} c
phn thnh M nhm Bi tng ng vi cc gi tr xi u truyn v xc sut truyn xi
vi iu kin nhn yj l p(X= xi /Y=yj thuc Bi)=1 ( vi M < L ).
c trng ca knh truyn khng mt tin l H(X/Y)=0. C ngha l lng tin cha bit
v X khi nhn Y l bng 0 hay ta c th hiu khi nhn c Y th ta hon ton c th
bit v X.
Dung lng: C=log2M (Sinh vin t chng minh, xem nh bi tp)
62/135
Knh truyn xc nh
M hnh: t tp hp cc gi tr c th truyn u truyn c phn thnh L nhm Bj
tng ng vi cc gi tr c th nhn c yj u nhn v xc sut nhn yj vi iu
kin truyn xi l p(Y=yj/X=xi thuc Bj)=1 (M>L).
c trng: ca knh truyn xc nh l H(Y/X)=0. C ngha l lng tin cha bit v Y

khi truyn X bng 0 hay khi truyn X th ta bit s nhn c Y.
Dung lng: C=log2L (Sinh vin t chng minh, xem nh bi tp)
Knh truyn khng nhiu
M hnh: l s kt hp ca knh truyn xc nh v knh truyn khng mt thng tin,
truyn k t no s nhn c ng k t .
c trng: H(X/Y)=H(Y/X)=0.
Dung lng: C=log2L=log2M (Sinh vin t chng minh, xem nh bi tp)
V d: ma trn truyn tin ca knh truyn khng nhiu vi M=L=3
63/135
Knh truyn khng s dng c.

M hnh: l knh truyn m khi truyn gi tr no th mt gi tr hoc xc sut nhiu
thng tin trn knh truyn ln hn xc sut nhn c.
c trng: H(X/Y)=H(Y/X)= max
Dung lng: C=0 (Sinh vin t chng minh, xem nh bi tp)
V d: knh truyn c ma trn truyn tin nh sau:
Knh truyn i xng

M hnh: l knh truyn m ma trn truyn tin c c im sau:
+ Mi dng ca ma trn A l mt hon v ca phn phi P={p1, p2, , pL}
+ Mi ct ca ma trn A l mt hon v ca Q={q1, q2, , qM}
V d: cho knh truyn i xng c ma trn truyn tin nh sau:
Xy dng cng thc tnh dung lng knh truyn i xng

Do H(Y/X) khng ph thuc vo phn phi ca X => Max ca I(X/Y) c quy v m
ca H(Y). Hay
64/135
Ta c th tnh d dng:
Do :
Do H(Y)<= logL => ta cn chng t = xy ra khi p1=p2=...=pL=1/L

Xt trng hp P(X=xi)=1/M, vi mi i => chng minh P(Y=yj)=1/L vi mi j
Tht vy :
T A ta nhn thy:
= tng cc phn t ca A.Do
=>
=> H(Y) t max l logL khi P(Y=yj)=1/L hoc P(X=xi)=1/M
65/135
Vy: C= log L H(p1, p2, , pL ) hay
Ch : trng hp knh 1 bit vi nhiu

Ma trn truyn tin
Dung lng
nh l v dung lng knh truyn

Gi s ma trn A c dng vung v c ma trn nghch o l A-1
K hiu A=||pij|| vi i=1,2,...,M v j =1,2,...,M
A-1=||qij|| vi i=1,2,...,M v j =1,2,...,M
t tham s
Nu dk>0 th dung lng knh truyn c dng:
66/135
Gi tr cc i t khi tn hiu vo X=X* tha phn phi P(X*=xk)=2-Cdk

Hay C=max I(X/Y)=I(X*/Y)
Ch :
- iu kin dk>0 cho php hm I(X/Y) l hm li => Tn ti Max tuyt i ti phn
phi ca X* vi p(X*=xk)=2-C dk =pk (vi mi k).
- Nu iu kin ma trn vung hoc ma trn ngch o khng tha th gi tr cc i max
s nm trn ng bin ca min xc nh {pk>0 v -pk=1}
Bi tp
Cho mt knh truyn c ma trn truyn tin nh sau:
Tnh dung lng knh truyn.

Chng minh cc cng thc tnh dung lng knh truyn trn.
67/135
lc gii m
LC GII M
Mc tiu
Bit t vn bi ton gii m,

Hiu cc khi nim c bn ca k thut truyn tin,
Bit v hiu cc dng sai s c bn ca k thut truyn tin,
Hiu phng php xy dng lc gii m ti u,
Vn dng xy dng lc gii m ti u v tnh cc dng xc sut truyn sai.
t vn bi ton gii m
Phn tch yu cu gii m:
Khi truyn gi tr xi, ta s nhn c yj.
i vi knh truyn khng nhiu th yj chnh l xi. i vi knh truyn c nhiu th yj
c th khc xi. Do ta cn tm cch gii m yj v gi tr xi khi knh truyn c nhiu.
Php phn hoch cc gi tr u nhn:
Php phn hoch tp cc gi tr u nhp yj thuc Y l php phn chia tp Y thnh cc
tp con Bi sao cho:
1.
(mi i j)
2. Khi nhn yj thuc Bi th gii m v xi.
V d bi ton gii m
Cho tp cc t m truyn X v tp cc dy n bit nhn c Y nh sau:
68/135
X={0000, 0101, 1110, 1011}

Y={0000, 0001, 0010, 0011, 0100, 0101, 0110, 0111,
1000, 1001, 1010, 1011, 1100, 1101, 1110, 1111}
Gi s ta c th phn hoch tp Y thnh cc tp con Bi nh sau:
B1={0000, 1000, 0001, 0010}
B2={0101, 1101, 0100, 0111}
B3={1110, 0110, 1111, 1100}
B4={1011, 0011, 1010, 1001}
Gi s nhn yj = 0011 th gii m v x4 = 1011 v yj thuc B4.
Cc khi nim c bn ca k thut truyn tin
Xt s truyn tin nh sau:
Din gii:
Ngun pht tn hiu (hay thng bo) vi vn tc R (tn hiu/giy).
Tn hiu c m ha t b k t m.
Tn hiu m ha c truyn trn knh vi vn tc C (k t/giy), C ng thi
l dung lng ca knh truyn.
Tn hiu truyn trn knh c th b nhiu vi xc sut P(e).
Trc khi nhn, tn hiu m ha c gii m theo mt phng thc ti u v
chnh xc cao nht c th c.
69/135
Bi ton t ra y: tm gii php to m sao cho sai s u nhn c xc sut nh hn

bt k ( < P(e)) ng thi vi ng b ha: vn tc pht thng bo ngun R v vn
tc truyn ti C (C l dung lng knh).
Cc khi nim c bn:
T m: l dy n k t truyn hay dy n k t nhn ng.
B m (S,n): l tp hp gm S t m vi di mi t m u bng n v c k hiu
l x(1), , x(s).
Lc gii m: l mt hm gn cho mt dy n k t nhn c yj mt t m ca b
m W = {w1, w2, , ws}. K hiu: g(yj) = wi
Lc gii m ti u: l lc gii m sao cho tng xc sut truyn sai l nh nht
hay tng xc sut truyn ng l ln nht.
Ngha l: khi nhn yj th ta gii m v wi* sao cho:
P(wi*/yj) = Max{P(wk/yj)}
mi wk thuc W
V d minh ha cc khi nim c bn
Gi s knh truyn tng bit vi C=1, ngun pht thng bo vi tc R=2/5 bit/giy
(R<C). thun li cho m ha v gim nhiu, ta xt tng khong thi gian n = 5 giy.
Nh vy trong khong thi gian n = 5 giy, ta c:
Tp hp cc tn hiu khc nhau l 2nR = 4. Gi s 4 tn hiu l m1, m2, m3, m4.
S bit c pht ra l nR=2 bit v mt tn hiu dng mi c kt cu bi mt
dy cc bit.
Qu trnh m ha cc tn hiu m1, m2, m3, m4 cnch l: mi mi cn c m
ha vi s bit ti a l nC=5 bit. Vy, ta c th m ha cc tn hiu mi theo 2
cch sau:
Cch 1:
m1=00000
m2=01101
70/135
m3=11010
m4=10111
Cch 2:
m1=00
m2=01
m3=10
m4=11
Nu s dng cch 1 vi di 5 bit, trong 5 bit c th hiu l c 2 bit thng tin cn
truyn v 3 bit con li l 3 bit c b sung pht hin nhiu theo mt phng php
no s c cp cc ni dung tip theo sau. Vi cch m ha ny, ta c nhiu
kh nng pht hin v sa sai do nhiu.
Nu s dng cch 2 th trng hp c 1 bit truyn sai s dn n trng lp sang mt
trong cc tn hiu khc. V d truyn m1=00 v nhn 2 bit l 01 (do nhiu), trong trng
hp ny 01 chnh l m2, y l mt tn hiu ng nn ta khng th pht hin c nhiu
hay khng nhiu.
Nh vy, trong khong thi gian truyn v dung lng knh cho php, ta cn m ha
mi tn hiu cng di cng tt nhng khng c vt qu di m cho php. Trng
hp vi thi gian n=5 v c= 1 bit th nC=5 l s bit ti a c th truyn nn ta ch m
ha tn hiu vi di m ti a l 5 bit.
Cc dng sai s c bn
Xc sut truyn sai t m xi:
Xc sut truyn sai trung bnh:
Xc sut truyn sai ln nht:
71/135
Phng php xy dng lt gii m ti u

Theo cng thc Bayes:
Ta c: P(wk/yj) = [p(wk).p(yj/wk)] / p(yj) vi (mi wk thuc W)
T nh ngha lc gii m ti u:
tm wk sao cho P(wk/yj) Max <=> p(wk).p(yj/wk) Max.
Nh vy, ta c th xy dng lc gii m ti u theo cc bc sau:
Bc 0: Khi to cc Bi = (mi i)
Bc lp: xt vi mi yj thuc Y
+ Tnh:
p(w1).p(yj/w1)
p(w2).p(yj/w2)
p(wM).p(yj/wM)
+ So snh cc gi tr tnh trn v chn gi tr w*i sao cho p(w*i).p(yj/w*i)= Max
{p(wk).p(yj/wk)} (mi wk thuc W)
+ Bi = Bi + {yj} v g(yj) = w*i.
Minh ha xy dng lc gii m ti u
Bi ton:
Cho ma trn truyn tin A v xc sut u truyn nh sau:
72/135
Vi p(x1)=1/2; p(x2)=p(x3)=1/4. Hy xy dng lc gii m ti u.

p dng phng php xy dng lc gii m ti u:
Bc 0: B1={}; B2={}; B3={};
Bc 1: Nhn gi tr y1, ta tnh:
+ p(x1).p(y1/x1)= 1/2.1/2 = 1/4 (Max)
+ p(x2).p(y1/x2)= 1/4.1/3 = 1/12
+ p(x3).p(y1/x3)= 1/4.1/6 = 1/24
Do p(x1).p(y1/x1) ln nht nn lit k y1 vo tp hp B1 tng ng vi x1.
=> B1={y1}.
+ p(x1).p(y2/x1)= 1/2 . 1/3 = 1/6 (Max)
+ p(x2).p(y2/x2)= 1/4 . 1/6 = 1/24
+ p(x3).p(y2/x3)= 1/4 . 1/2 = 1/8
=> B1={y1, y2}.
+ p(x1).p(y3/x1)= 1/2 . 1/6 = 1/12
+ p(x2).p(y3/x2)= 1/4 . 1/2 = 1/8 (Max)
+ p(x3).p(y3/x3)= 1/4 . 1/3 = 1/12
73/135

=> B2={y3}.
Kt qu:
Phn hoch: B1={y1, y2}, B2=
{y3} v B3={}.
Lc gii m ti u:
Minh ha cch tnh cc sai s
Xt li v d minh ha xy dng lc gii m ti u trn, ta c:
- Ma trn truyn tin A:
- Xc sut u truyn: p(x1)=1/2; p(x2)=p(x3)=1/4.

Lc gii m ti u:
74/135
- Phn hoch: B1={y1, y2}, B2={y3} v B3={}.

Tnh cc xc sut truyn sai:
Xc sut truyn sai mt t m:
Bi tp 1
Cho ma trn truyn tin sau:
Bit xc sut u truyn: p(x1)=5/10, p(x2)=3/10, p(x3)=2/10.

Xy dng lc gii m ti u.
Tnh cc sai s p(e) v pm(e).
75/135
Bit xc sut u truyn: p(x1)=1/3, p(x2)=1/3, p(x3)=1/3.

Xy dng lc gii m ti u
Tm cc sai s p(e) v pm(e).
Bi Tp 2
Bit p(x1)=1/2, p(x2)=1/4, p(x3)=1/4.

Bit xc sut truyn p(x1)=0.4, p(x2)=0.4, p(x3)=0.2.

76/135
nguyn l khong cch nh nht hamming

SA LI
Mc tiu: Xy dng nguyn tc sa li da vo khong cch Hamming. Trn nguyn
tc ny, phng php sa li kim tra chn l (parity check) c xy dng v to ra
quy trnh sa li ti u v ph hp vi cng ngh truyn tin hin nay.
NGUYN L KHONG CCH NH NHT HAMMING
Mc tiu:
Sau khi hon tt bi hc ny bn c th hiu:
nh ngha khong cch Hamming

Knh truyn i xng nh phn v lc gii m ti u
Quan h gia xc sut gii m v khong cch Hamming
Nguyn l khong cch nh nht ca Hamming.
Khong cch Hamming

nh ngha:cho v1 v v2 l 2 dy nh phn di n bit, ta gi khong cch Hamming gia
2 dy v1, v2 l s bit tng ng khc nhau. K hiu: d(v1, v2).
V d:
v1=10101010
v2=10101111
Ta nhn thy rng bit th 6 v bit th 8 gia gia v1 v v2 l khc nhau nn s bit tng
ng khc nhau gia v1 v v2 l 2. Do , ta ni khong cch Hamming gia v1 v v2 l
2 hay d(v1, v2) = 2
Knh truyn i xng nh phn v lc gii m ti u
Xt knh truyn i xng nh phn. Gi s ta truyn cc dy t m nh phn c di n
bits vi xc sut truyn sai 1 bit l beta.
1-beta
77/135
Gi W = {w1, w2,,ws} l tp s t m truyn, di mi t m u bng n bit.

V = {v1, v2,., v2n} l tp cc dy n bit nhn c cui knh vi W c phn phi
u, xc sut nhn vj khi truyn wi l p(vj/wi) = pij.
Theo lc gii m ti u ta c: khi nhn vj th gii m v wi* sao cho:
P(wi*/vj) = Max{P(wk/vj)}
(wi W)
Ta c: P(wk/yj) = [p(wk).p(yj/wk)] / p(yj) vi (wk W)
=> P(wk/yj) Max <=> p(wk).p(yj/wk) Max.
Do W c phn phi u nn P(wk/yj) Max <=> p(yj/wk) Max
Vy: tm wi* sao cho P(wi*/vj) = Max{P(wk/vj)} ta ch cn tm wi* sao cho
P(vj/ wi*) = Max{P(vj/ wk)} (ch da vo ma trn truyn tin A)
V d knh truyn i xng nh phn
Xt ma trn truyn tin A v xc sut u truyn nh sau:
da vo lc gii m ti u ta c:
Nhn v1 gii m v w1
Nhn v2 gii m v w3
Nhn v3 gii m v w2.
78/135
Quan h gia xc sut gii m v khong cch Hamming

Gi s nhn c v:
Xt 2 t m w1 v w2 cn chn gii m cho v.
Nhn xt: xc sut gii m cng ln th khong cch Hamming cng nh.
Nguyn l Hamming
nh l: trn knh truyn i xng nh phn vi s t m u truyn c di n bit,
lc gii m ti u c th thay th bng lc gii m theo khong cch Hamming
vi nguyn l: nu nhn c v, ta s gii ra w*i
sao cho d(v,w*i)=Min d(v,wk) (vi wk W).
V d: xt b m W={w1=00000, w2=10011, w3=11100, w4=01111}
Gi s nhn c dy v=01011.
ta c: d(v,w1)=3; d(v,w2)=2; d(v,w3)=4; d(v,w4)=1.
vy v c gii v w4 v khong cch Hamming gia v v w4 l nh nht.
Bi tp
1. Cho b m W={w1=000000, w2=101010, w3=111000, w4=111111} v nhn c
dy v=010111, khi gii m v t m no? din gii?
2. Cho b m W={w1=000000, w2=010101, w3=000111, w4=111111} v Nhn c
dy v=010111, khi gii m v t m no? din gii?
79/135
B v t sa li v cn hamming
t vn : mt t m w di n bit khi c truyn tun t tng bit c th sai e bit. Vn
t ra l khong cch (Hamming) gia cc t m v sai s e quan h vi nhau nh
th no c th phn bit tt nht ng thi tt c cc t m? B sau xc nh quan
h ny.
B :
Xt b m W={w1, w2, , ws} gm c s t m nh phn di n bit v 1 s nguyn dng
e.
Nu d(wi, wj) _>2e+1 (vi mi i khac j )
Khi : tt c cc dy nhn c v c s bit li <_ e th v c th t iu chnh (hay t
sa li).
Nu d(wi, wj) _>2e (vi mi i khac j )
Khi : tt c cc dy nhn c v c s bit li < e th v c th t iu chnh. Tt c cc
dy nhn c c s bit li = e th ta ch pht hin l v c li v khng th t iu chnh
c.
Ngc li;
Nu v c s ch s bit li <_ e v c th t iu chnh th d(wi, wj)_>2e+1 (vi mi i
khac j ).
Nu v c s ch s bit li <_ e-1 t iu chnh c v tt c cc tn hiu vi s ch s
bit li <_ e c pht hin th khong cch gia cc t m lun tha: d(wi,wj) _>2e (vi
mi i khac j ).
Chng minh v minh ha b
Gi s: d(w, w) _>2e+1 vi mi i khac j . Nu w v w c cng khong cch i vi
dy v th d(v,w)=d(v,w)_> e+1. Vy , nu d(v, w*) <_ e th v c th c gii m ra
w*.
Nu d(wi,wj)_>2e vi mi i khac j, c kh nng c v, w v w vi s ch s li l:
d(v,w)=d(v,w)=e (d(v,w)+ d(v,w) _> d(w,w)_>2e). C th pht hin ra cc t m gn
v, nhng do tn ti cng lc nhiu t m gn nht vi v dn n khng gii m c,
ngc li hon ton tng t.
80/135
Minh ha:
d(wi, wj)= 2e+1= 7, e=3
Nu vBi th v c gii m v wi
Nu vBj th v c gii m v wj
d(wi, wj) = 2e = 8 (e = 4, e - 1=3)

nu vBi , vBj => cc im cch tm khong cch 3 th lun c gii m, cn cc
im cch tm 4 th ch pht hin li ch khng th gii m c.
M 3 chiu (x, y, z) bt u t gc 000. C mt tn hiu thay i th m b y i theo 1
cnh, chng hn:
000 cch 010, 001 bi 1 cnh,
011 cch 010, 111 v 001 bi 1 cnh.
Nh vy, nu ta chn w1=010, w2=001, w3=111 th khong cch gia chng l 2
d(w1, w2)=d(w1, w3)=d(w2, w3)=2
yvy nu c li pht sinh th ch pht hin ch khng sa c.
Cn Hamming.
t vn : trong tng s 2n dy nh nhn di n bit c th chn ra bao nhiu dy to
thnh mt b m c th t iu chnh c e bit li. nh l cn Hamming cho chng
81/135
ta xc nh s t m c di n bit vi gi thit: c kh nng t sa c e bit li (iu

kin cn t sa li).
nh l: Nu b m W c s t m c di n bit c th t sa c e bit li th
Ghi ch: Cni = n!/(i!*(n-i)!)

Chng minh:
Xt t m nh phn wi c di n bit v c kh nng t sa c e bit li.
S dy vj sai khc vi wi t 0 n e bit l
:
Tng ng vi s t m, tng s dy vj c th t sa li l :
(2n l tng s dy nh phn di n bits).
Phn cc dng li
Gi s ta truyn t m n bit wi W ( 1 <_ i <_ s) v nhn c dy n bit vj ( 1<_ j <_
2n).
Cc loi li c th pht hin sau:
82/135
Li c th t iu chnh:
Trong trng hp ny tn ti duy nht t m w*i sao cho d(vj, w*i)= Min d(vj, wk) vi
miwk W.
=> vj c gii m v w*i
Li ch pht hin khng iu chnh c:
Trong trng hp ny tn ti t m w*i v w**i sao cho
d(vj, w*i)= d(vj, w**i)=Min d(vj, wk) vi miwk W
=> vj khng th gii m chnh xc.
Li khng pht hin c.
Trong trng hp ta gii m ra w*i nhng khc vi wi truyn.
Bi tp
Cho n=7 v e=2, hy p dng nh l cn Hamming cho bit s t m ti a ca b m
W.
Cho n=7 v e=2, hy p dng nh l cn Hamming cho bit s t m ti a ca b m
W.
Hy cho mt v d c th minh ha cc trng hp phn loi li.
83/135
m kim tra chn l

M KIM TRA CHN L
Mc tiu:
Hiu b m kim tra chn l,

Hiu phng php kim tra chn l,
Bit tnh cht c bn ca phng php kim tra chn l,
Hiu v vn dng tt phng php sinh m kim tra chn l,
Hiu v vn dng tt nh l quan h gia di m n, s bit kim tra m v s
li t sa e,
Vn dng cho cc bi hc tip theo.
B m kim tra chn l
B m kim tra chn l l b m gm s t m, trong mi t m c dng sau:
Ghi ch: trong mt s trng hp sinh m theo phng php kim tra chn l, th t
cc bit kim tra v cc bit thng tin c th xen k nhau (theo mt th t no , chng
hn nh m Hamming,) hay cng c th theo mt th t khc (theo quy c khc).
y, ta chn th t cc bit kim tra chn l v cc bit thng tin nh trn d tnh ton
nhng vn mt tnh tng qut ha.
Trong : w vit theo dong l chuyn v ca w (w c vit theo ct)
+ ri: l bit th i ca t m ( 1<_ i <_ n).
+ n: di ca t m hay s bit ca t m chn l.
+ m: s bit kim tra.
84/135
+ k = n-m: s bit thng tin => s=2k (v vi k bit thng tin th ta ch c th biu din ti
a 2k trng thi thng tin k bit).
+ on kim tra: gm m bit dng kim tra m sai.
+ on thng tin: gm k bit thng tin.
Mi on m thng tin c duy nht mt on m kim tra v c xc nh bi h
phng trnh tuyn tnh nh phn sau:
Gi
Ma trn A c gi l ma trn kim tra chn l c hng l m (hay Rank(A) = m).

Cc php ton trong Modulo 2 (+,-):
0 + 1 = 1 + 0 = 1; 0 1 = 1 0 = 1;
1 + 1 = 1 1 = 0;
Phng php kim tra chn l
Gi w=r1r2rn l t m truyn (hay dy n bit truyn) v v=r1r2rn l dy n bit nhn
c.
Qui c:v, w (ln lt l chuyn v ca v v w) c vit theo dng. Cn v, w c
vit theo ct.
Nu A.v = 0 th v = w, ta gi v l chn (trng hp nhn ng)
Nu A.v khc 0 th v khc w, ta gi v l l (trng hp nhn sai).
Ta gi z = v-w l b li gia v v w. Ngha l ti cc v tr z = {0} th bit nhn c
tng ng l bit ng v ti cc v tr z = {1} th bit nhn c tng ng l bit sai (hay
bit li).
85/135
Ta gi C = A.v l b sa li (hay b iu chnh li).

Ta c C = A.z = A.(v-w) = A.v-A.w = A.v => C = A.v = A.z
Tnh cht ca b sa li: dy n bit nhn c v v b li tng ng c cng b iu
chnh.
Phng php sinh m kim tra chn l
Gi s: cho trc ma trn kim tra chn l A vi Rank(A) = m.
Tm b m chn l W={w1, w2, w3,,ws}
Bc 0: Xc nh cc gi tr n, m, k, s
di ca t m n= s ct ca ma trn A.
S bit kim tra m= s dng ca ma trn A.
S bit thng tin: k = n-m.
S t m s=2k ca b m.
Bc i: Tm cc t m th i (1<_ i <_ s):
Gi kpi l trin khai nh phn k bit ca s i
T m cn tm l: wi=r1r2..rmkpi
Gii h phng trnh A.wi=0 tm m bit kim tra ng vi k bit thng tin (kpi) bit
=> t m wi
V d sinh m kim tra chn l
Xy dng b m kim tra chn l c sinh t ma trn kim tra A nh sau:
Bc 0 :
86/135
n=6 (= s dng ca ma trn A)

m=3 (= s ct ca ma trn A)
S bit thng tin k = n m = 3 => S t m s=2k=8 t m.
Bc i: Tm t m th i (1<_ i <_ s):
w1=r1r2r3000 (000 l trin khai nh phn k=3 bits ca s i=0)
Gii h phng trnh A.w1=0
Gii h phng trnh A.w2=0
Gii tng t cho cc trng hp cn li ta c:

87/135
w0=000000, w3=110011, w4=110100,

w5=111101, w6=001110, w7=000111.
=> W={000000, 001001, 111010, 110011, 110100, 111101, 001110, 000111}
nh l quan h gia di m n, s bit kim tra m v s li t sa e
iu kin cn (Cn Hamming):
iu kin cn b m chn l c di n bit c th t sa c e bit li vi k bit thng
tin v m bit kim tra l:
iu kin ( K Vasharmov-Gilbert-Sacks):
iu kin b m kim tra chn l c di n bit vi m bit kim tra chn l c th
t sa c e bit li l:
c
Ghi ch:
V d tm m nh nht t n v e
Gi s bit trc n=7 v e=1. Tm s bit kim tra ti thiu cn thit ca b m chn l.
Theo nh l iu kin cn (Cn Hamming):
88/135
Vy s bit kim tra ti thiu cn thit l m = 3.

V d tm e ln nht t m v n
Gi s cho trc m=3, k=2. Tm s bit li ln nht c th t sa e?
Theo nh l iu kin (K Vassharmov-Gilbert-Sacks):
Vy s bit li ln nht c th t sa l e = 1.
Bi tp
Xy dng b m kim tra chn l c sinh t ma trn kim tra A nh sau:
Tm b m kim tra chn l c sinh t ma trn kim tra A nh sau:
Gi gii bi tp 1 & 2: da vo phng php sinh m kim tra chn l v tham kho
v d sinh m kim tra chn l.
Xt b m kim tra chn l di 15 bit c th t sa c 1 bit li trn ng truyn,
hy cho bit s bit kim tra chn l ti thiu?
Xt b m kim tra chn l di 8 bit vi 4 bit kim tra chn l. Hy cho bit s li t
sa ti a ca b m?
Gi gii bi tp 3 & 4: da vo inh l iu kin cn (Cn Hamming) v iu kin
(K Varshamov-Gilbert-Sacks).
89/135
nhm cng tnh v b t m chn l

NHM CNG TNH V B T M CHN L
Mc tiu.
Hiu Khi nim nhm cng tnh,
Bit cc tnh cht ca b m chn l,
Vn dng sinh ma trn kim tra chn l t b m kim tra chn l.
Vn dng tt phng php sinh b m kim tra chn l t cc t m c lp tuyn tnh
ca b m.
Khi nim nhm cng tnh.
t vn :
Nh chng ta bit, phng php sinh m kim tra chn l gip ta sinh b m kim
tra chn l vi s t m tng ng l s=2k. Vi phng php ny, ta phi xc nh tng
t m mt (bng cch gii h phng trnh tuyn tnh nh phn). Gi s: k=5 ta phi xc
nh s=25 =32 t m hay k=10 ta phi xc nh s=210=1024 t m,iu ny s mt
nhiu thi gian nu k cng ln. Vn t ra y l tm ra mt phng php sinh b
m kim tra chn l nhanh hn v mt thi gian. Phng php sinh m kim tra chn l
da theo l thuyt nhm s gii quyt vn ny.
Khi nim nhm cng tnh:
Nhm G c gi l mt nhm cng tnh nu G c cc tnh cht:
Nhm G l nhm hon v (nhm Aben) nu a,b thuc G=> a + b = b + a.
V d:
90/135
- Tp hp cc s nguyn vi php + thng thng l nhm Aben.

- Tp hp cc s nh phn c di n bit cng vi php + trong Modulo 2 to thnh
nhm Aben.
Tnh cht ca b m chn l
Tnh tng ng ca b m nhm cng tnh v b t m kim tra chn l c th hin
qua 2 nh l sau:
nh l 1: tp hp cc t m trong b m kim tra chn l l mt nhm cng tnh.
( ngh sinh vin chng minh nh l ny da vo cc tnh cht ca nhm cng tnh)
nh l 2: Nu tp hp W l tp cc dy nh phn vi di cc dy cng bng n v W
l mt nhm Aben vi php cng Modulo 2 th W c th xem nh mt b m kim tra
chn l c sinh ra t ma trn A c dng nh sau:
Trong :
- Ma trn A c m dng v n ct.
- Im : l ma trn n v cp m.
- k: l s dy nh phn (hay t m) c lp tuyn tnh ln nht.
- n: l di ca t m v m = n-k:
- bij: c xc nh bng cch da vo h phng trnh tuyn tnh (*) v k t m c
lp tuyn tnh nh sau:
on kim tra on thng tin
91/135
Th k t m c lp tuyn tnh vo h pt (*) tm cc bij => ma trn A.

V d minh ha
Xt tp hp M gm c 8 dy nh phn di 6 bits nh sau:
r1 r2 r3 r4 r5 r6
w0 = 0 0 0 0 0 0
w1 = 1 0 1 0 0 1
w2 = 1 1 0 0 1 0
w3 = 0 1 0 1 0 1
w4 = 0 1 1 0 1 1 (w1+w2)
w5 = 1 1 1 1 0 0 (w1+w3)
w6 = 1 0 0 1 1 1 (w2+w3)
w7 = 0 0 1 1 1 0 (w1+w2+w3)
Ta thy {w1, w2, w3} l tp hp ln nht cc t m c lp tuyn tnh t tp hp M:
w1 = 1 0 1 0 0 1
w2 = 1 1 0 0 1 0
w3 = 0 1 0 1 0 1
=> n=6 v k=3. => m = n k = 3.
Nh vy: ma trn kim tra chn l c dng nh sau:
92/135
Vy ta c th s dng nhm M nh l mt b m kim tra chn l.

Phng php sinh m kim tra chn l nhanh
Bc khi to:xc nh cc gi tr n, m, k, s.
Bc 1: sinh k t m c lp tuyn tnh (ltt).
Bc 2: cng t hp cc t m:
+ Cng cc t hp ca 2 t m t k m ltt => c t hp chp 2 ca k t m.
---+ Cng cc t hp ca k t m t k t m ltt => c t hp chp k ca k t m.
Bc 3:Cng s-1 t m tm c tm t m cui cng => t hp chp 0 ca k= 1
t m.
93/135
V d sinh m kim tra chn l nhanh

Tm b m nhm khi bit trc ma trn kim tra
Bc khi to: n = 6, m = 3, k = 3, s = 2k = 8.
Bc 1: Sinh k = 3 t c lp truyn tnh: w1=001001, w2=111010, w3=110100
Bc 2: Cng t hp cc t m.
+ Cng cc t hp 2 t m ltt:
w4=w1+w2=110011
w5=w1+w3=111101
w6=w2+w3=001110
+ Cng cc t hp 3 t m ltt:
w7=w1+w2+w3=001111
Bc 3: xc nh t m cui cng:
w0=w1+w2+w3+w4+w5+w6+w7=000000
Bi tp
S dng phng php sinh m nhanh cho b m t ma trn kim tra A nh sau:
94/135
S dng phng php sinh m nhanh cho b m t ma trn kim tra A trong cc trng
hp sau:
95/135
lc sa li ti u
LC SA LI TI U
Mc tiu
Bit c vn ca bi ton,
Hiu nh ngha Hip hp,
Vn dng xy dng lc sa li theo cc hip hp,
Vn dng xy dng lc sa li thng qua b sa li,
Vn dng tnh Xc sut truyn ng cho lc sa li,
Kin thc t c s l c s cc bn c th ng dng cho vic thit k mt
h, thng m ha, gii m v bo mt thng tin.
t vn
Trong mt h thng lin lc truyn tin, bn cnh cc yu cu thit b (nh ngun pht,
b m ha, knh truyn, b gii m,) m bo tt cho vic truyn v nhn d liu th
cn c cc kha cnh khc nh phng php m ha v gii m sao cho ti u l phn
rt quan trng trong h thng. Vn lun c t ra y l lm th no ch ra
mt phng php gii m ti u, c ngha l h thng phi c kh nng pht hin v sa
li mt cch chnh xc nht c th c khi nhiu xy ra. y chnh l vn chnh c
tho lun trong sut bi hc ny.
nh ngha Hip hp
Gi W={w1, w2, ,ws} l b m kim tra chn l.
V ={v1, v2, , v2n} l tp hp cc dy n bit c th nhn c cui knh.
Ta gi mt hip hp ca W trong V l tp hp c dng z + W (z l b li)
V d: Cho ma trn kim tra chn l sau:
T A, ta c th xy dng c b m tng ng sau: W={w0=0000, w1=0101,

w2=1110, w3=1011}.
96/135
Ta c th thy rng, cc b li mt bit khc nhau c th c l z={1000, 0100, 0010,

0001}. Do cc hip hp ng vi cc b li 1 bit s l:
Trong : hip hp i = wi + zi, cc bn c th xt thm cc b li sai 2 bit, 3 bit,
c cc hip hp ng vi cc b li sai 2 bit, 3bit,.
Lc sa li theo cc hip hp
Bc 1: Lp bng cc hip hp ng vi cc b li cn thit
Dng u tin vit cc t m wi thuc W.
Cc dng tip theo ng vi ct w0 = 0000 vit cc b li z (cc b li 1 bit, 2
bit,).
Cc dng ct th i c xc nh bi z + wi
Bc 2: Qu trnh gii m
Gii m: khi nhn v, ta xc nh ct th i cha v v gii m v wi tng ng.
V d: xy dng lc sa li theo cc hip hp cho b m c sinh t ma trn kim
tra chn l sau:
T A, ta c th xy dng c b m tng ng sau: W={w0=0000, w1=0101,

w2=1110, w3=1011}.
Bc 1: Lp bng cc hip hp ng vi cc b li cn thit:
Ta xy dng cc hip hp ng vi cc b li sai 1 bit. Vy z={1000, 0100, 0010, 0001}.
97/135
(Bng cc hip hp)

Bc 2: Qu trnh gii m:
Gi s nhn v = 0111. Tra tm v trn bng cc Hip hp ta c v ct 1. Do , v c
gii m v w1 = 0101.
Gi s nhn v = 1010. Tra tm v trn bng cc Hip hp ta c v ct 2 hay ct 3. Do ,
v c gii m v w2 hay w3, trong trng hp ny gii m khng chnh xc. ngh
cc bn lu v cho kin ca bn v cc trng hp gii m khng chnh xc ny.
Lc sa li thong qua b li
xy dng lc sa li thng qua b sa li, ta da vo tnh cht ca b sa li.
Nh vy ta c th thy lc gii m gm 2 bc sau:
Bc 1: Lp bng sa li: B li (Z) B sa li (C=A*Z).
Bc 2: Qu trnh sa li
Khi nhn c dy n bit v thuc V, ta xc nh b iu li C cho v vi C=A.v
Tra bng sa li tm b li z0 ng vi C.
Gii m w=v+z0.
V d minh ha lc sa li 1 bit
Xt b m c sinh t ma trn
B m tng ng c xc nh l: w1=000000, w2=101101, w3=111010, w4=010111
( ngh cc bn tham kho phng php sinh m chn l v xy dng li b m t ma
trn kim tra chn l A).
98/135
Lc sa li:
Bc 1: Lp bng sa li: B li- B iu chnh (e = 1)
B li (z) B iu chnh (C=A.z)

Bc 2: Qu trnh sa li
Gi s nhn v=001101, tnh C = A.v = 1000
Tra bng sa li tm b li z0 ng vi C, ta c z0 = 100000
Gii m w = v + z0 = 001101 + 100000 = 101101 = w2
Lc sa li:
99/135
010100 0101
(Tt c cc b 2 li cn li c trng b iu chnh vi cc b trn)
Bc 2: Quy trnh sa li
Gi s nhn v=100111, tnh C = A.v = 1100
Gii m w = v + z0 = 100111 + 110000 = 010111 = w4
Lc sa li:
(Tt c cc b 3 li cn li c trng b iu chnh vi cc b trn)

Bc 2: Quy trnh sa li
Gi s nhn v=011001, tnh C = A.v = 1101
Gii m w=v + z0 = 011001 + 110100 = 101101 = w2
Ch :
Tng s b iu chnh = 2m. Trong mt s trng hp, b m chn l ch cho php pht
hin li trn ng truyn v khng th gii m chnh xc do tng s b iu chnh =
2m v s b li c th ln hn nhiu (so vi tng s b iu chnh).
Xc sut truyn ng
Gi Ni l s b li ng vi i li c th t sa, khi xc sut truyn ng v t iu
chnh s l:
100/135
Vi n l di t m
V d: xt trng hp cc v d trn vi n= 6 v t sa e = 3 bit li. p dng cng thc
trn ta c:
Bi tp
Cho ma trn kim tra chn l sau:
Xy dng b m kim tra chn l.

Minh ha quy trnh sa li 1 bit.
T kt qu ca bi tp 1, hy minh ha lc sa li thng qua b iu chnh trong
cc trng hp li 1 bit, 2 bit. Tnh xc sut truyn ng cho cc trng hp c th t
sa c.
101/135
m hamming
M HAMMING
Mc tiu
Hiu M Hamming,
Hiu tnh cht ca m Hamming.
M Hammin
M Hamming l mt dng m nhm (m kim tra chn l) c xc nh t ma trn
kim tra chn l A c dng sau:
Ct th j ca ma trn A l biu din nh phn m bit (m l s bit kim tra chn
l) ca s j theo qui c biu din nh phn ca s j c vit theo th t t
di ln trn (vit theo ct), tng ng vi vit t tri sang phi (vit theo
dng).
Cc bit v tr 2i ( i = 0, 1, 2, ) c chn lm bit kim tra.
V d 1: biu din nh phn ca s j = 3 c m = 3 bit nh sau:
Vit theo dng: 011 (vit t tri sang phi)
Vit theo ct:
(vit t di ln)
V d 2: ma trn kim tra chn l vi n=6, m=3 c th vit nh sau:
102/135
T m Hamming c dng: w=r1r2r3r4r5r6. Trong , r1r2r4 l cc bit kim tra v r3r5r6

l cc bit thng tin (v cc bit v tr 2i (vi i = 0, 1, 2, ) c chn lm bits kim tra).
Tnh cht
Nu cho trc s bit (m) v s bit li t sa (e) th s bit ti a ca b m Hamming (n)
c th c c lng t bt ng thc sau:
V d minh ha
Tm b m Hamming vi n = 6 v m =3
Ta c th vit ngay ma trn kim tra chn l cho b m Hamming
T A => k = n m = 3.
Cc bit cc v tr 1, 2, 4 c chn lm cc bit kim tra.
=> s t m ca b m Hamming l s = 2k = 8
Tm k t m c lp tuyn tnh c dng:
w1=r1r20r401
w2=r1r20r410
w3=r1r21r400
Gii cc h phng trnh: A.w1=0, A.w2=0, A.w3=0
Cc t m cn li c xc nh theo phng php sinh m nhanh.
Ghi ch: Kt qu chi tit xy dng bng m Hamming dnh cho sinh vin t lm.
Bi tp
103/135
Vit ma trn kim tra chn l cho b m Hamming vi n = 15.

T kt qu bi tp 1, hy tm cc t m Hamming c lp tuyn tnh tng ng.
Xt b m Hamming vi s bit kim tra cho trc l m, khi :
di m ti thiu l bao nhiu?
di m ti a l bao nhiu?
104/135
thanh ghi li tng bc

THANH GHI LI TNG BC
Mc tiu
Sau khi hon tt bi hc ny bn c th bit:
t vn v thanh ghi li tng bc,

Cch biu din vt l ca thanh ghi,
Cch biu din ton hc ca thanh ghi,
Tm chu k ca thanh ghi.
t vn
Nh chng ta bit, phng php sinh b m kim tra chn l da trn l thuyt nhm
cho php chng ta sinh m nhanh bng cch ch sinh ra k t m c lp tuyn tnh trong
tng s s=2k t m, t k t m ny ta c th xc nh cc t m cn li (bng cch cng
t hp cc t m). Vn t ra y l lm sao tm ra mt phng php sinh m
khc sao cho s t m sinh ban u nh hn k (k l s t m c lp tuyn tnh ca b
m kim tra chn l) v t y ta c th xc nh nhanh cc t m cn. C th da trn
m hnh ca thanh ghi li tng bc c th gii quyt c vn ny.
Biu din vt l ca thanh ghi
gi mt cch ngn gn, ta qui c gi thanh ghi thay v goi thanh ghi li tng bc.
Biu din vt l ca thanh ghi c th thy nh hnh v di y:
Fm-1, Fm-2, , F1, F0 : cc bit lu tr d liu nh phn.

am-1, am-2, , a1, a0 : cc cng tc (switch) dng ng (=1) hay m ( =0
: l b lm tnh cng trong php ton mudulo 2 sau mi xung ng h vi d
liu do cc bit ca thanh ghi gi v.
105/135
Qu trnh dch chuyn li tng bc: sau mi xung ng h th d liu trong bit Fi s
c chuyn v lu tr bit Fi-1 (F1-> F0; F2->F1; ; Fm-2->Fm-3; Fm-1->Fm-2). Tt
c cc gi tr trn cc Fi (trc khi c xung in) s c chuyn v b cng (ty theo
cc cng tc ng hay m), tng ca cc gi tr ny s c a vo lu tr bit F m-1.
Ta s nghin cu thanh ghi ny c th hn trong cc ni dung tip theo nhm tm ra mt
phng php sinh m m ta c th gi l m xoay vng. y cng l mt dng m kim
tra chn l.
Biu din ton hc ca thanh ghi
Mc tiu ca vic biu din ton hc l tm ra cc m hnh tnh ton phc v cho vic
nghin cu sinh m xoay vong chn l t thanh ghi.
Gi x = (x0, x1, , xm-2, xm-1) l gi tr cc bit ca thanh ghi ti thi im trc khi c
nhp xung ng h.
x = (x0, x1, , xm-2, xm-1) l gi tr cc bit ca thanh ghi sau khi c nhp xung ng
h.
Khi ta c:
x0=x1
x1=x2
x2=x3
---------xm-2=xm-1
xm-1=a0x0 + a1x1 + + am-1xm-1.
Hay vit theo dng ma trn ta c x = T.x
Trong :
106/135
T c gi l ma trn c trng ca thanh ghi li tng bc.

Qu trnh dch chuyn li tng bc ca thanh ghi:
Gi
l vc t ch gi tr ca thanh ghi ti thi im ang xt.

Gi tr ca thanh ghi sau 1 xung ng h l x(1)=T.x(0)
Gi tr ca thanh ghi sau 2 xung ng h l x(2)=T.x(1)=T2.x(0)
----------V d thanh ghi lui tng bc
Cho thanh ghi lui tng bc sau:
T thanh ghi, ta c: m=4, a0=1, a1=0, a2=1, a3=0.

Ma trn c trng ca thanh ghi
107/135
Chu k ca thanh ghi

Nh trnh by trn v qu trnh dch chuyn li tng bc ca thanh ghi:
Gi tr ca thanh ghi sau 1 xung ng h l x(1)=T.x(0)

---------------Gi tr ca thanh ghi sau n xung ng h l x(n)=T.x(n-1)=Tn.x(0) (bi v s trng thi
thng tin khc nhau c th c l 2m)
Vy chu k ca thanh ghi l s xung nhp ng h thanh ghi lp li trng thi ban
u. Ngha l nu x(0)khc0 v tn ti n>0 sao cho x(n) = x(0) th ta ni n l chu k ca
thanh ghi.
Lu :
Cch vit biu din nh phn cho gi tr ca x(i) theo th t t trn xung (theo ct),
tng ng vi vit t tri sang phi (theo dng). V d: biu din nh phn ca x(i) = 3
c m = 3 bit nh sau:
Vit theo dng: x(i) = 011 (vit t tri sang phi)
Vit theo ct:
108/135
(vit t trn xung)

V d tm chu k ca thanh ghi
Cho thanh ghi lui tng bc nh hnh sau:
109/135
Tm chu k:
Bi tp
Tm cc chu k ca thanh ghi lui tng bc nh hnh sau:
Tm cc chu k ca thanh ghi lui tng bc nh hnh sau:
110/135
111/135
m xoay vng
M XOAY VNG
Mc tiu
Bit cch xc nh ma trn kim tra chn l cho m xoay vng (hay cn gi l
m vng),
Hiu nh ngha m xoay vng,
Vn dng xy dng b m xoay vng,
Vn dng phng php sinh nhanh b m xoay vng sinh b m kim tra
chn l.
Ma trn kim tra chn l m xoay vng
nh ngha: ma trn kim tra chn l c thit k t thanh ghi li tng bc l ma trn
c dng sau:
A=[x(0)| T x(0)|T2 x(0) ||Tn-1 x(0)] vi n l chu k ca thanh ghi (n > m)
Trong :
T l ma trn c trng ca thanh ghi.

x(0) 0: l gi tr khi to ca thanh ghi.
n : l chiu di ca t m v cng l chu k ca thanh ghi.
m: l s bit kim tra hay s bit ca thanh ghi.
V d: xt li v d tm chu k thanh ghi, nu chn gi tr khi to ca thanh ghi l x(0)

= 1 th ta c ma trn kim tra vi chu k n=6 nh sau:
nh ngha m xoay vng
112/135
M xoay vng l m kim tra chn l c sinh ra t ma trn kim tra chn l ng vi
chu k n ca thanh ghi li tng bc c dng nh:
A=[x(0)| Tx(0)|T2x(0) ||Tn-1x(0) ]
V d: xt li ma trn kim tra chn l trn
Ta c n = 6, m = 3, k = 2 => s = 2k = 22 = 4 t m.
p dng Phng php sinh m nhanh b m kim tra chn l ta c b m kim tra chn
l gm 4 t m sau : w0 = 000000, w1 = 101010, w2 = 010101, w4 = 111111, y chnh
l mt trong cc b m xoay vng sinh t thanh ghi li tng bc nu trn (Cc bc
sinh m nhanh ngh cc bn t lm)
Phng php sinh nhanh b m xoay vng
Cch sinh nhanh k t m c lp tuyn tnh ca b m vng t a0,a1, a2, , am-1:
Bc 3: xc nh cc t m cn li ca b m
113/135
Cc t m cn li gm (2k k t m) c xc nh bng cch cng t hp ca 2, 3, ,

k t m t k t m c lp tuyn tnh trn.
V d sinh nhanh b m xoay vng
Cho thanh ghi lui tng bc nh hnh sau:
T thanh ghi, ta c: m=4, n=6, a0=1, a1=0, a2=1, a3=0.

Bc 1: Sinh m xoay vng u tin
w1=101010
Bc 2: Sinh k -1 t m c lp tuyn tnh cn li
w2=010101
Bc 3: Xc nh cc t m cn li ca b m
w3 =111111 (w1+w2), w0 =000000 (w1+w2 + w3)
B m vng va sinh l W={000000, 101010, 010101, 111111)
Bi tp
Cho thanh ghi li tng bc sau:
Tm ma trn kim tra chn l c s ct n > 4

T kt qu cu a, xc nh b m xoay vng tng ng.
Tm b m xoay vng theo phng php sinh nhanh b m xoay vng
Cho thanh ghi li tng bc sau:
114/135
Tm ma trn kim tra chn l c s ct n > 4

T kt qu cu a, xc nh b m xoay vng tng ng.
Tm b m xoay vng theo phng php sinh nhanh b m xoay vng.
115/135
a thc c trng ca thanh ghi

A THC C TRNG CA THANH GHI
Mc tiu
Hiu nh ngha a thc c trng ca thanh ghi,

Hiu Quan h gia chu k n, a thc c trng v a thc (xn + 1),
Vn dng sinh thanh ghi li tng bc,
Lm c s vn dng sinh b m vng.
nh ngha a thc c trng ca thanh ghi

nh ngha:a thc c trng ca thanh ghi c ma trn c trng l T l a thc c dng
gm(x)=a0 + a1x+ a2 x2+ +am-1xm-1 + xm.
vi a0, a1, a2,, am-1 l cc cng tc ca thanh ghi v m l s bit ca thanh ghi
V d: xt li thanh ghi nh hnh sau:
a0 = 1, a1= 0, a2 = 1, a3 = 0
a thc c trng ca thanh ghi c dng: gm(x)=1 + x2 + x4.
Quan h gia chu k n, a thc c trng v a thc (xn + 1)
a thc c trng ca thanh ghi gm(x)=a0 + a1x+ a2 x2+ +am-1xm-1 + xm lun chia
ht a thc (xn + 1).
V d: xt li thanh ghi lui tng bc nh hnh sau:
116/135
T thanh ghi ta c th xc nh cc kt qu sau:

a0 = 1, a1= 0, a2 = 1, a3 = 0
a thc c trng ca thanh ghi c dng: g4(x)=1 + x2 + x4.
Thanh ghi ny c chu k n = 6.
Thc hin php chia a thc (x6 + 1) : (1 + x2 + x4)= (x2 + 1) => chia ht.
Ghi ch: php ton trn a thc nh phn vn l php ton Modulo 2.
V d: xt li thanh ghi lui tng bc nh hnh sau:
a0 = 1, a1= 0, a2 = 1, a3 = 0
a thc c trng ca thanh ghi c dng: g4(x)=1 + x2 + x4.
thanh ghi ny c chu k n = 6 v (x6 + 1) : 1 + x2 + x4 = x2 + 1.
Th tc sinh thanh ghi li tng bc
sinh thanh ghi li tng bc vi s bit l m v c chu k n, ta c th thc hin theo
cc bc sau:
Bc 1: xc nh a thc c trng ca thanh ghi
Tm 2 a thc gm(x)=a0 + a1x+ a2 x2+ +am-1xm-1 + xm
v hk(x)=h0 + h1x+ h2x2 + +hk-1xk-1 + xk sao cho (xn + 1) = gm(x)* hk(x).
Nu tn ti (xn + 1) = gm(x)* hk(x) th ta chn gm(x) lm a thc c trng cho
thanh ghi (v s bit kim tra ca b m l m) v thc hin bc 2.
Ngc li: khng tn ti thanh ghi theo yu cu.
117/135
Bc 2: v thanh ghi
T gm(x)=a0 + a1x+ a2 x2+ +am-1xm-1 + xm => a0, a1, a2,, am-1 => thanh ghi c
dng:
V d minh ha
Thit k thanh ghi c m=3 bit v chu k n=7, ta thc hin theo 2 bc sau:
Bc 1: Xc nh a thc c trng ca thanh ghi
Ta c (x7 + 1) : (1 + x2 + x3) = (1 + x2 + x3 + x4)
Do m=3 nn chn g3(x) = (1 + x2 + x3) lm a thc c trng ca thanh ghi.
Bc 2: V thanh ghi
T g3(x) = (1 + x2 + x3) ta c, a0=1, a1=0, a2=1
Bi tp
1. Trong cc thanh ghi sau y, thanh ghi no sinh ra b m vng c di n=15
bit?
118/135
Nu cc bc cn thit thit k b m xoay vng di 15 bit vi s bit kim tra l

V s thanh ghi dng tng qut.
119/135
phng php sinh m xoay vng

PHNG PHP SINH M XOAY VNG
Mc tiu
Hiu cc phng php sinh m vng,
Bit bng lit k mt s a thc c trng,
Vn dng sinh m vng theo nhiu cch khch nhau.
t vn
sinh b m kim tra chn l, ta c th da theo nhiu phng php khc nhau nh:
sinh m da theo l thuyt nhm, m Hamming,... Vn t ra y l lm sao
sinh b m xoay vng vi di n bit v m bit kim tra chn l. Phng php sinh m
xoay vng da trn l thuyt v a thc c trng nh phn ca thanh ghi gip ta c ci
nhn tng qut v vn sinh b m xoay vng theo nhiu cch khc nhau.
Phng php sinh bng m xoay vng
sinh m xoay vng di n bit vi m bit kim tra v k bit thng tin, ta c th thc
hin theo cc bc sau:
Bc 1: tm 2 a thc gm(x)=a0 + a1x+ a2 x2+ +am-1xm-1 + xm
v hk(x)=h0 + h1x+ h2x2 + +hk-1xk-1 + xk sao cho (xn + 1) = gm(x)* hk(x).
Nu tn ti (xn + 1) = gm(x)* hk(x) th chuyn sang bc 2
Ngc li khng th sinh b m vng theo yu cu.
Bc 2: ta c th sinh b m xoay vng theo cc cch nh di y:
Cch 1: Chn a thc gm(x)=a0 + a1x+ a2 x2+ +am-1xm-1 + xm
=> a0, a1, a2,, am-1
=> thanh ghi => ma trn c trng T
120/135
=> chu k n => ma trn kim tra chn l A.

=> B m xoay vng.
Cch 2: chn a thc gm(x)=a0 + a1x+ a2 x2+ +am-1xm-1 + xm
=> a0, a1, a2,, am-1
=> Sinh nhanh k t m c lp tuyn tnh vi t m sinh c lp tuyn tnh u tin c
dng: w1=a0a1a2am-1100000 => B m xoay vng.
k-1 bit 0
Cch 3: chn hk(x)=h0 + h1x+ h2x2 + +hk-1xk-1 + xklm a thc sinh ma trn kim
tra chn l cho b m vng c dng:
=> Sinh b m xoay vng theo Phng php sinh nhanh b m xoay vng.
Nhn xt: kt qu theo 3 cch sinh b m xoay vng ni trn la nh nhau (cho cng b
m).
V d minh ha 1
121/135
122/135
=> B m xoay vng vin=14, m=4, k=11.

V d minh ha 2
Chn a thc gm(x)= 1+x+x4 => a0 = 1, a1 = 1, a2 = 0, a3 = 0.
Bc 1: Sinh m xoay vng u tin
w1 =110010000000000
Bc 2: Sinh k -1 t m c lp tuyn tnh cn li
w2 =011001000000000
w3 =001100100000000
w4 =000110010000000
w5 =000011001000000
w6 =000001100100000
w7 =000000110010000
w8 =000000011001000
w9 =000000001100100
w10=000000000110010
w11=000000000011001
Bc 3: Xc nh cc t m cn li ca b m
(215 - 11) t m cn li c xc nh bng cch cng t hp 2, 3, 4,.., k = 11 t m t
k=11 t m c lp tuyn tnh.
V d minh ha 3
Chn hk(x)= 1+ x + x2 + x3 +x5 + x7 + x8 + x11lm a thc sinh ma trn kim tra chn
l cho b m vng => h0 = 1, h1 = 1, h2 = 1, h3 = 1, h4 = 0, h5 = 1, h6 = 0, h7 = 1, h8 =1,
h9 = 0, h10 = 0.
123/135
k mt s a thc c trng
Bi tp
Tm b m vng c di 7 bit.
Tm thanh ghi sinh b m vng c di 15 bit.
Tm thanh ghi sinh b m vng c di 31 bit.
124/135
bi tp tng hp
BI TP TNG HP
Mc tiu
Hiu r hn v ni dung mn hc.
Vn dng ni dung mn hc gii quyt mt s bi tp tng hp.
Bi 1
Xt mt m hnh chn on bnh t cc triu chng: A, B v C; chn on 1 trong 4
bnh: 1, 2, 3 v 4 vi ma trn chn on (hay ma trn truyn tin).
Yu cu:
Cu 1: V s m t m hnh chn on bnh trn v din gii cc ngha ca s .
Cu 2: Nu phn phi ca Triu chng c dng:
Tnh cc lng sau :

Lng ngu nhin (Entropy) ca Triu chng .
Lng ngu nhin ca Bnh.
Lng ngu nhin ca Bnh khi bit Triu chng.
125/135
Lng chn on ng.(Lng thng tin bit v Bnh thng qua Triu chng) v t l
chn on ng l bao nhiu phn trm.
Cu 3: By gi ngi ta s dng 2 bit m thng tin v Triu chng (c 1 triu chng
d tr) v 5 bit m cc triu chng khi chn on bnh trc tuyn. M t cc on
ca dy 5 bit trong phng php kim tra chn l.
Cu 4: Nu s dng ma trn kim tra chn l dng:
Tnh cc t m.
Xy dng B sa li 1 bit dng cho t ng sa li ti u trong qu trnh chn on trc
tuyn. Cho mt v d.
Bi 2
Xt mt knh truyn tin c bit dng : Truyn X -> Nhn Y.
Truyn mt gi tr ca X c th nhn c nhiu gi tr khc nhau ca Y vi cc xc
sut khc nhau. Bng xc sut truyn X v nhn cc Y khc nhau c cho di y:
Yu cu:
Cu 1: V s m t knh truyn tin trn v din gii cc ngha ca s .
Cu 2: Nu phn phi ca X c dng :
126/135
tnh thng lng v X truyn trn knh.

Cu 3: Phn phi ca X cn c dng nh th no thng lng truyn trn knh l ln
nht. Tnh dung lng knh truyn.
Cu 4: By gi ngi ta s dng 2 bit m thng tin v X v 4 bit m cc gi tr
truyn trn knh. M t cc on ca dy 4 bit trong phng php kim tra chn l.
Tnh cc t m.
Xy dng B sa li dng cho t ng sa li ti u trong qu trnh truyn tin. Cho mt
v d.
Bi 3
Ngi ta cn nh gi knh truyn tin v chun b thc hin truyn mt loi tn hiu c
bit: X = {x0, x1, x2, x3}
Cng vic u tin l phi kho st knh truyn. Kt qu kho st cho thy:
Knh c th truyn nhn c 8 gi tr khc nhau, c kh nng pht hin li hoc
iu chnh li. Ma trn truyn tin c dng:
127/135
Yu cu:
Cu 1: V s m t knh truyn tin trn v din gii cc ngha ca s . Nu phn
phi ca X c dng :
tnh thng lng v X truyn trn knh.
Cu 2: Phn lp cc gi tr ca Y v cc lp B0, B1, B2, v B3 dng gii m ti u Y
tt nht v cc gi tr tng ng ca X.
Cu 3 : By gi ngi ta s dng 2 bit m thng tin v X v 4 bit m cc gi tr
truyn trn knh. M t cc on ca dy 4 bit trong phng php kim tra chn l.
Tnh cc t m.
Xy dng B sa li dng cho t ng sa li ti u trong qu trnh truyn tin. Cho mt
v d.
Bi 4
Xt mt m hnh chn on bnh t cc triu chng: A, B v C; chn on 1 trong 4
bnh: 1, 2, 3 v 4 vi ma trn chn on (hay ma trn truyn tin)
Yu cu:
Cu 1: Gi s ngi ta bit thm 3 triu chng gy bnh khc l : D, E v F v mun
ghi li cc triu chng ny thng qua bng k hiu A = {+, - }.
Hy kim tra tnh tch c ca bng m sau :
128/135
Cu 2: Nu cc triu chng cu 1 c phn phi :
Gi s c mt ngi bnh vi 1 trong 5 triu chng trn n khm bnh v bc s s hi

bnh vi nguyn tc, sao cho ngi bnh ch tr li bng 2 cu : ng hoc Sai.
Tm phng php hi bnh vi s cu hi trung bnh t nht.
Tnh s cu hi trung bnh.
Tnh lng ngu nhin ca Triu chng.
Nhn xt g v s cu hi trung bnh v lng ngu nhin ca triu chng.
Cu 3: By gi s dng m hnh 3 triu chng {A, B, C} v 4 bnh. V s m t m
hnh chn on bnh v din gii cc ngha ca s .
Cu 4: T kt qu cu 3, ngi ta s dng 2 bit m thng tin v Triu chng (c 1
triu chng d tr) v 5 bit m cc triu chng khi chn on bnh trc tuyn. M t
cc on ca dy 5 bit trong phng php kim tra chn l.
Tnh cc t m.
129/135
Tham gia ng gp
Ti liu: gio trnh l thuyt thng tin
Bin tp bi: duongvanhieu
URL: http://voer.edu.vn/c/f89a66f8
Giy php: http://creativecommons.org/licenses/by/3.0/
Module: gii thiu tng quan gio trnh l thuyt thng tin
Cc tc gi: phantantai, lequyetthang, duongvanhieu
URL: http://www.voer.edu.vn/m/30e410f1
Module: yu cu
URL: http://www.voer.edu.vn/m/9b6d8aa4
Module: ni dung ct li
URL: http://www.voer.edu.vn/m/9600f1ba
Module: kin thc tin quyt
URL: http://www.voer.edu.vn/m/de994fe4
Module: phng php hc tp
URL: http://www.voer.edu.vn/m/4ccd5e4c
Module: gii thiu
URL: http://www.voer.edu.vn/m/8f0b05d3
130/135

Module: m hnh l thuyt thng tin theo quan im Shannon
URL: http://www.voer.edu.vn/m/9c2f478f
Module: nh l c s ca k thut truyn tin
URL: http://www.voer.edu.vn/m/a129c076
Module: khi nim v dung lng knh truyn
URL: http://www.voer.edu.vn/m/43d5e8c1
Module: o lng tin
URL: http://www.voer.edu.vn/m/c96ed136
Module: cc tnh cht ca entropy
URL: http://www.voer.edu.vn/m/9f8faa55
Module: entropy ca nhiu bin
URL: http://www.voer.edu.vn/m/fe891a68
Module: minh ha cc entropy
URL: http://www.voer.edu.vn/m/1b1bf672
131/135
Module: O LNG TIN (MESURE OF INFORMATION)

URL: http://www.voer.edu.vn/m/9c96388e
Module: SINH M TCH C (Decypherable Coding)
URL: http://www.voer.edu.vn/m/5ef4b4d3
Module: quan h gia m tch c v di m
URL: http://www.voer.edu.vn/m/308cad63
Module: tnh ti u ca di m
URL: http://www.voer.edu.vn/m/f7ece9b2
Module: knh truyn ri rc khng nh
Cc tc gi: phantantai, lequyetthang
URL: http://www.voer.edu.vn/m/949250e5
Module: cc dng knh truyn
URL: http://www.voer.edu.vn/m/601a2f4a
Module: lc gii m
URL: http://www.voer.edu.vn/m/fe4959d1
Module: nguyn l khong cch nh nht hamming
132/135

URL: http://www.voer.edu.vn/m/75f79adc
Module: B v t sa li v cn hamming
URL: http://www.voer.edu.vn/m/7ed3305d
Module: m kim tra chn l
URL: http://www.voer.edu.vn/m/a819e6b1
Module: nhm cng tnh v b t m chn l
URL: http://www.voer.edu.vn/m/0747bad7
Module: lc sa li ti u
URL: http://www.voer.edu.vn/m/51dd4e88
Module: m hamming
URL: http://www.voer.edu.vn/m/1dda7d8e
Module: thanh ghi li tng bc
URL: http://www.voer.edu.vn/m/b201ac43
Module: m xoay vng
133/135
URL: http://www.voer.edu.vn/m/5a903fdf
Module: a thc c trng ca thanh ghi
URL: http://www.voer.edu.vn/m/e8f5e086
Module: phng php sinh m xoay vng
URL: http://www.voer.edu.vn/m/e7d5ae9f
Module: bi tp tng hp
URL: http://www.voer.edu.vn/m/8d3d974e
134/135
Chng trnh Th vin Hc liu M Vit Nam

Chng trnh Th vin Hc liu M Vit Nam (Vietnam Open Educational Resources
VOER) c h tr bi Qu Vit Nam. Mc tiu ca chng trnh l xy dng kho
Ti nguyn gio dc M min ph ca ngi Vit v cho ngi Vit, c ni dung phong
ph. Cc ni dung u tun th Giy php Creative Commons Attribution (CC-by) 4.0
do cc ni dung u c th c s dng, ti s dng v truy nhp min ph trc
ht trong trong mi trng ging dy, hc tp v nghin cu sau cho ton x hi.
Vi s h tr ca Qu Vit Nam, Th vin Hc liu M Vit Nam (VOER) tr thnh
mt cng thng tin chnh cho cc sinh vin v ging vin trong v ngoi Vit Nam. Mi
ngy c hng chc nghn lt truy cp VOER (www.voer.edu.vn) nghin cu, hc
tp v ti ti liu ging dy v. Vi hng chc nghn module kin thc t hng nghn
tc gi khc nhau ng gp, Th Vin Hc liu M Vit Nam l mt kho tng ti liu
khng l, ni dung phong ph phc v cho tt c cc nhu cu hc tp, nghin cu ca
c gi.
Ngun ti liu m phong ph c trn VOER c c l do s chia s t nguyn ca cc
tc gi trong v ngoi nc. Qu trnh chia s ti liu trn VOER tr ln d dng nh
m 1, 2, 3 nh vo sc mnh ca nn tng Hanoi Spring.
Hanoi Spring l mt nn tng cng ngh tin tin c thit k cho php cng chng d
dng chia s ti liu ging dy, hc tp cng nh ch ng pht trin chng trnh ging
dy da trn khi nim v hc liu m (OCW) v ti nguyn gio dc m (OER) . Khi
nim chia s tri thc c tnh cch mng c khi xng v pht trin tin phong
bi i hc MIT v i hc Rice Hoa K trong vng mt thp k qua. K t , phong
tro Ti nguyn Gio dc M pht trin nhanh chng, c UNESCO h tr v c
chp nhn nh mt chng trnh chnh thc nhiu nc trn th gii.
135/135

Giáo Trình Lý Thuyết Thông Tin

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Giáo Trình Lý Thuyết Thông Tin

Uploaded by

Copyright:

Available Formats

gio trnh l thuyt thng tin

gio trnh l thuyt thng tin

Phin bn trc tuyn:

gii thiu tng quan gio trnh l thuyt

kin thc tin quyt

Phn phi xc sut ca s xut hin ca cc tn hiu.

m hnh l thuyt thng tin theo quan im

t BNN Y l BNN v s u hnh m c sau 2 ln tung. Khi ta c th xc nh

Phn phi ca Y khi bit X=2 c dng:

nh l c s ca k thut truyn tin

Minh ha k thut gim nhiu

Vy truyn sai khi Y thuc {2, 3} c xc xut l:

khi nim v dung lng knh truyn

Do lm sao ua khi tin nhn c v khi tin ban u 01010101, y chnh l

Hiu c cc khi nim Entropy,

R rng, ta cn phi xy dng mt i lng ton hc rt c th c th o c lng

Khi nim entropy

Nu gi Ai l s kin X=xi, (i=1,2,3,..) th Entropy ca Ai l: h(Ai)= h(pi)

nh l dng gii tch ca Entropy

Qui c trong cch vit: log(pi)= log2(pi)

Xt BNN X c phn phi sau:

H(X) = H(1/2, 1/4, 1/4) = -(1/2log(1/2)+1/4log(1/4)+1/4log(1/4)) =3/2 (bit)

Trong : x1, x5 ln lt l tn ca 5 ngi m ta cn nhn ra vi cch xc nh tn

Bi ton v cy tm kim nh phn - Din gii

0.2, 0.3, 0.2 ta ch cn tn 2 cu hi.

tm x4, x5 vi xc sut tng ng 0.15, 0.15 th ta cn 3 cu hi.

Tnh H(Y) vi phn phi sau:

cc tnh cht ca entropy

H(p, 1-p) l hm lin tc theo P.

Ta c th gom cc s kin x1, x2, x3 li thnh mt s kin mi l x123 c xc sut xut

n y cc bn c th p dng cng thc tnh, so snh cc Entropy v nhn xt tnh

R rng H(p, 1-p) l mt hm lin tc theo p.

Khi , ta c H(p1, p2, ,pM)=

Tnh H(X), H(Y).

Cho bin ngu nhin X c phn phi sau:

Ta c th gom cc s kin x1, x2, x3 li thnh mt s kin mi l x123 c xc sut xut

- Tnh entropy ca X, X* v kim tra li tnh cht 3.

entropy ca nhiu bin

Hiu bit cc nh ngha Entropy ca nhiu bin v Entropy c iu kin,

nh ngha Entropy ca nhiu bin

Mt cch tng qut:

V d Entropy ca nhiu bin

- H(X,Y) =H(0.125, 0.25, 0.125, 0.125, 0.25, 0.125)=2.5 (Bit)

Entropy ca Y vi iu kin X xy ra c nh ngha l:

Phn phi ca Y c iu kin X:

Entropy ca Y/X=1 v Y/X=2 nh sau :

Sinh vin t chng minh

Phn phi ca Y c iu kin X:

Tnh cc Entropy sau: H(X), H(Y).

t bin ngu nhin Y l s u hnh m c sau 2 ln tung:

Phn phi ca Y khi nhn c ng tin c 2 mt u c u hnh (Y/X=2)

Minh ha Entropy H(X), H(Y) v H(X,Y)

= -(0.5)log(0.5) -(0.5)log(0.5) = 1 (bit)

Tng t: H(Y,Z/X), H(Z/X,Y)

O LNG TIN (MESURE OF

Bit bi ton tnh lng tin,

t bin ngu nhin Y l s u hnh m c sau 2 ln tung. Khi ta c th xc nh

Trng hp 2: Phn phi ca Y khi bit ng tin l gi (X=2) c dng:

Ta c th tnh d dng phn phi ca Y nh sau:

Nhn xt da theo entropy

Nhng ngi dn ca mt lng c chia lm 2 nhm A v B. Mt na nhm A chuyn

SINH M TCH C (Decypherable

Bit yu cu ca bi ton sinh m,

Huffman (1950) a ra qui trnh xy dng mt bng m ti u tha yu cu ny.

Gi s W* ={w1, w2, , wM-2, xM-1,M} l bng m ti u ca X. Khi :