I1 fl

1
f l \
I
1 /
I
l.h: I m; "ln fur r;i; Lm n 1~1
r:1, l , • J",
I 1
111
•1 \\' ~ .-,
, r •
•l c:r •' r r, , , " . 11 1 , 111 tJT\l,' ... ' l h ~ •
~ tJ,:
Wl rn{ ' .' rr: "' ~ I 1 •• r ' ,. l•t,: ' \ ,,,le',' r.,,, ,,,
I 11'~11 ... !J_1, .,p
i I]
, i►
.... ·ron ~ 1)1• 1 1J f11m 1 111L n\ i v l1 tit ' 111 )
1L , u1 •.•~, 1Luo:c- nr ·_p ItL ....
,,n LIIHt•~· , n.1:,.. rn !lt,c:;
~-
111 • '\' Il k' ~rn:::fh o., •• 1c '-'" ~11 i;f IHfi 1, , ,,h n, ~ I:! IP' .!II b,:( ,,,\ •
ton c r"'r .m,1 ,,1 b~ fl ,::: t;11lrr.r , ·I lnh.111nn'l ,('1!1 lh~
l CI f ;-.: hL
t ,( I • 11 'Nl
1,.' .c U
"' IN \\ luat 1, ,11~ unl l u L L··1
U 1• •' 1h~ 111e~m n~ o:- C. D, p

ho " ' m i.cl ,

,e tu. ~~ u,~11 ,~,, , hl'a t~ Hm ad u p . l 1rr1
f,t.' \ .i) ti) f w ~ li1H c:c: t •• ,,.e .. 'rJ I\
r , .... r~r r , ,J ••
tr ~•u •' •1 I• l,11 • lu.t
',::- ,1 ,
-...u _ , ._ _ •
Lhi,e r. .. c.: gr11 JI.!'\ ., lnllld i:
A.- ..\. '
u ltK • 1 rn1om u)l i110 ll1c:-,,n1 t 11 0 ut' in h 1a J,, ~
111 1
,j T Ille ·ui
1.. 111{ ,,nn a,l .. m
,.., '1-1,il ,; ,lc. h, ~~)
\ ,-nu ,.;h
Un w 1111
m1d , ~li ~rn:h n11 t T.lld e-, u•<t 1 u1n1,;.h ln l1~ mu\10 fl do •

l
_yn I ym ,r 1!' .~dd Iv 11ol I I h) hQ ,,,
m.:..rh L ,.., ftc 11 Iha cc..u:.i\u &cll.!1
rt!,,l 'lll"' .: ...0 ► rl I
1 ,w :) lUH C' ",,;\.:.( :t.lJ i,; I
S S 11,.,~ ,1,~~h11. ,,d, i 111• I
h hl:' !iH 1fx ,I I S o'a ~1
,t, LC) Ei..>.p l_J ,n ?IH !'."•1111\k:
1 11!• ,~
mu 111 illy l't;c - 1\1.1 1\·..e t-H•
l 1i 11>d in.: 1l.i rm..ll_ \l !\ l\
"t I \ 1 I/
t• ,..n .
{ Lli) Dlilfi11c h11 n 1lly uu
knc ,~rJ 1-, t.:\1 1

J.• u ruh, fur <tllr,,p, II(:-, i IJ j ' ,rt• I I

, !<I i'r1m 1· llm Lli•
11 t!,J inJ 7 ~ - i:u 1 a .d , l1 ~:... r;.
cu ,', J 1 : ~ 1 , r ml ,.m~ i"'·~, 11 .
·l<;H , ,\ ct,n W ns ,,,~ ~d ;,,
,, p H,u
~ U\ll1 I• ,r •n d ,. grC:~[1 b~,l
lt r,ir th re~ liuJ,,C[. n.r ...
, ,- \. 111t! IJ • 1.1.
~ Ul lJJ nd , nru1 u bu:t J, .• w
.__ and lnM
\•lu l IL,L , \hor t••Q lai , p ,l[J
((Mk; rrro u1 . S ot11cun ~ \I Utn b Lho::
,,1 1, 111\J h ( t h ,II t-= h1
LH h i:s ~l i.:.cn .\ h.1l
bull rrlH ri It . lurc h~ m,,m ""= . rT1t, ',/.'~ ..lt 12 1\ ~
h.._1m l}o ~ J\, h, tlw. c;i,. 1w r ,111
pr1,h,1 !1il It.) 1h ; ,1 ,,. , Llr•.w.:r. i\, f! h ri~•rt ' '" t ,c-d b;;' 1~ 11
ii~ rJ • f~t. l lJ di " 1 I 1c:- ~h ,:,-.:-11'.? \\ l .:il I\!
rirL•CfJ l 11
1hc 1m 1t1.
.
an: d r.i.-.11 \.\ll hou r
ciLn11 J Yy1.1,, ,w b.h Jk Th ree
tb) ; ~ ~.,g -.. n1,t,·11n.5 5 Z"'"t.l,. \ ~ .,.re
r1•pl. 1~L!mc.11 r d 11Jw ,rn.U \lolH b,:.: i.f r;,\\ ',1,
l1litl Jll1 lj ll11tl 11 t &.:J , .1 tJ.l i;r;: 11 111,~t u
Wlr .. t ' "' th~ p-ro
grc~n.
11 Wh n1 h, 1hr r o hnb ,ht
l rh.1 t tllc l11Et ~ull dru \\11 ,.,,·ill ln.~
Q l l. Eva Juatc {-~I) J

Q I:!. Th~ device m,ed in ii d~ltJ i;umm11J~cllcir-11 ni:t\~~•rk h1 ~rfn rm \h'-7

i.:ait\lt'rsion
ben, een c!.nalogue and dighn' sib11.i~s. -~.i1l 1~d rn1fr~·, n• ~L)J pr0..: i:•5~n r th) moo
t:·m.
l i,;)dec.odt:r (d) cuulti plc'\cr
Q 1J. The unit of Channel capw:it~ .~,-- - ·-- ~--~
 11 It
th, r
1
tlh U 11# t\ ti t

Of HH flt I I c ~,
I h ,,
I ,.,....,, pcd)lhllllltl

a,
\hltU\\ tth Ut

OU \\ h 1 ' lh
...."""' I. jili-1 fNMn • fflllSIP
• pratMlbUll ~ c t X p)
t' , \ htt , i r11 urc dr wn with nrt
I) I J ball I~
h:p I \ cmcn1 \\Ihm , 11 Ill
rn ll l ~1

Q, ( l\ (1) I ~- I
• - • , P nu~ Shu11non drlinhJcm Selrln(, "1\1111 ~n IU l Wh tl 1~ Iha.: 11 lll\C llSC'-1
[\>1 llus sell mfo1mmlo11 b) ,.._.

( h}, l),e line It1fc1nn,ttion i,t1t,~ wuJ ~eri\e ttw funnuhu

(c) Find lhe in lbml.3lio11 comem (IJ cteh \mtx,l ml~ al~u th.; enuup) (11) of lhc ~oun:e
belo,,;

1I
•
 l n) I I . ._. ,rn mI H
'"hc,c .tJi: 11 •t•htth hk I I ) 11 .., 11111 r I
1
~ h.J \ l 1U \ 1 "' it I (I ) h t\~ 111
t:>..J.., t r ci '1\:
,~u!l (if\l'! !\,U( 1).1 J
\., I I pr>t Tl
Lu t.:UtHc I h X) I md I ll Inf• ,11 .all H1 L111 U \H II
I Ul>I I)1 C UtUpalh; lh t"iC ,. ll~lU.:"i ,, ltlt L ,pi. t, I 1
1l

(..lJ(a l In u i:1..!n.un p1..,ptl!(Jtu,n O' •t 1h-. ,,c, r It it, \ t .1 r 1 tj t t n

'J oh'f\• C choJ· e,:a I n ,u,~r·11 h:•U: 8° o I tl\ c: • ~flf.: rnui t,ub~ n '" 1
j l 1 •n Ll i
40 0
hu\ c buhunic pJngtJ\." dnd i.:l 1Ldetu 1111d J~i'., hlWC t1U t 1 t "
\\ hat r e11..;cn1a~c nr the population h.1~ rn,inc • f Lhc lhrec h J

(b) A bag A CCH~~11ns cJc·v1.;11 r~d r:md ~c, en gr1::i.:11 ball._ U ~(11•Hun inur 11;d
b.dl,. and ( coinaal'-. I\\O red rn1d six green bulls. I l1e du,...: u~ ire p1d\o.'-!.d in llir ro m '
n tnn .sturnbfes rn10 thi: l(lon1 , nd grope uic,uncl LO [inJ u bag. dra\a. ~ u haH lr ,m u und l~~n c!I
the roon 1• Me ~ooJ .... at ,Ju~ bal.. 11 is green. \\ hut is the prob~tbthL) that ~t \\\1 dra,\n ir 1l'1 haf.!
1\. Jn lhis 1txperjmcn1 \Vl1w is 1he prol:Mbiht\ 1hn1 u 1~u ball \\ ill b~ cl u-sl!rf! Whal i~ \ht 1

pr~11,onion of rct:l bal~s in lht> rocm1'? ·

04, {a) 0)\Vria~ the for1111dd fo1 ~he an1ount ofinfonr1.uio1,) L, gained frt11,1 n n1c5c;u~-=
(ii) \Vlun is 1he def1ni1io11 ornh: entropy H(X) or u probabBi\ 1 ~pacl! tX .. p)?
(Hi) Under ,vhut coudilio11s is lhl;! entrupy ~ hX) equal to zero·~

(bJ. ,. l"'incl Jhe infurmBtion cona~nt (l) 1n cac·h S)tnbuls ,n the titiu1cc hclo,v Ltnd alsu lhc
~t, ._., 1n ~ ,h.~ ~, o•,r,p lt..:hrnv \,vnrk.in~is)
 U~JI 'j I 0\\ 1 h
i ...
J~~ JIU ( I lO\:s,
' .\Ill rti,u
Sce1ion A ( \ m,n t•t •U>

0 f I nnr htit S)n

r ~?(ind 1g ·1 1 I ifi ~l
(1 bif..,: 't111 Ii ti

Q A {rau~. ~UJ I cJ1(lun~I firt

(~> ov.s9 106 bu ,. ., (b)\s 111

Q3, &tropy or a hia11try .smm::

{l,9S9 (d) 0.5~Q c'

Q4 Om: Hanley ft,

- oil r J 1
QS. AUl1riliJ1 to N}q1,1 t'~ tl ,
1..hanrn:I for' n grvc11 bm1cl\\ idjh it;,
freqimncy (cJ:f\, ice hjghest 5i , 1 ~

Q6. { ~ LUliJS llk------~----~-----

Q7. Lo,g L.inttr; are-------- -

QS. Lo~ units ure-----·-------

Q9. I og 1 uni s re calk ~ ·- ~

QIO Log, urti1s ~, calla:d--,........... =a--.--_ _ _ _ _

)J L E:u11uate r Zv
JI.., ned C U!led in il d lo r . . .,,. . . ,. t
ntl dti-rim sr nn
r d n u tlpfexer
I• •)::. 1.1)1 1.' Dc11nr: " l11 lonn111ion u1eorv"
/ (ii) Who is rhc Fathcr o f l11fo~~11ion 11,cory'/
b ) 111) )("What is the 1lllc o f1hc: po...,.r
1 ~. G'"ft
,-- he wro•-·, •· ~ ll1c. )'<'llr .. 1 p u b ltC'1l1\1
( ( ~. • 1~C lhrc:,· npplicolion., ofco 111m11nlc11llon lhcllr\ ri
c) ~11 ) \\ .nte ~n : fl) on the or igin "r infunnntion 1hc1>1) (mu, 111 lln
( I he l11 s1.:1r1c on11 ulu by 1hc Father or lnfo mrntion th. · •I c:~J
C ~ 11 log i( I - P/ N) ,my"!,; vc n t->cln y. :
Give rhe mcu n ing <> f C . n. p nnd N. \\'lmt b the unit l)f (.''
QJ. 1:i) \\ 'hut 1s the e ntropy H. in bits. o f the followin 5
C ~ J have the proba.bilities shown? 1/4 1/8 If.! l/ 8 g . ourcc: a lphabet \\hCJsc h:ttc:r,
•
l ,\ 11
(b) Sketch n lhe
in fonnatio . Venn dmgram showing the rel 11t ions
' ·h'tp be· twcen entropy and. m utual

(c) A so urce gcnc ratt:.s three symbol~ w· h b T

symbo ls per second Assuming indc. ~ pro ab1 ,~, 0.25, 0.25. 0.50 at a ratc of JOOO
encoder ""ould have average bit ratc:i:; cnt generation of symbo ls. the moSt etlicient source

Q4 (a) no
W~itc: shon note o n the follo wi ng. (i) Ergodic process
ise
(ii) self info rmut·1on
• (iii)
(b) Consider figure I below:
( j) \Vhich area of information theory is figure associated'?

(l l 11n1,tho1Q;Jl
l_ 1•··•~ Ull )

•· n, • > i,llu u t clen >pU 111;

M c-ssagc M - - ~ , C..- c ~! - . ~ - <-- ", D-d(l.. . ~ ~
'=f"' puh llc chu nn.,I '-,-/ \ ~I

l<cy K • ' L · \:ii

"'l"C1tra• c ha,vu,1 o:: 1

{ii) What are the entities involved in the infom1a1lon process'?

(iii) Wit h speci lie detai ls., analyse the l1ow(s) or iuformation fro m the sender to th.:
rcec:i ,,er
(iv) \\/hat ·'could possibly be the Amount of infonnation ( I) gained from the
messnge seni"
(v) State n clear 1em1s ·'the: func ti on o f the K'. in the model

Q5(a ) "lnfonnation theory is the study of attainable or realizable bounds for

communication and it is largd y probabilistic and analytical in nature.'' Explain this
su1ernent in the ligbl of sending and receiving of messages; Hence, what are the major
"delivery detenninar ts" of a message (from sender to
rece iver) .............. ......... ............... ..Smks
(b) Given thal Ll1•! following alphabetical list cumulates into certain bits/sci!
A = J /2. B= I '4. C= 1/8, D= 1/8, K= I/32, M = 1/4 , R "' I/ 16, Calculate the urnowll of
in formation , , : (i) MARK & (ii) ADA.BA in bits of information (4 marks)

(c)
For O:::~n. v. hat is (nk) to the least tenn.