Chapter 7 Estimation Theory 7a INTRODUCTION In ths chapter, we presenta canal eximstion thor. There ae two ase type of etimation probleme ne type ve inert in etatng be praise saree a the Senn ype, we ate nteraid So esimating the vale fap inscribe rw. Yin term of the ‘bseration of an ass r X 172. PARAMETER ESTIMATION Let X be ae. with pfs) and Xy, ns 2st of independent reac with pl /l). The et of sh Tandon same (ot sample eter) of ne maf X- Any esos ction seabed» tae ‘Le ich depends ona unknown parameter 8 Let (Xy...%) Bea random sample of Xs hs a, he oh pel Xs gen By Ayman on where ny. Ax arte vas ofthe observed data taken rom the random sample ‘An cimatr oi any Sse sy. Xl dened ORD 0 For a gata et of servations 2) = Xe th va ofthe einste 5) wll Ica sp eimate of and dented by Thus a existor war anon etna jst Feazaton of ie not necesny that ah etinat of» parame be one ingle vue feted he {stmat cold bea range of aes Estes Which spel) sage Vl af led pon extn, nd etnater wich spy range of ene secede eat. Ss )=10 8) 72 PROPERTIES OF POINT ESTIMATORS A. Untied Estat Anetitor © = 4. Xe nd ob a bid eimaor othe parame Fe) =6 ws {or al publ values a 010 a uniued eximator, hen it meen guar ero ven by 10 ~ 97} = £{[© ~ 201") = Varo) on “Tatas mean equ ror oui arian iit Etinatre: ‘Animator ©, it ido bes reece eniator ofthe parameter 8 than the estimator @, 1. @,and © are both unbaedeximators of a728 ESTIMATION THECRY ccna 7 2 Venlo.) < Vax. “Tae instr Oyy = 4X Ky dt Be 3 ost ficient oF mimi sarin unbiased ‘ximstor the parameter 8 1. isan embed ett of 2 VaxOy) Vax) fal © Contes Estimator: ‘The eststor ©, of bused ona random sample f sien si tobe coment fray sal on lim 118, -81<0) os) ov equtalely, im MIO.~8i>a=0 v8 “The following two condition ae sic! to dine conten (Prob. 79 1. tm HO) = on 2 im vate =0 oa 14 MAXIMUM-LIKELIHOOD ESTIMATION Lat fu 8) = fli dete be oat pen te 4 Xan Xa whe hey re lee, net tenis phen they ae coningot, Let UO) = 06, = fle os [Now Uf) rerests the itibood thatthe values, Xl observed when 8 the ee ae tthe purarver Ths Li cen weld tothe liken fit of ne random spe Let Buc aby. be te miming sae of LO} tat Hoya) x UO) 10) ‘Then the masimum-tketood estimator of it Oe = AX Kd en) and the maimumbenood exinae of ‘Shee 10 product eer pop wl nae be pov or the range of posite vats of Thus te L( eao always be deed, and in determing he asim vue a 8 te ‘fen ll 1 ae thet tt Lt st Te 1) ave te mime a the same vl of 8 Hens, emay aio cbs By by meximiig In LB, 15 BAYES’ ESNMATION Suppose hatte unknown prarter@ consered t bec, baing some fed ditibuton ‘ox pt pao) -Teen 0:0) nom ewe att eandnona pl snd wien /(r0h nd me can ‘pet he oat pe tbe random ape (Xe and 8 $8 BS oy SLB) oun,nar. 7 matioy tHroRy Fr andthe margial ofthe sample given by Serene emery nen an eA uo foo bn ‘eens our iformatinn abot Baer having obese “The consional mean 9, ened by tee Hn abe sn sod un ‘sealed be Bayes oxime of, end BOK an Xe cu8, Ieee he Rayer enimator of 116 MEAN SQUARE ESTIMATION 1 thie section, we dal wth he cond pe of etintion problema i tinting te value ofan aacecsblet Yin em ofthe oberon ofan aco Xo ger be einator F GLY gen by 4 scion of AX) Then YY ~ 900 ie eal the eninatin aor, tad ‘here 8 cu ios with hs enor CLY ~ gi) We ae need in ding the foto 2) That imines thi crt. When X and ¥ age contnaoue cv the meen sare a ema sce ‘ed asthe et fonction, LY = x= £10 — ax) an 1: ct be shown thal the etinator of ¥ given by (Prob. 117, Paa)= YI) an) lathe bet eximatri the ete hat the ms. er defied by (7.17) 8 mini, 17 LINEAR MEAN SQUARE ESTIMATION Now conser the estimator of ¥ pen by Pagar se ous We wou te 0 ic te values ofan Bauch thatthe err defied by = BLY ~ PI IEY aX +60) 20 ‘sminimam, We masta tat and ros be such that (Pro> 7.20) B{CY —(ex + nx} 0 e2n sod and bare piven >y per DE y= Oh wy24 ESTIMATION THEORY rowar 9 ap the minim ms: eo Pr. 722) ten eel par 29 ster on CoM, Yan poy ee orton coc of Xan Y. Noe ht Be 2) te (at +0) is onbogona othe observation X. This known atthe eshogonay pine, Te ne Yan bison calle ragetin ine Nast, we conde the enmator F of ¥ wih « linear combintion cf the stagom sample yonkdby 29 ‘esi, we msinain ta in onder to produce the inst etinstor wih he misma ms ro the ‘ficients us este tha the followieg orthogonally condone are std (Prob 735) Alr-Bax)efee jetone 3s Solving Bo. (229 fo. ae 726 here and Rise ver of R ayn bakit Solved Problems [PROPERTIES OF POINT ESTIMATORS 1. 1X) bea andor sample of having unknown meth Show tha te etna of ior oan ‘san voiaed eimator of Note that Xu known athe sample mean (Pro. 464) By Ba (410m nan =o(2 Sox)ad fm ‘Thay M ian ete nitro FL Le ky X) bea random sample of having unknown mean and vane, Show hat Westar ee died ty selina 28)HAP 71 ESTIMATION THEORY a where Pie he sample ean a tasdetinator of Feet = ay ft fea ]nefl Sn nt} t= nst-t} af Eb ot ero 1 Bax,~9~ eu ~ By Ean can 727 we be etemn if om Thue an0 a hat in an entre? FR. Ley XD Be andom sample a Poison ry X wth vknown parameter (@) Stow tat LEK, and Ae diy are both uit etinstor of (0) Wich inate ore ctisent? 1s) By 2anang tion wehae aye! $a sym =f) bewueneteia amt vara 400 hah ite> 2A, ea moe at ima ota A oe Hn a FA Lan hQ bea ndom ape of X wth etn id vacant oA nar einer of ‘Seed 1 be a eer Topcon f Xow, Nyse - Xo) Show thatthe lear esto ‘dene by fe 27. Bx22 ESTIMATION THEORY ona mean ‘ean elt laa hue int an M, St M, nie et ave y= Eatry= aE eh imps at. = 1 1200, verte Let and Vette elt or fare! om Comer hese ced (ved) eg sBe-tae! (vated ehh by ampin 7.30 tha Thi iin n= I pig Dat M her Show that lim B@)=8 and im Varo,) ‘ben tbe etinatr ©, inconstent Using Cheaters ingen (297, ween ate Jee, mess 203- 6) =A Hue, ne? + tno a +16, - Neste) = oven + eQHeg- 0} +2400,-FeKEe) a) Imno)=# sd tgvanoy=0 en union sizame hai ©, coment ne 18 Lay 9 Bea candor sample «worm rove ah where i knows Show ‘tat ally Kas KD 3p ‘i aconssea estimator ofthe parame 0our. ESTIMATION THEORY Ey 10 ner dtd one, hen om Ea 248 248 4569 Po, 30 he pl tanenantraar=2()" ocsce oy hor ta [‘torteed [eae toe “~ wasn we se ane Tha ae and 7 Ai coment ciate parame MAXIMUNLLIKELIHOOD ESTIMATION AI. at Oy Be racom smple ofa Nnomil £8 wih parameters (9h bee mi ‘sound ibe known and p unknown, Determine the minor -iMenod eniate f ‘ett om nets oine(Zpr-are (Spur OQ ate str oan eaves w ruremer Earn fata @ ean 38 TR Leon bea rndom sample of Poison. ith unknown parameter 1. Deter bea ssuMan enar 7 ‘ee bteoed ton es yg (240) fannie EE Th ube -atstarSantne ~ funnel fn Seng i Uta =O, emanation int of oie at wat ba | ee ey ‘Deere the meximim celnood eanator of The toad oneness by 8g (28) Bye feyo2s De Ths nuda nnd abe, on Nace kn ond 09 TAO Let yu be rnd sample of normal random r¥X with unkown eth and ‘koe anaes Determine te asia tebood eximator ef and” Teaibodfneton i en y [5 (252) no) enon tam ff Ee nf = Sst nF] “Fatcura ESTIMATION THEORY 25 - ee Singh tt tn ca eg han En . tude! Bie. Ba? wy ones, mania ebod eimai and te epee By ek ean ate} faa om DAL Let js Xp Bee anon sample of Bemoull tv. X wih pm gen by (Ba (2320 foripre pli=p* xe Qt waa where 9.0.2 7 unknown. Assume that pi orform rove (0, Find the Baye stair op “Ter lp te worm pata Jm1 O del MD — saalet fee 8 + tual fall 15 35) ben set Uae os 1728. Coes normal with sine 146 nd aay sean Fad be erent conc tie Bcmess dona eaten pi fa 1 Aeshown in Po 716 fet = 095 we bats = Ses = 196 an lef = 198) — 08 Ths by Ba (754th prc conkence mera ora ea. +08) MEAN SQUARE ESTIMATION “UG Find the ms. estimate afar Y by a oonean = 925.77 hem errs cur o= f° yore os,us. 7) ESTIMATION THEORY 29 Cy ste taint $--2[Lo-armeeo : frases anne TUT. Find the maeainaor oft 29, ¥ by fenton AX) of ther: By Ba 717, hemaemors ec eor—aonnty f° fo aor nerdy Slee =A 0 ween me «= [Lrolfp-aeorrse S eee [ho-mo¥ie von i mina or every x. Conpasing F758 wit Eg (8) ro 16 ee hat heya he ae fom fc ao an med Tony ea a 16 ah oe peun= [soins aria os, Heche ms intros Feom=nria ean TAK Find berms erie) = AY a) shes esti oY ‘Awe ermB (158) te onal meas HY|a) of en al X 0 8 ton amt yeah ayiny= a cay ‘Shay, te cndoealmea EX, of 9 79 ge Bat Xa a aon ai, ede te cen Bt, MX} ers The ats niny =f [errno ove af flacaonmnes aft [lornne ndeaye tan. a ote (7h etn of 7 New, we moet gut) Ea = oad MD a0 ESTIMATION THEORY eHar > “They Bat (742 28s AN weave Tawa = EI OAD = Eases) ony ow seine 0X) = AN) and = YE (a ig (728 tn la) = ae = AT “Tous ema caries CY = 9} = = ELON + et S20) Aa 00) 239, Leb =X and X bea uso toner (=) Find the ms. extiator of a ro Xa By 7 he ms timate of Yip by l= AY) = BX eat fox 08) ee eicr aunt =e a'r} <0 6n [LINEAR MEAN SQUARE ESTIMATION 7% Derive the orthogonaiy principle (7.2) and Eq 7.22, By Ha C2 heme ori ab) CY = 0 4) (lacy. tems tore aowion uae suai minima #2 = 04nd ab Now IY ~ 0X + B= ¥) = BEY — ax ¢ SD BI ~ ak +b 1) = 28107 0x +99) Seung df= Dad lb 0 oa BY x vane) =0 ean AY ax +=0 ow Noth Ea (she orgpnty e775 eaangog Ea and at we re Eun =x) axe += 2 be ota Be 204 ee Ane “ae ceo 08) a8i0)= me wee Weave ted Es 230 (od 859 121, Show that ms. ero ined by Ee (720 minima when Fs (758 an (769 sti,car 7) ESTIMATION THEORY 261 m ama. 724, ue at =X 4 dee a drab onan. Then = BY —10k + 4} = {CY a $8) + eb 4 OY —tor + IF) 3 le ek 2a) +a OEICY = + HY) + 29 EICY —ax + OD) ta) + Ea ait 40-09") Phe blty tox +654) + ab asity ex + om) ‘The st wo terms che igh ste to whe Eu) and (79) ae ae, ne sein nde not ca Sn ef a my an Hae a) Dasve Eq (720, 37 Ba a7 49, web ICY ~ (+ bx) = 0m EY —(0¥ +8) Ten 2) ICY ~ a +!) = BLY = (08 + DIOY =x +49) Seley ox + sr) = tr) eeury)~ 080) sn 23 3 a9 ana be a C220 abr wee B20, Let Y= X2, and lt X be aifrm a, over ‘eit of Yi terme of X andes err Yeorse 1) 6 Prok. 719, Find the bear ma we aan be ow ny Ean 24H) ad 24, aye ax) eu)=! Brey 050, eay= ContY) = BY) UNELYI=0 “Tan =Dond b= Yan hermits P-b-an 20 Bray am Fag the miioen ms. eoretinator of ¥ interme of X when X and ¥ ae joy nomen)22 ESTIMATION THEORY conar? By Ea 71 hemmam seer fn emo. Now when Xan Y ely aomal by E4100 P18 ASI, webae avinieon enc he minimum menor of Yi Fe Bi =o wn \Conpsiog (2) with Ege 15 and 22, es tha iy nova he ie meme ‘Supplementary Problems 2S Let Oy 4p be random simple of X dang kw mean 9d vane, Show tha he str of a by Eu.- 2 eae a ube ein of, Ne a5, en cls song Shane we £4 729, 1 eto bea mndom sample of having own a and noun vans, Show hat he ecw initsacaaaceanemeta eels teroce Hin We Ea 28 22k ed Aaa mmo a ofa Bea hate A =O 1 ete 129. Te los a tem ape 29,05, 17,43 and 3. ae oan rom 2 tat iformty ‘tual veh anon otra Fd te mani ace its fan fis tng t= 95, Bye A7 In antigo of wae ops edn ban, he Sins Go mn tween also 10 (etme ate reared at $2, Sk th 1234 64 1519 and 90 Am at ener te ‘Saneagocenal esd pre td he mserum elo inate tS YOM Let hyo XD 8 anon so gal wi noon mem and unknown vnc. Find nane ttlhond enmat o IR La fy 9 tte dre ample fx noma xX with mean an vin, whee it ‘owas hat ToS tmal Fwd ae pan ans Fn The Bes need) ‘ci egal ch hat he wih 98 pose coher eal 3? dn net ae hs dns a= BYE ef the mena emia a fd he ima acer 7 Dene pe (129)en4 (728,