MACHINE LLNS MD ho MITCH DRE Ca INDIAN EDITION7 : ONIT=T ——— Introduction The field @ MLS Concemed with “how 4 conshuck Coraguber reggromms Hook aukormetically Tmprve uirh experience \ Succewtl ML apelieations \ Dakeawining Groqram Hut Lean te detect fraudulent | credit card Lramactions 2: La fomekon Filtering system jk Learn “users reading _ Preferences" 3 Aukeremow velicles dat beam ty rye on qublic Nighwsou, e * ML concerts and is erulty come fom many freloy incheding- = Stabidics = biology — AL De cognitive Science = Philosophy ~ Compukational, Com plexity. — Leametion Theory = tealre| Hreory We gock ¢ this course 1, te Prerent te “ey Alsorithras awek decory oat form Ha €or op ML. SME: Ts twherently 4a mulkidiscs plinemy Feld + A sHecephd orderstonncting of hod to make compukers Learn eseuld FORTE many news vies a comple « + fk Seems mL wil glay an nerenainghy Central role in Computers science:wedonok yer Know how te make computers Learn Nearly ariel however obqeritnes (alg 5) hove been geogie Learn ay invented Ask we effective Fa certain typer Reanning tasks EA certain types sp Hearning * such a> speech recognition NLP’ (Natural Lempuoge Procensing Paying Games avd Docker Minang, ( crbact trocledye fom Longe des) - Desa (Gotine op oan) y ~ Gemgulers Lenawivs from wedical reconds to ove treatment fa | new AiSeares, Howes leaning Fram eryerience te optimize eneray costs bayed on the: Joie rage patterns af their oceuganh, = ee fearwang Ake interests of Ahety users inorder to igh Wight velevank stories Frm dhe online motming news papers D well- gosed Lecaning. problems Leer sty Computer freqram Rok Improves ts Rertrmance ab Some tase Arreugh experience « + ih general te hae well-defined Leann qotlerns we must identify 2 peatiney + Ate claw fp tasks + Ate mearure of YeTfamance to be improved Ae Sowee op experiences ae Computer Qrarain Aut Leann to qlay “checkers” wigek Ampove ths perfamane ay mearured by Its Isel if ability te en ecarouh Paferiente obtairrea & Plering gpme ont | | || ose Hardwviting Recognition Aesrnivg problem : Task + feoynizirg acd cownifying handwritten Words within \mages Rerfoimance: Measure 0 Fercent ap werels correctly claified Treining experience £5 A Dw a haduritten words with Given clawification : | &x@: Task Driving on Public Aare highesays vsing Vision Sensors Ferfrmamce Measure PL Ay distance traveled befre an errrdg (@ dyed by human overseer) Theawing Experience E> A sequence 9 Woges ard steering Commend recorded ubile observing a leman Arwer (P) Designing a Jecoming System (5) + choosing be Areining, experience : Choosing, Ale Target Function + choosing a reprerentation fn Ale Target Function cheesing 4 Fanction Ppp re ximetion algottnm (2) he Final design: BD chering Ha traning Engerione | + The Pmt design choice is te choose Me type y training Experience Bam whith sur System will Leann. ~ Th cam bee Sgificant impact on Suce or foilune op the desrney- r2| / = es 7| = one Rey allvibute 15 wheter Ae Lrainirg exepeniene (rbd Provider — divect cr indirect feedback regarding Abe “Chisices WY He Perfrmance System: Er while Leanning to play peckers , a the System wig heem from dareck examples camnisting op tndivdluat checker beard stokes ad Ma covect move Ar each. ~ Auckireet infe, commisting of Ake move sequence mmol final Sukcomen sf Veriows Fpomes Played . Note. « Learning for direct breinng feedback 6 easier than » 6 wdivect feedback, | | | | | + A second tw albvibute oy dhe FOr is ale | | ea" examples. degree to which He Learner controls Mu Sequence Krein + Ger The Aewmer wight Tely m He teacher te select beard shedes and te pryide Ae Correct prove FA each » Er) - Te heanner might itself Propose Leeard sheder 4Lt it Fincls confusing ad ark tte teacher fr dla cored move Gy) - We deamner mow have. Complete Conte) over Lot, the boewd deter ard reiving clavthcation ae : rd altvibute ai XP tg hes well tt repreyet dhe iter cecal, Mele: qe crucial aysumplion Abt dhe cichabudion op ‘rai. enamples Ke identical 40 the — Aigwitition og dest enaimgles. AUS budion ge exampfer +| GED choosing a Target eunction Te net design cloie is te determine erocty what tyre "| Rnoukedge wil be Leamed and hoo Ads will be wicd by Rerfamomce Program. | + Ger checkers~ playing groqram atk can gprerede Als hegat moves from any beard state: ~ Te. rq weeds ony to hear how 42 choose jhe beat won From cmong Arese Legal wes: | | c iiss [tLe us Call Ais function (fa) 2 ChoosetMove 2 B—>M | | Ve indicates Adin ty accepts asinqut omy Lert From B- (® = set # hesal board stote) avd produces ay oakput | | Some wove fom M ( ser # leged wwoves) - + dmither Larger ty 18 am evaluation ty MePR, dr denste Abst ov maps amy Legal beard Shle frm & to Seme veal value R- - ¥ akign higher Scores fo better board states. ga darapt Velne VL) as Fallows : "Lek us define 1. TB & isa final board inte Ht is Wen, then v&) =100 ac ae =e “ » dost, then *VEJ= -tod | | a ae a ‘ * drawn then vib) = © ; STP bis a nok a final sete in Ae ame, then viey=Ve') Be ete Lat finek beard siete from b ah playing i optimally unt] dhe end of te epme- | '| &D Chossing & Yepresentation for dhe ‘Taract Function * + we wwst — dhovse a repretectalion tat Are Learning Program will vse to deacrite Ate fy Qo At ik uf) Leann pv | we coud allen dhe prgram te = 0 eittver ee Lang fable wills iskinet ent fn cach didinct beard tebe () ~ wing celleckion ap ruler tuk wutdh aspinst board Feshues Gr | quadratic Polynomial fn ge predefined beard features (67) ~ avkicied neural network + Br Lek wr cheese a Singh regrerertation! EA any given Loord state, des fa GF with cabeabeted ma bineor combinetion 9 dle Following board features H+ Ae morep black pieces om Ake board tr. te red 0 " Fa ey «Gack Kings " Hy t “ ‘ey red “ “ Ris 7 az 0 black pieces AHyreate ned by rvecdcie) which Can be captired on red ect turn Xe dhe no: ge yedd pieces threatened by Lack, Thos our hearning fregpram will ye present VW) aa Linearty @ tte — Boum [Pereos weet 18 + Asks 48 ety Beko tdg | where We te Op ae numerical CoeKiclents or weights. ‘Aissten. by de Aesaming ag. which determine the velakive Importance dhe Verious board fecbures +| @) roo sing « Funckion Agproximation Alserithm ot ty Ven he Aenggt 0 wee_reqpitt set ge Lvaingcam pled in Ake form lee b, ers Eo Tee flaming Arnining ex Aeseriber a board ste & , in which black hes won Ae at o LL Hd, HHO, KHEV, MH he =O, 2% : Mele? 7,25 Mea no ved pieces remaining . tele) og Estimating trosning values bee ALjuating he weit + Eshimatirg Lrotning oluen ha Iwler mediate Goard States as follow I je 9 (socomatt)) seer (VD where Succemor(b) dewter die next ae sea clebe ollouing, b + Adtysting Ae weights is 4 specify Ate hesrwing oy fr choosing Abe weight 4 to bert fit Ake set sp training Champley | | § de, awit > 4 | . | Batt means definig Me Lest hyoduests oy set weights as Me A Riess Me squared exnr E bekween - and fe Vober _redictest by je ye ai vl Ake red. Valo ae _ Lo = So) E= 4 1 Mena) re. E TreinExanglen re As| Several alg ane Known fa Finding weights oP a hiner fin flat winimize defines? 4 Hay vey & LOS (Least Meon Squeer) ‘ Los weight update Rule - €or eae Aveiving ene db, Vaan) - A "Use AL. cued eights fe caladate Vb) "FO cack weight lowes Wr (Ya lar Ve) x ‘\ - Here Yc Swell Update ab as Canton (Ex: 0.1) tut moderates Ma size 4 4 Goeight update « * when 4 en Abe ener 1g 27 Ho coeigits ow Changed . % 4 then (Mea. ve) is 4ve (i Ub) Is tee how ) 7 VS increavect in Alen -eacl eiefat- Prprtion bake value ote Corverpading fecdure This will poche ener. a a a a % 's zew 1 den ts veidte 4S nok akered regedlers Perey: © Tee Final Design The final design any learning System can be roturelly described by 4 Preqram wodules + Lrg v J "WF rmource System *! “Gite Generalizer <\ Experiment Generator - : oy=Tervtamance System! ts Ma module Wat wut Sele tte givens RErfamarce desk vsing hewmed Compt functions - ~ NF Aeker aun instance Poo new protlem ao input aed ~ grees a hace og Ws Sluhion (gem history ) as ofp ~ Trowr ca, Mor Sealey wed by Mis | whedale te As select nex\ Gest move at cack Step Using evaluation tal): Qritic + = a tarea ar Yp Ate kistry al produces a set F training exams zt tbe Acyrget function ~ Ta cur care, this module corresponds te Ma Lrainwg role (rey Generalizer 5 - ge tae as ‘ve Mu Krainin exs hypottesis tat ts Hy erkumale op Als Lacget fe and produce an ofp — Te Wyfferse sizing x general fy Mat covers give Lrevining €xcimtles, awl other Car alio = Jr our caw, Was module currespards to Ae Lins aa Experiment Geweretor | AR hares od Sp que current hy portaesis ( curently Jeamned fn Jaa aes a new problem FMS permone System te explore , Note Many ML Systems can be vsefully chancterized Th terms op Mee A gerteric modules ome Sequence ap design choices mace fy He checkers preqram Is Summorized jw [ Figure ve] be - te]@ Perspectives avd Iscues WW ML * Perspectives ~ PML Anvelves oy 4 & Very hare space p possitie ly potheses fe determine asp est Fits Me observed data ~The LMS aly fo Filing wserghts | tkeredively Auning Ame warts , adding Copyections to cach weigdt each Lwne fhe hypothesized eval. fy gredicts « Volue Atak dilfers Pram he Kroawing data: “t Merent hygodresis regresenkotions ane agpropriake fr heanning AR Kinds og ctonagt functions: Assuey (Humber $F gpneric question qhowk tw) = whet algorithnn exists fd Learning oynerch Aorget Fn's From specific Ayoaning exam les Sit alas qertormn beat, fy dich ty 964 + poblenn freprenattiont ~ How much Arearing Aska 1S suicienk 9 ~ Caw Prior knowledge be helpful even when ik is only agproximately Corveck 4 — whet | Wo Ake best way de reduce Abe hewrning et An agpreximstion pratlems J Hd Can dee Learner auttometically alter ts regreseadation to Twngrove Ws ability to represent and Learn dhe target fn i Course 1g concerned with Amsuering age above question,| Concept Learning Task | + Muda heernivg, ‘volves aqwiring general Concepts from sqecttic Lretining Catarmgles. Fora given Ayeining examples labeled as members oy nonmembers of Ake concept 6 referred o> “cence gt leaving! ©) agtrorimading a Lrclean-velued fn Prom examples: [Table oy alle 2) dercribey a sd ap examfle due — each re prevented by a sk ge nthriluter © the abtvitute “EnjoySpert" indicokers Whedler or not 9 person | Enirys kis favorite Welter spat on this daw. \ 4 "The tose ts to Ream to prediet Ae Ualue g alvdecte “Exjoysyert” PA an ay bikrony day bared ot the values % ks cher odfributes- \ wyfotbesis + Let each hypothesis bea vector of Six conhainh, ; : a Specehying the valor sp tte sin edbviluken wm AK Tablery + For Each abtritule, He ryfowesis iW either ga B , repuredul L~ twcltester, drat omy value %¢ acceptable fy stu oblvitute » ¢- “ "no value is acceptable “ « rw \wqpesis Abas a fersen enjoys his Favorite sport only on MA dows eomk th Kigh Ian Omcagendek op te ole addi bube a) regrrnted by Aa enpreniion fea] 7 |EEO TL modk gener hypoikesis — Ast every doy Sa te keane 1S veprembed by rapa agg LUA | * The definition =e the “Ewioy sport” concert henoning hack | im Abe General form 1% given [Tate vr) Twtences; The sek op thems over wich Re concept te AeFinad Ws called a Sek @ imsbemces -densted by KX The — Concegt 07 function 40 be Learnect 1 called Abe torept Concept denled by c- Th agnert ‘ce! can be any boclean urduest fy Refned over X. i’ cr X> fon coat 1 Exiysqaty = yes (fositwe Exsmp ie) cay =o if Eriyyszat = No (Megane Semple ) COHN a abso Collect Member op the Lerget Concept « Oe 8 thes is Note — she Goat # the heonney is te fined o hy pothesis h Such Het WOW =e) Fa a a in X: ‘eatectve Aevwing hyteeris: tony hy pohesis fuind appmsnse He Lavet 21 well overa Shi ciently derge set op Araining exlangle Wi abe Gpprtimste well over the, Unobserved exempler « Ths 8 th firdamental ti incluctive Leasing, — oeC2 comce tt Lesrning 04 Search "The oak of iy Sear is te find He hypothesis tt best Fis te brauning rermpler - y ~Ceider Gor nncty X cand hypotefis Hin she "Bajey sent" earning teak + ‘Sky’ nthille her 3 potitle valu ay Airleme Hewitt, Wind, wader od fore cost each have 2 pot tae a > Whence Space X — conteins erally B+2+2-2-22 =96 distinct Veluers = Sinclely SKYUth AY GAG = S120 Syntacticetl, Aislinet hy pie ses ain MS ee sry sintle fecrnivg AER wiih Swall Fiwike bypotteio space Mose Proctical Le maning Aerka invelue much, Langer , Some biomer infinite nplesis space Fw aed 15 find cloprittens capable 4 Mcveth, ee hypettesis det Let At tL. TD (rainy Data) « General b Specific ordering of Hippies Conder two hypothesis (yp) he S sm, 1,4, Swog, 4, 2> Fie ce age ee Vere Ve ptten, fewer comshoiah on the instances , Tk claaBes Wak iwhewces as tye. adPy tartemee Clam fied te by Ly will abo be clanitees ave by he Hr 1S mae qeneret Ly * Gwen Wy ant Ie hy i “ade general Abe or 2g tual 46 hw, 1 VEE aany whence Rut Sabis tien hy alto cake fies hy | Gro ou =1) —(h; @e= ) | [ Fe a] : The me velation 6 imp becawe (+ Provider a wcbd Shruckwe over He hyp» space H fy any * concepr Aa mrning Problem eek eee LA maximally Specific hy potlesis : Final S Algpritlen : a a T, oe Wy & Me est Spedfic hype in e i heen Pestve raiweg nttance 1, 7 ORR eaeh albu Cantal a wh TE He coment A, is oats felt by x An do welling de refine ow by Ae West mde gener 1S setisfiey by x, byt h “Th fist She # Fads IER Wwittabze oh dy sant h< £86, 0,0,0 05 Seunider fink ave iusemue (mw) ian Teele 21 rot Cabs Bing Combine 3+ outgue syeceh, 7 cMSo each iS peglaced by the Net Wd Generel Cambria Host fits Hu ex | h ae Sunny , Werm , Normal, Strorg. Warm, Same Ss ‘ tmuoh we seh Nery cgecific > 4 Dy Keni ex Rexeey Me ay te fur her Fewerelize h, | hed Sunny wean , 7 ) SITE Wer Same > PR FA kate : | Bi Reiiiex, & (goes ve creampie. | | A | ~? FP AM teniniey ex, ead & Beetles ener lization h e.| a se | We Gsm tom, L shee 1 TS No: pk enh Ghaye, Abe hw ‘is Me oct Specific hye. Cons tet with AL rect eX clterved ug AMIS Pritt = CLence | mame Fords) + | (Fa, rr} | Nee: Find-S lc quemanteed to cutgut Ale imesh ¢ pecific hyp e| toubin HL ALES cenaistent eal Mle AVE traaing eracmple. Ts find hyp. will abe be coustet with He Ve example 1 | ID ID) Version sgaces ond de Cardidete- Elimination imabion “Alger + Comdidale Elimination (CE) aly aatrenies Several Luvttakion of Find-< joe | The RY Mea in Me co alq ig te outgut 4 dercripliow -f Ay “set # all lypedaeses” contistent with «the Lreunieg exermples 6 coy ad Find-s dg ome himted ey bce perform Poorly when Wen noisy tunis dota - x wisy 9) dake, ie+ Applicekions “ cE-aly aan + hemrning regueution tw” chemi col mes spect swpy" avd ~ heanivg conto] ruler H "heyristic Search" “B Consistent — defjnation A bypothesis W 1S consistent wilh A sek sf kei: Examgler D wee Wy = CO) Fr each ex Lx, GOD MD | consistert(h, D ) (44x oy ED) ke =e | sokisty: pn ee ete said solisfy lnypoanesis by whew hoy =) wegerdles 4 gheter 1m ne ~ ctamtle: § @ Consistent; An 2% 1 ie conisterk with Wh depends on Ae Leryet Concept je hG) =c@)). Version sgace: subset ad hypetesis 1s cae the version space tort hypotlesis space +. Tae = [he [emsnalio)) | subset ot hypoteesds from H consistent wir Ae Aeaining Cxarutly in D> + we Com represent dhe Version Space in terma g tks wos specific ¢ aad most apnerel memberlG) pluc hae hypesi: UE Lbtucen Gad S wm Me qoattally ordered hyp desis space [V0 ek | (4s €5)(34€6)( 34 hays) j [a3 )The CE -ahy computer de VS (erin spate) Containing all byphests Fem H Hub om comstet uith au observed Seqwence ge keainey etarmpler * Tk begin, by Wwikiebzing AL WS te dln set gp all hyp. in H, by iwi kcabizang Ate G bumdey see te GO §

and (wikis Me Leandiony Set te Me mmesy specie. hyp So <— § £6,¢.0,0,6,0> 4 ad + even other bypebess wm Wis in between G, ond Se | (Tables) [Rg 24) | Figure pt shuws ce-aly applied te de fink tue Keesiosey eonmple from Feuble a | 7 Pestve training Ok. may Ane da S burday op fe VG bk lecome wereaurgly qpneral: eo. Negative trainiey exs Woy Frrcing Me G bey fe bewme | Wworentmgty specific Commer dhe Aud Arininy Ge, Sheen in (Fy rs | © The yp. in Ae G ww be specialized Until ip corseetly clemities War new — neqalive €amgle oe ote ts excluded becom wh is Imcewistent sith He previolily enteuntercd prstne mmples The a teeing Ge, as thee tn (re a6 | | «Overall “VS 1S sham in LFig 24 | 9Ehense! the WS earned by he CEraly dat correctly dercri ber Ahe Layer Concept — providect (when) Vi here a Ro errors fn Ahe training Excmplea Be Tew 15 Some yp. Tn Ho dla Correctly dearriber dha tavggl Conceg What training Be should dhe _Leamer Request Next 9 ~ Uf bis point Se have attumedt te Loniniry "But if incervectly covers da 3 nega trees ex» DPT gable 1s aur be he bined dhe hearer de conider oly Conjunciwe ly petheses + Leb vc reformulate flu Exyy Stat earning fark in an Unboiasect Way by defining nee hypothesis space yy) te allow arbitrary | Aisjunckiens Conjunction, aud Regalions ap cur easler hyp... SO Samy LAD VC chk (1,1,3, 21> ) S$. £(WVUVG }} Aisyonction Gq: § 4 (x4 V ty) \ neglion Aurdludive | Lessniny Algeritina (1) = deductive bias gL te be Me Set 4 assumption B Such det Ar alk new instances 1%, (BA Der x) ELA 1) [ Notation Yb 2 indiceter dt 2 Allens deductively tom ¥- [¥% 28 |CD) vecision ee Leaning — traduction Decision Tree (DT) tering ts cme fp dhe westly, widely wed ard prackicd metiod -fr incluctive infe rence + Dr Ts yebust to netsy dakar acl Capable oe Lemming Aigjunctive ex gresions Tormily 9p 7-L algorithms one 103, ASSISTANT ad Ch-S. + Lewmed trees com abo Le vepeprted 2, set | # Wren | whe bt imprve hwman readability | » Aoficettons + Applied te a breed Youxe ¢ traks fry deannivg, & diagnose medical cass tu @ deeming fo aren credit risk lean epplicelrends Decision Tree hepresentation ; Toukleck | Sunny / ove e yes Lwin] nig” Shey _ No yes No Ye ® [e037 [TRAST 6 feomed pp te clanity Sodurcley martina, gccordingy whelker | they awe Saiki ble fy play ny Kennis . Jo general OT eget a disgunction op Conjuncdion x | compainks on He athitute values ff imbenes - 6 Gack goth fom He eee root be a Leak corerparch © 4 conjunction cp atbyiluke teats ad | Me keee thelf fg dis inction gf soe Conjunction. | EY (ouklor = sunn : 1 A Ruwidity = Hormel) ¥ (cuter suerest Vv (Cowllor= Rain A inde eak) «* Each node wm the bree Speci ties a Ley F Some albyibute te witance ard + 20. branch der corcting fom Aet node eovrerpands & one et. fesible values fr his odtystake GD Aerrepecte pubtem fo pT-Leawing (071) ' DAL 1s Feverclly bese Suited te Problemy will Whe fellesing chavact evi stics V Daskances on Yepreiertecd by tlriluke value pairs. (Purnia = bah 2. The Kewaet function bar dis cvete cak put values - SIA clothes Tiscele: Ase g data ® sad fo be discrete abe values belonging be MRE ae Aistind v sep. anu 2 disjunctive description may be required , GoM. fresning data s Th wey contain ervers Sin ‘y tal va MSSING Athalde vals Cece, a ‘J : CD eave bre (Pere Len LAkyrbon )~ 193 3 Selecting, whidh cthibde v The cervel clove Wa pz aly 1s Re CO AE each vetle 4g) Me tae © WS wes Ty pte Gain measure te select ook 1S mosh wet Pa chow Fy; whale ory He tree - (Fave 31] = OTL alain “ Comments Wer Ae" Ear, Characteri2es He Cm) purity Marlys hey Callechion -y ekamples = Ie dhibate we eames ok cack sep Ta fanetion Gai TY) mesure Abek te| Given a Collection Ge Comeswindy Positive anf pee "GW A | F Some eseh Gncept ; SN Crm les Sen) 6. fe bay ie (okg.fe) )| te "S$ tee Dreger tion F Positive ewamples in cs . fo ‘ it newline i | ' To understand Consider [ taue or] wher Ss ' ~ WS Collection "4 enamples — inebudks F posite ad negative excmpley + e ~ (Yu) 14, Vy - (uy ) de Eres (44 5 ) = ] ‘be (s/ ) ~ e940 eee cee eee NAC Grhepy is few) 1p Enbo py iS Tne) wshes dha Conta aq excl no. £ PSitive 1 all members FS Lelong be A Game clay Od necahire 7 TE He S cont Sere OS os Pest ad Agjalee ex’, jae | Emery IS behead \. - ~ More generally, A doret athitate ean fore on ¢ dip Wali, Ahem He enhepy 11 $ Sb Ry Cute Shem ety is Exbora(s) = 7 “Py bog, r there Pi iS he Propatin gs Ce te claw 4 Informer Gain ee (Bain) « iS fw eXpecdkedt reduction in coheyy comnd by Pevtitionng Abe Srtampley According 4, Abn obit. -Sv Gain (5,8) BE Entropy (5) — F_ vs Entvopy(Sy) ve VeluesCay Ee Asm Ss Collection @ Me Crcempler (a+s-] SOROS atbinte tind have 2 vols je Lean endl strong a (4+, 5. J Sweag = [es, 2-J Pale a Isv| ea Gain(S, und )= Exoty(S) ~ > Te ota(S,) eS teas shag = Eniery (S) — i BrbeP9 (Suen) ~ € St Sony) OHO (Yq) o21 (ig) 0 = C04g a ~~ Ci) 4.4%) -(%) la&% ~ Gan (Fa 33] shows tbr Gan F Hmiding and Wind attitdes 4 determine wich & better athikube F clerifying. the Leeining CXerm plon hie. 403 determines He mEr opin fy each otbinte then one wh Kees opin value Gains, aikleck) = 01246 W~ (ifort vale ) rans, Huity ) = OS] Sin 5, wind ) = bo4g Sain (3 Fearpersbn) = 0025 owtlcer is selected a AL decision atbibute fa Re rect node and branches an creakect fy each of its possible clues ¢ Sommy , overemst , vein) 4) Shoon in [Fa a4 | * — ontloee = Sumy ] ok ae outlook, = ore Sill have won-zew enbopy. | heme Ae” DT wil be further elaboraked ay shun in (Fa % | selects yy CY Aly pothesis. SPS WM DTL (decision Treelearni ) LE [Decision Tree Lenaning The bye Syre Semrchet & 103 ‘s * ee seh PANE Recisin trees. + 13 (ert a Siangle - te complen bill climbing search Horcusle Mis hyqothe sis space fing cSt, fhe crnths tree hon emwadering Prgrersively mac eloberake Wyethess tm Search ot a py ay coveectly lena Pies ee te ty Using, inf mek ot BM Measure, Capabilities y Limittion op ID3, BH 9s's Irypedhe sis Syace Fall DT is a Complete Space ¢ finite Atscrete-valid tas A aveilatte otbittey ~ Trails He map 94 © 103 mointosing onl, « Single ey Ae space ¢ DT. Yow SKF Search Weanplete hyp Spaces. mre” hyp. as it seraches Aorensy Ty dees't have Ae ability Ae determing mong allernchve DT an Gusset with de awilate Yay ales© 103 ts Pure foo Performs he back va ching: iw tts Search. This focably optimal Solution may be Lem desirable Ahoy bees Hoe csould bese been encoutenesl hag diff branch “f be Searls. | (D3 uses all Keating exompler at each Steg Mm de Search bo make gets Siabistically bared Becisions « 103 Cam be ently erlendid ft handle netsy erasing ee + Muck Lets sensitive t2 evrers sbule Seancling Gy) Selective Bios OTL + Seductive pian is the Set of assumptions cht fo gti Wit | Areawing, date , deducbively Jwbity tle clamiticolions eavigneel &y te kearner te Lubin inskeamees. We aly cheeses te fir actegtalle tree ik encounters imide Siew fle fe compen | bill climbing Search Awouge the space posible brees- Loa Search Shrabeay is () Selects dn tawr of shorter bees over longer ones () Selecks trees tuk place dhe atlricates with higheat inPamta, Spin closest 4© the yoot - Preterome bias Vs Restriction bins + she inductive bias 103 is Ha a preference. fr certain hyp. oer ders ( €g: 42 shoter We) calls a preference bias - Preferene bias Is we desirable dhan a yestrtation Las , beraue 1k allows the Learner f wort widhin A Comyle te lryp- Spat ° “Ge continst, restriction ies Alot Limits strictly Ake Set frtential hyp is qenerally lets dlesiva ble cE prefer de srptat byp- Abst fits 4 data) Focuen in Decision Tee Learning (5) 1s OFFI be dato Le Guarding aginst Hemndling Continuous valued ablrintes & Welling missing cdbviute values S Herdbiog cablributes wth differing, costs: bak abritute chorees Ue over fitting tha data Definitions Givens hyp-gpace Ho hyp HEN is Said te overfit dhe Areinivg doka If dhere exists Some alternative hyp- hie} Sach Abst W has sweller ever jhon hover tle foinry temps buk h* hes swuller ervey evra than h over Jte entire Aishribukion Iwutemees ( fe tmclading slammer Leyord le traning set ) Hes Con me prevent overfitling 7 > dott dey fe fi all eremples, Stog befre Ae Kraining Bet i exhaust : et FE alk examples then Prune Re vesltont tree. 2 Eruanding egeinvh bad cbtvibute choices ~The nf gain measured les abi Mat Fouors aityitutes With any values over Ahebe wilh ondy 4 few + GRr Date allrble sth Unie Valuer fn each traning €X + gbviowly no cher edttitute can cl better , + This ot peylt in a very Sroad bee depth | eats: Gusad GaimBodio 6A) = inst ALIS, we Can WE GainRekio (5A) ited ¢ Gain 6A) manent Gain( < Ng 2) > FIGURE 11 Final design of the checkers leaming program Determine Type of Training Experience Games against expen Table of eotec value Determine Representation ‘of Leamed Function Potynomist (Lincar function Arifcial neural ‘fem fearures network Determine Learning Algorithon Linear “Grasiont sat \\ programming Completed Design FIGURE 12 Summary of choices in designing the checkers learning program.Example Sky ArTemp Humidity Wind Water Forecast’ EnjoySport 1 Sumy Warm Normal Suong Warm = Same Yes 2 Sumy Warm High —Srong Warm Same Yes 3 Rainy Cold. “High Strong Warm Change -——_(No 4 Sumy Warm High Strong Cool Change Yes. TABLE 21 Positive and negative training examples for the target concept EnjoySport. + Given: - ‘@ Instances : Possible days, each described by the attributes ‘Sky (with possible Values Sunny, Cloudy, and Rainy), AirTemp (with values Warm and Cold), Humidity (with values Norma! and High), Wind (with values Strong and Weak), Water (with values Warm and Coo!) and Forecast (with values Same ard Change). 1 Hypotheses #7: Each hypothesis is described by a conjunction of constraints on the at- tibutes Sky, AirTemp, Humidity, Wind, Water, and Forecast. The constraints may be “?” ny value is sccepible), “O" (no value is acceptable) or a specific value. 4 Target concept e: EnjoySport : X ~ (0,1) ‘© Training examples : Positive and negative examples of the target function (ee Table 2.1) © Determine: ‘© A hypothesis fin 2 such that h(x) = e(x) forall x in X. seoee TABLE 22 ‘The Enjonsport concept learing tsk. 1, Initialize 4 to the most specific hypothesis in AT 2. For each positive training instance x © For each attribute constrain: a in h Ifthe constraint a; is satisfied by ‘Then do nothing Ble replace a in h by the nett more general constraint that is satisfied by x 3. Output hypothesis TABLE 2.3 Finv-S Algorithm. The List-THex-ELIMIvare Algorithm L. Version Space « a list containing every hypothesis in H 2. For each training example, (x,c(x)) remove from VersionSpace any hypethesis for which h(x) x e(x) 3. Output the list of hypotheses in VersionSpace TABLE 2.4 ‘The List-THEN-ELIMuNATE algorithm,Instances X Hypotheses Ht 2 ow lt ah a - AON j= Sin: Warm, High ght Warm Same ye Sunn 222 FIGURE 2.1 Instances X Sree y= <2.2,2,0,2,0> x © «Say Warm NomalSrong Worm Same>, «hy = Suey arm Hick Srons Warm Sme>, hy Rain Cold High Strong Warm Change>,- hy = -,+ hy = [} _— | 2, Strong, 2, ?> — <2, Warm, ?, Siromg, 2, ?> NAN (eSuamy, 2,7, 2.2, 2>,<2Warm, 7,22) FIGURE 2359: |{<0,0,0,2,0,071 { } I Sq: [ () Gy G:[ (RnR RAID ‘Training examples: 1. , Enjoy Sport 2. , Enjoy Sport FIGURE 24 Sy 32 | } Gx | {, EnjoySport=No FIGURE 25 CaNDIDATE-ELIMINATION Trace 2. ‘Training example 3 is a negative example that forces the G2 boundary to be specialized to G3. Note several alternative maximally general hypotheses ace included in Ga,5: [[)] Ph G4, [[) I G3: [[eSunny, 2, 2, 2,.2.2> <2, Warm, 2,22, 2> <2.2% 2%? Same>} Training Example: 4., EnjoySpori = Yes FIGURE 26 < NON FIGURE 27 {) ‘The final version space for the EnjoySport concept learning problem and training examples described earlier. Instance Sty AirTemp — Humidity — Wind A Sumy — Warm Normal Strong B Rainy Cold. ~—Normal—_Light C Sumy = Warm = Normal Light D Sumy Cold. Normal Suong TABLE 26 New instances to be classified. Water Warm Warm Ferecast EnjoxSport Change ‘Same Same ‘SameTndactive system ‘Classification of Taig enampicr | Etanson “fonttknon? Algorithm ee ‘New imtance__,| Using Hypothesis Space Equivaeat deductive system | Ciasatteaon of Toning examples ‘Theorem Prover New instance Assertion “Heontatns the target concept i / Inductive bias ‘made explicit FIGURE 28 3CExamptes, Targer-atiribute, Attributes) ‘Examples are the training examples. Target.attibute is the atribute whose value is 10 be predicted by the tree. Attributes isa list of osher attributes that may be tested by the learned decision tree, Returns a decision tre that correctly classifies the given Examples. 1 Create a Root node for the tree ‘ Ifall Ezamples are postive, Return the single-node tee Root, with label = + ‘« If all Examples are negative, Return the single-node tree Koot, with label = ~ 1+ If Artrioutes is empty, Return the single-node trve Root, with label = most common value of Target.attribute in Examples «© Otherwise Begin (2.4 = the atribute from Attributes that bost"classifics Examples The decision attribute for Root < A «For ezch possible valu, vj, of A, 12 Adi a new tre branch below Root, corresponding to the test A = vs «© Let Examplesy, be the subset of Examples that have value vy for A eo 1f Examples, is empty ‘© Then below this new branch add a leaf node with label value of Target.attribure in Examples + Else below this new brasch add the subtree D3 Examples, Target-attribute, Attributes —(A})) ‘most common © End ‘© Return Root *"The best attribute is the one with highest information gain, 2s defined in Equation (3.4). TABLE 3.1Outlook Temperature Humidity Wind —PlayTennis Di Sunny Hot High Weak No D2 Sumy Hot Strong No. D3 Overcast. = Hot Weak Yes, D4 Rein Mild Weak Yes DS Rein Coot Weak Yes De Rain Coot Strong No. D7 Overcast. Cool Strong Yes Ds “Sunny Mild Weak No D9 Sunny Coot Weak Yes DIO Rain Mild Weak Yes DIL Sunny Mild Strong Yes DI2- Overcast Mild Strong Yes DI3. Overcast Hot Weak Yes Did Rein Mild Strong \ TABLE 32 Training examples for the target concept PlayTennis. Sted ses E0940 50910 amity wn oka (6124 13034 098s B08 E100 ame (S. Hamidty ) Gain 8 Wind) ‘49 -criansas -c.e = s9-caneatt igi = 949 -cay9n3 7 Whick attrBoute should be tested here? Sunny = (BLD2.DB.DSDL Gain Sumy Hain) 70 - GI5)00 -2/5)00 = 970 Gain Sunny. Temperature) = 97) - @1S) 09 - (IS) 1.0 - (U5) 00 = 500 Gain Sumy: Wind) = 910 ~ 28) 1.0 ~ (3) 918 = 019 FIGURE 34