Chng 9: Hc my

Chng IX HC MY (MACHINE LEARNING)

Ni dung chnh: Trong chng ny, chng ta s tm hiu v mt nhnh nghin cu hin i ca Tr Tu Nhn To, l hc my bao gm gii thut ID3, mng neuron, v gii thut di truyn. Mc tiu cn t : Sau chng ny, sinh vin c th : Hiu c mc tiu ca lnh vc my hoc. Bit 3 tip cn hc ca lnh vc hc my. Vn dng gii thut ID3 vo cc bi ton thc t Hiu khi nim mng neuron v cc vn c lin quan Hiu gii thut di truyn v ng dng ca n vo cc bi ton thc t.

Kin thc tin quyt: Biu din tri thc dng lut, tm kim trong khng gian trng thi, khi nim Entropy trong L thuyt thng tin. Ti liu tham kho : [1] Geogre F. Luger Artificial Intelligence, Structures and Strategies for Complex Problem Solving Addison Wesley Publishing Company, Inc 2002 (trang 349 381, 417 438, 469 480) [2] Tom M. Mitchell Machine Learning McGraw Hill, Inc (trang 52 65) [3] Wikipedia Bch khoa ton th m - Hc my: http://en.wikipedia.org/wiki/Machine_learning [4] Ender ZCAN, Murat ERENTRK A Brief Review Of Memetic Algorithms For Solving TSP : http://physics.yeditepe.edu.tr/merenturk/tainn04.ppt

GII THIU:

Khi c hi v nhng k nng thng minh no l c bn nht ng thi kh t ng ha nht ca con ngi ngoi cc hot ng sng to ngh thut, hnh ng ra quyt nh mang V Hunh Trm Trn Ngn Bnh 153

Gio Trnh Tr Tu Nhn To tnh o c, trch nhim x hi th ngi ta thng cp n vn ngn ng v hc. Tri qua nhiu nm, hai lnh vc ny vn l mc tiu, thch thc ca khoa hc TTNT. Tm quan trng ca vic hc th khng cn phi tranh ci, v kh nng hc chnh l mt trong nhng thnh t quan trng ca hnh vi thng minh. Mc d tip cn h chuyn gia pht trin c nhiu nm, song s lng cc h chuyn vn cn hn ch. Mt trong nhng nguyn nhn ch yu l do qu trnh tch ly tri thc phc tp, chi ph pht trin cc h chuyn gia rt cao, nhng chng khng c kh nng hc, kh nng t thch nghi khi mi trng thay i. Cc chin lc gii quyt vn ca chng cng nhc v khi c nhu cu thay i, th vic sa i mt lng ln m chng trnh l rt kh khn. Mt gii php hin nhin l cc chng trnh t hc ly cch gii quyt vn t kinh nghim, t s ging nhau, t cc v d hay t nhng ch dn, li khuyn,... Mc d hc vn cn l mt vn kh, nhng s thnh cng ca mt s chng trnh hc my thuyt phc rng c th tn ti mt tp hp cc nguyn tc hc tng qut cho php xy dng nn cc chng trnh c kh nng hc trong nhiu lnh vc thc t. Chng ny s gii thiu s lc v lnh vc nghin cu ny, ng thi i vo chi tit mt s gii thut hc quan trng.

I.1

nh ngha hc

Theo Herbert Simon: Hc c nh ngha nh l bt c s thay i no trong mt h thng cho php n tin hnh tt hn trong ln th hai khi lp li cng mt nhim v hoc vi mt nhim v khc rt ra t cng mt qun th cc nhim v nh ngha ny mc d ngn nhng a ra nhiu vn lin quan n vic pht trin mt chng trnh c kh nng hc. Hc lin quan n vic khi qut ha t kinh nghim: hiu qu thc hin ca chng trnh khng ch ci thin vi vic lp li cng mt nhim v m cn vi cc nhim v tng t. V nhng lnh vc ng ch thng c khuynh hng l to ln, nn cc chng trnh hc CTH (learner) ch c th kho st mt phn nh trong ton b cc v d c th; t kinh nghim hn ch ny, CTH vn phi khi qut ha c mt cch ng n nhng v d cha tng gp trong lnh vc . y chnh l bi ton quy np (induction), v n chnh l trung tm ca vic hc. Trong hu ht cc bi ton hc, d liu luyn tp sn c thng khng m bo a ra c mt khi qut ha ti u, cho d CTH s dng gii thut no. V vy, cc gii thut hc phi khi qut ha theo phng php heuristic, ngha l chng s chn mt s kha cnh no m theo kinh nghim l cho hiu qu trong tng lai khi qut. Cc tiu chun la chn ny gi l thin lch quy np (inductive bias). C nhiu nhim v hc (learning task) khc nhau. y ch trnh by nhim v hc quy np (inductive learning), y l mt trong nhng nhim v hc c bn. Nhim v ca CTH l hc mt khi qut (generalization) t mt tp hp cc v d. Hc khi nim (concept learning) l mt bi ton hc quy np tiu biu: cho trc mt s v d ca khi nim, chng ta phi suy ra mt nh ngha cho php ngi dng nhn bit mt cch ng n nhng th hin ca khi nim trong tng lai.

154

V Hunh Trm Trn Ngn Bnh

Chng 9: Hc my

I.2

Cc tip cn hc:

C ba tip cn hc: tip cn k hiu (symbol-based learning), tip cn mng neuron hay kt ni (neural or connectionist networks) v tip cn ni tri (emergent) hay di truyn v tin ha (genetic and evolutionary learning). Cc CTH thuc tip cn da trn k hiu biu din vn di dng cc k hiu (symbol), cc gii thut hc s tm cch suy ra cc khi qut mi, hp l, hu dng v c biu din bng cc k hiu ny. C nhiu gii thut c a ra theo tip cn hc ny, tuy nhin phn II ca chng ny ch trnh by mt gii thut c s dng rng ri trong s , l gii thut quy np cy quyt nh ID3. Ngc li vi tip cn k hiu, tip cn kt ni khng hc bng cch tch ly cc cu trong mt ngn ng k hiu. Ging nh b no ng vt cha mt s lng ln cc t bo thn kinh lin h vi nhau, mng neuron l nhng h thng gm cc neuron nhn to lin h vi nhau. Tri thc ca chng trnh l ngm nh trong t chc v tng tc ca cc neuron ny. Phn III s i vo chi tit ca tip cn ny. Tip cn th ba l tip cn ni tri m phng cch thc cc h sinh hc tin ha trong t nhin, nn cn c gi l tip cn di truyn v tin ha. Phn IV s i vo chi tit ca tip cn ny.

II TIP CN K HIU: gII THUT QUY NP CY QUYT NH ID3

II.1 Gii thiu

Gii thut quy np cy ID3 (gi tt l ID3) l mt gii thut hc n gin nhng t ra thnh cng trong nhiu lnh vc. ID3 l mt gii thut hay v cch biu din tri thc hc c ca n, tip cn ca n trong vic qun l tnh phc tp, heuristic ca n dng cho vic chn la cc khi nim ng vin, v tim nng ca n i vi vic x l d liu nhiu. ID3 biu din cc khi nim (concept) dng cc cy quyt nh (decision tree). Biu din ny cho php chng ta xc nh phn loi ca mt i tng bng cch kim tra cc gi tr ca n trn mt s thuc tnh no . Nh vy, nhim v ca gii thut ID3 l hc cy quyt nh t mt tp cc v d rn luyn (training example) hay cn gi l d liu rn luyn (training data). Hay ni khc hn, gii thut c: u vo: Mt tp hp cc v d. Mi v d bao gm cc thuc tnh m t mt tnh hung, hay mt i tng no , v mt gi tr phn loi ca n. u ra: Cy quyt nh c kh nng phn loi ng n cc v d trong tp d liu rn luyn, v hy vng l phn loi ng cho c cc v d cha gp trong tng lai. V d, chng ta hy xt bi ton phn loi xem ta c i chi tennis ng vi thi tit no khng. Gii thut ID3 s hc cy quyt nh t tp hp cc v d sau: V Hunh Trm Trn Ngn Bnh 155

Gio Trnh Tr Tu Nhn To

Bng 9.1 - Tp d liu rn luyn cho khi nim C i chi tennis khng? Tp d liu ny bao gm 14 v d. Mi v d biu din cho tnh trng thi tit gm cc thuc tnh quang cnh, nhit , m v gi; v u c mt thuc tnh phn loi chi Tennis (c, khng). Khng ngha l khng i chi tennis ng vi thi tit , C ngha l ngc li. Gi tr phn loi y ch c hai loi (c, khng), hay cn ta ni phn loi ca tp v d ca khi nim ny thnh hai lp (classes). Thuc tnh Chi tennis cn c gi l thuc tnh ch (target attribute). Mi thuc tnh u c mt tp cc gi tr hu hn. Thuc tnh quang cnh c ba gi tr (m u, ma, nng), nhit c ba gi tr (nng, mt, m p), m c hai gi tr (cao, TB) v gi c hai gi tr (mnh, nh). Cc gi tr ny chnh l k hiu (symbol) dng biu din bi ton. T tp d liu rn luyn ny, gii thut ID3 s hc mt cy quyt nh c kh nng phn loi ng n cc v d trong tp ny, ng thi hy vng trong tng lai, n cng s phn loi ng cc v d khng nm trong tp ny. Mt cy quyt nh v d m gii thut ID3 c th quy np c l: Hnh 9.1 - Cy quyt nh cho khi nim C i chi tennis khng?

156

V Hunh Trm Trn Ngn Bnh

Chng 9: Hc my Cc nt trong cy quyt nh biu din cho mt s kim tra trn mt thuc tnh no , mi gi tr c th c ca thuc tnh tng ng vi mt nhnh ca cy. Cc nt l th hin s phn loi ca cc v d thuc nhnh , hay chnh l gi tr ca thuc tnh phn loi. Sau khi gii thut quy np c cy quyt nh, th cy ny s c s dng phn loi tt c cc v d hay th hin (instance) trong tng lai. V cy quyt nh s khng thay i cho n khi ta cho thc hin li gii thut ID3 trn mt tp d liu rn luyn khc. ng vi mt tp d liu rn luyn s c nhiu cy quyt nh c th phn loi ng tt c cc v d trong tp d liu rn luyn. Kch c ca cc cy quyt nh khc nhau ty thuc vo th t ca cc kim tra trn thuc tnh. Vy lm sao hc c cy quyt nh c th phn loi ng tt c cc v d trong tp rn luyn? Mt cch tip cn n gin l hc thuc lng tt c cc v d bng cch xy dng mt cy m c mt l cho mi v d. Vi cch tip cn ny th c th cy quyt nh s khng phn loi ng cho cc v d cha gp trong tng lai. V phng php ny cng ging nh hnh thc hc vt, m cy khng h hc c mt khi qut no ca khi nim cn hc. Vy, ta nn hc mt cy quyt nh nh th no l tt? Occams razor v mt s lp lun khc u cho rng gi thuyt c kh nng nht l gi thuyt n gin nht thng nht vi tt c cc quan st, ta nn lun lun chp nhn nhng cu tr li n gin nht p ng mt cch ng n d liu ca chng ta. Trong trng hp ny l cc gii thut hc c gng to ra cy quyt nh nh nht phn loi mt cch ng n tt c cc v d cho. Trong phn k tip, chng ta s i vo gii thut ID3, l mt gii thut quy np cy quyt nh n gin tha mn cc vn va nu.

II.2

Gii thut ID3 xy dng cy quyt nh t trnxung

ID3 xy dng cy quyt nh (cy Q) theo cch t trn xung. Lu rng i vi bt k thuc tnh no, chng ta cng c th phn vng tp hp cc v d rn luyn thnh nhng tp con tch ri, m mi v d trong mt phn vng (partition) c mt gi tr chung cho thuc tnh . ID3 chn mt thuc tnh kim tra ti nt hin ti ca cy v dng trc nghim ny phn vng tp hp cc v d; thut ton khi xy dng theo cch quy mt cy con cho tng phn vng. Vic ny tip tc cho n khi mi thnh vin ca phn vng u nm trong cng mt lp; lp tr thnh nt l ca cy. V th t ca cc trc nghim l rt quan trng i vi vic xy dng mt cy Q n gin, ID3 ph thuc rt nhiu vo tiu chun chn la trc nghim lm gc ca cy. n gin, phn ny ch m t gii thut dng xy dng cy Q, vi vic gi nh mt hm chn trc nghim thch hp. Phn k tip s trnh by heuristic chn la ca ID3. V d, hy xem xt cch xy dng cy Q ca ID3 trong hnh sau t tp v d rn luyn trong bng 9.1.

V Hunh Trm Trn Ngn Bnh

157

Gio Trnh Tr Tu Nhn To

+: :

D3, D4, D5, D7, D9, D10, D11, D12, D13 D1, D2, D6, D8, D14

Quang cnh? Nng

+: : D9, D11 D1, D2, D8 +: :

m u
D3, D7, D12, D13

Ma
+: : D3, D5, D10 D6, D14

Hnh 9.2 - Mt phn cy quyt nh xy dng c Bt u vi bng y gm 14 v d rn luyn, ID3 chn thuc tnh quang cnh lm thuc tnh gc s dng hm chn la thuc tnh m t trong phn k tip. Trc nghim ny phn chia tp v d nh cho thy trong hnh 9.2 vi phn t ca mi phn vng c lit k bi s th t ca chng trong bng. * ID3 xy dng cy quyt nh theo gii thut sau: Function induce_tree(tp_v_d, tp_thuc_tnh) begin if mi v d trong tp_v_d u nm trong cng mt lp then return mt nt l c gn nhn bi lp else if tp_thuc_tnh l rng then return nt l c gn nhn bi tuyn ca tt c cc lp trong tp_v_d else begin chn mt thuc tnh P, ly n lm gc cho cy hin ti; xa P ra khi tp_thuc_tnh; vi mi gi tr V ca P begin to mt nhnh ca cy gn nhn V; t vo phn_vngV cc v d trong tp_v_d c gi tr V ti thuc tnh P; Gi induce_tree(phn_vngV, tp_thuc_tnh), gn kt qu vo nhnh V end end end ID3 p dng hm induce_tree mt cch quy cho tng phn vng. V d, phn vng ca nhnh m u c cc v d ton dng, hay thuc lp C, nn ID3 to mt nt l vi nhn l lp C. Cn phn vng ca hai nhnh cn li va c v d m, va c v d dng. Nn tip tc chn thuc tnh m lm trc nghim cho nhnh Nng, v thuc tnh Gi cho nhnh Ma, v cc v d trong cc phn vng con ca cc nhnh cy ny u thuc cng mt lp, nn gii thut ID3 kt thc v ta c c cy Q nh hnh 9.3.

158

V Hunh Trm Trn Ngn Bnh

Chng 9: Hc my

+: :

D3, D4, D5, D7, D9, D10, D11, D12, D13 D1, D2, D6, D8, D14

Quang cnh? Nng

+ : D9, D11 : D1, D2, D8

m u
+ : D3, D7, D12, D13 : Yes

Ma
+ : D3, D5, D10 : D6, D14

m? Cao
+: : D1, D2, D8 No TB + : D9, D11 : Yes

Gi? Mnh
+: : D6, D14 No

Nh
+ : D4, D5, D10 Yes

Hnh 9.3 - Cy quyt nh xy dng xong. Lu , phn loi mt v d, c khi cy Q khng cn s dng tt c cc thuc tnh cho, mc d n vn phn loi ng tt c cc v d.

V Hunh Trm Trn Ngn Bnh

159

Gio Trnh Tr Tu Nhn To * Cc kh nng c th c ca cc phn vng (partition): Trong qu trnh xy dng cy Q, phn vng ca mt nhnh mi c th c cc dng sau: 1. C cc v d thuc cc lp khc nhau, chng hn nh c c v d m v dng nh phn vng Quang cnh = Nng ca v d trn => gii thut phi tip tc tch mt ln na. 2. Tt c cc v d u thuc cng mt lp, chng hn nh ton m hoc ton dng nh phn vng Quang cnh = m u ca v d trn => gii thut tr v nt l vi nhn l lp . 3. Khng cn v d no => gii thut tr v mc nhin 4. Khng cn thuc tnh no => ngha l d liu b nhiu, khi gii thut phi s dng mt lut no x l, chng hn nh lut a s (lp no c nhiu v d hn s c dng gn nhn cho nt l tr v). T cc nhn xt ny, ta thy rng c mt cy Q n gin, hay mt cy c chiu cao l thp, ta nn chn mt thuc tnh sao cho to ra cng nhiu cc phn vng ch cha cc v d thuc cng mt lp cng tt. Mt phn vng ch c v d thuc cng mt lp, ta ni phn vng c tnh thun nht. Vy, chn thuc tnh kim tra c th gim thiu chiu su ca cy Q, ta cn mt php o o tnh thun nht ca cc phn vng, v chn thuc tnh kim tra to ra cng nhiu phn vng thun nht cng tt. ID3 s dng l thuyt thng tin thc hin iu ny.

II.3

Thuc tnh no l thuc tnh dng phn loi tt nht?

Quinlan (1983) l ngi u tin xut vic s dng l thuyt thng tin to ra cc cy quyt nh v cng trnh ca ng l c s cho phn trnh by y. L thuyt thng tin ca Shannon (1948) cung cp khi nim entropy o tnh thun nht (hay ngc li l pha trn) ca mt tp hp. Mt tp hp l thun nht nu nh tt c cc phn t ca tp hp u thuc cng mt loi, v khi ta ni tp hp ny c pha trn l thp nht. Trong trng hp ca tp v d, th tp v d l thun nht nu nh tt c cc v d u c cng gi tr phn loi. Khi tp v d l thun nht th c th ni: ta bit chc chn v gi tr phn loi ca mt v d thuc tp ny, hay ta c lng thng tin v tp l cao nht. Khi tp v d c pha trn cao nht, ngha l s lng cc v d c cng gi tr phn loi cho mi loi l tng ng nhau, th khi ta khng th on chnh xc c mt v d c th c gi tr phn loi g, hay ni khc hn, lng thng tin ta c c v tp ny l t nht. Vy, iu ta mong mun y l lm sao chn thuc tnh hi sao cho c th chia tp v d ban u thnh cc tp v d thun nht cng nhanh cng tt. Vy trc ht, ta cn c mt php o o thun nht ca mt tp hp, t mi c th so snh tp v d no th tt hn. Phn k tip s trnh by cng thc tnh entropy ca mt tp hp.

160

V Hunh Trm Trn Ngn Bnh

Chng 9: Hc my

II.3.1

Entropy o tnh thun nht ca tp v d

Khi nim entropy ca mt tp S c nh ngha trong L thuyt thng tin l s lng mong i cc bt cn thit m ha thng tin v lp ca mt thnh vin rt ra mt cch ngu nhin t tp S. Trong trng hp ti u, m c di ngn nht. Theo l thuyt thng tin, m c di ti u l m gn log2p bits cho thng ip c xc sut l p. Trong trng hp S l tp v d, th thnh vin ca S l mt v d, mi v d thuc mt lp hay c mt gi tr phn loi. Entropy c gi tr nm trong khong [0..1], Entropy(S) = 0 tp v d S ch ton v d thuc cng mt loi, hay S l thun nht. Entropy(S) = 1 tp v d S c cc v d thuc cc loi khc nhau vi pha trn l cao nht. 0 < Entropy(S) < 1 tp v d S c s lng v d thuc cc loi khc nhau l khng bng nhau. n gin ta xt trng hp cc v d ca S ch thuc loi m (-) hoc dng (+). Hnh 9.4 minh ha s ph thuc ca gi tr entropy vo xc sut xut hin ca v d dng. Cho trc: Tp S l tp d liu rn luyn, trong thuc tnh phn loi c hai gi tr, gi s l m (-) v dng (+) p+ l phn cc v d dng trong tp S. p_ l phn cc v d m trong tp S.

Khi , entropy o pha trn ca tp S theo cng thc sau: Hnh 9.4 - Entropy(S)

Entropy( S ) = p+ log 2 p+ p log 2 p

Mt cch tng qut hn, nu cc v d ca tp S thuc nhiu hn hai loi, gi s l c c gi tr phn loi th cng thc entropy tng qut l:

Entropy ( S ) = pi log 2 pi
i =1

II.3.2

Lng thng tin thu c o mc gim entropy mong i

Entropy l mt s o o pha trn ca mt tp v d, by gi chng ta s nh ngha mt php o hiu sut phn loi cc v d ca mt thuc tnh. Php o ny gi l lng thng tin

V Hunh Trm Trn Ngn Bnh

161

Gio Trnh Tr Tu Nhn To thu c, n n gin l lng gim entropy mong i gy ra bi vic phn chia cc v d theo thuc tnh ny. Mt cch chnh xc hn, Gain(S,A) ca thuc tnh A, trn tp S, c nh ngha nh sau:

Gain( S , A) = Entropy ( S )

| Sv | Entropy ( S v ) | S | vValues ( A )

trong Values(A) l tp hp c th c cc gi tr ca thuc tnh A, v Sv l tp con ca S cha cc v d c thuc tnh A mang gi tr v. Cu hi : p dng gii thut ID3 kt hp vi cng thc entropy v gain to ra cy quyt nh n gin nht cho bi ton phn loi xem ta c i chi tennis t tp d liu rn luyn cho trong bng 9.1.

II.4

Tm kim khng gian gi thuyt trong ID3

Cng nh cc phng php hc quy np khc, ID3 cng tm kim trong mt khng gian cc gi thuyt mt gi thuyt ph hp vi tp d liu rn luyn. Khng gian gi thuyt m ID3 tm kim l mt tp hp cc cy quyt nh c th c. ID3 thc hin mt php tm kim t n gin n phc tp, theo gii thut leo-ni (hill climbing), bt u t cy rng, sau dn dn xem xt cc gi thuyt phc tp hn m c th phn loi ng cc v d rn luyn. Hm nh gi c dng hng dn tm kim leo ni y l php o lng thng tin thu c. T cch nhn ID3 nh l mt gii thut tm kim trong khng gian cc gi thuyt, ta c mt s nhn xt nh sau: Khng gian gi thuyt cc cy quyt nh ca ID3 l mt khng gian y cc cy quyt nh trn cc thuc tnh cho trong tp rn luyn. iu ny c ngha l khng gian m ID3 tm kim chc chn c cha cy quyt nh cn tm. Trong khi tm kim, ID3 ch duy tr mt gi thuyt hin ti. V vy, gii thut ny khng c kh nng biu din c tt c cc cy quyt nh khc nhau c kh nng phn loi ng d liu hin c.

Hnh 9.5 - Khng gian tm kim ca ID3.

Gii thut thun ID3 khng c kh nng quay lui trong khi tm kim. V vy, n c th gp phi nhng hn ch ging nh gii thut leo ni, l hi t v cc tiu a phng.

162

V Hunh Trm Trn Ngn Bnh

Chng 9: Hc my V ID3 s dng tt c cc v d mi bc a ra cc quyt nh da trn thng k, nn kt qu tm kim ca ID3 rt t b nh hng bi mt vi d liu sai (hay d liu nhiu). Trong qu trnh tm kim, gii thut ID3 c xu hng chn cy quyt nh ngn hn l nhng cy quyt nh di. y l tnh cht thin lch quy np ca ID3.

II.5

nh gi hiu sut ca cy quyt nh:

Mt cy quyt nh sinh ra bi ID3 c nh gi l tt nu nh cy ny c kh nng phn loi ng c cc trng hp hay v d s gp trong tng lai, hay c th hn l c kh nng phn loi ng cc v d khng nm trong tp d liu rn luyn. nh gi hiu sut ca mt cy quyt nh ngi ta thng s dng mt tp v d tch ri, tp ny khc vi tp d liu rn luyn, nh gi kh nng phn loi ca cy trn cc v d ca tp ny. Tp d liu ny gi l tp kim tra (validation set). Thng thng, tp d liu sn c s c chia thnh hai tp: tp rn luyn thng chim 2/3 s v d v tp kim tra chim 1/3.

II.6

Chuyn cy v cc lut

Thng thng, cy quyt nh s c chuyn v dng cc lut thun tin cho vic ci t v s dng. V d cy quyt nh cho tp d liu rn luyn trong bng 9.1 c th c chuyn thnh mt s lut nh sau :

If (Quang-cnh =nng) ( m = Cao) Then Chi-Tennis = No If (Quang-cnh =nng) ( m = TB) Then Chi-Tennis = Yes If (Quang-cnh =m u) Then Chi-Tennis = Yes

V Hunh Trm Trn Ngn Bnh

163

Gio Trnh Tr Tu Nhn To

II.7

Khi no nn s dng ID3

Gii thut ID3 l mt gii thut hc n gin nhng n ch ph hp vi mt lp cc bi ton hay vn c th biu din bng k hiu. Chnh v vy, gii thut ny thuc tip cn gii quyt vn da trn k hiu (symbol based approach). Tp d liu rn luyn y bao gm cc v d c m t bng cc cp Thuc tnh gi tr, nh trong v d Chi tennis trnh by trong sut chng ny, l Gi mnh, hay Gi nh, v mi v d u c mt thuc tnh phn loi, v d nh chi_tennis, thuc tnh ny phi c gi tr ri rc, nh c, khng. Tuy nhin, khc vi mt s gii thut khc cng thuc tip cn ny, ID3 s dng cc v d rn luyn dng xc sut nn n c u im l t b nh hng bi mt vi d liu nhiu. V vy, tp d liu rn luyn y c th cha li hoc c th thiu mt vi gi tr mt s thuc tnh no . Mt gii php thng c p dng i vi cc d liu b thiu l s dng lut a s, chng trnh tin x l d liu s in vo cc v tr cn trng gi tr c tn s xut hin cao nht ca thuc tnh . Bn cnh cc vn c bn c trnh by trong phn ny, ID3 cn c tho lun nhiu vn lin quan nh lm sao trnh cho cy quyt nh khng b nh hng qu nhiu (overfitting) vo d liu rn luyn, n c th tng qut hn, phn loi ng c cho cc trng hp cha gp. C nhiu gii php c a ra nh ct ta li cy quyt nh sau khi hc, hoc ct ta cc lut sau khi chuyn cy v dng lut. Mt vn khc na l nu nh mt vi thuc tnh no c gi tr lin tc th sao. Gii quyt cc vn ny dn n vic sinh ra nhiu th h sau ca ID3, mt gii thut ni bt trong s l C4.5 (Quinlan 1996). Ngoi ra, mt s k thut c to ra thao tc trn d liu nhm to ra cc cy quyt nh khc nhau trn cng tp d liu rn luyn cho nh k thut bagging and boosting.

III TIP CN KT NI: MNG NEURON

III.1 Gii thiu
Cc m hnh hc theo tip cn ny bt chc theo cch hc ca cc h thn kinh sinh vt. Cc h thng theo m hnh ny c khi cn c gi l cc h kt ni (connectionist systems), tnh ton neural (Neural computing), mng neural (Neural Networks), cc h x l phn tn song song (parallel distributed processing PDP). Khng ging nh cc gii thut ca tip cn k hiu, cc m hnh ny khng s dng k hiu mt cch tng minh gii quyt vn . Thay vo , chng gi cho tr tu pht trin trong cc h thng gm cc thnh phn (neuron sinh hc hay neuron nhn to) n gin, tng tc thng qua mt qu trnh hc hay thch nghi m nh kt ni gia cc thnh phn ny c iu chnh. Vic x l ca cc h thng ny c phn tn trn mt tp hp cc lp neuron. Cc h thng ny gii quyt vn song song theo ngha rng tt c cc neuron trong tp hp hay trong cc lp s x l tn hiu vo mt cch ng thi v c lp. Trong khi cc gii thut ca tip cn k hiu s dng k hiu m t cc mu ca bi ton nh ta thy trong gii thut ID3 th nhng nh thit k mng neuron phi to ra mt s 164 V Hunh Trm Trn Ngn Bnh

Chng 9: Hc my m ha cc mu (pattern) ca bi ton thnh cc i lng s a vo mng. Vic chn la mt s m ha thch hp ng vai tr quyt nh cho s thnh cng hay tht bi trong vic hc ca mng. Cc mu (pattern) ca bi ton c m ha thnh cc vector s. Cc kt ni gia cc thnh phn, hay neuron, cng c biu din bng cc gi tr s. Cui cng, s bin i ca cc mu cng l kt qu ca cc php ton s hc, thng thng l php nhn ma trn. S chn la kin trc kt ni ca nh thit k mng neuron gp phn vo tnh thin lch quy np (inductive bias) ca h thng. Cc gii thut v kin trc dng ci t mng neuron thng c hun luyn (trained) hay to iu kin (conditioned) ch khng c lp trnh mt cch tng tn. V y chnh l sc mnh ch yu ca tip cn ny. Cc phng php ca tip cn ny pht huy sc mnh ca chng trong cc bi ton m kh c th gii quyt bng cc m hnh k hiu. Tiu biu l cc bi ton i hi cc k nng da vo nhn thc, hay cc bi ton thiu mt c php nh ngha r rng. Cc bi ton thch hp vi tip cn kt ni thng l: Bi ton phn loi (classification): quyt nh mt gi tr a vo thuc loi hay nhm no, Bi ton nhn dng mu (pattern recognition): nhn dng cu trc trong cc d liu c th l b nhiu. Bi ton d on (prediction): chng hn nh nhn dng bnh t cc triu chng, nhn dng tc nhn t cc hiu ng, Bi ton ti u (optimization): tm mt t chc rng buc tt nht Bi ton lc nhiu (Noise filtering): hay phn bit cc tn hiu vi nn, tm ra cc thnh phn khng quan trng trong mt tn hiu.

III.2 C bn v mng kt ni:

Thnh phn c bn ca mt mng neuron l mt neuron nhn to, nh m t trong hnh 9.6 sau y.

III.2.1 Mt neuron nhn to

Hnh 9. 6 minh ha mt neuron nhn to bao gm:

Hnh 9.6 - Mt neuron nhn to. Cc tn hiu u vo, xi. Cc d liu ny c th n t mi trng, hay c kch hot t cc neuron khc. Cc m hnh khc nhau c th c min gi tr ca u vo khc nhau; thng thng cc gi tr u vo ny l cc s ri rc (discrete) ly t tp {0,1} hay {-1,1} hay s thc.

V Hunh Trm Trn Ngn Bnh

165

Gio Trnh Tr Tu Nhn To Mt tp cc trng s (weight) c gi tr thc, wi. Cc trng s ny dng m t sc mnh kt ni, hay sc mnh ca cc kt ni thin lch (bias link) Mt mc kch hot (activation level) hay hm kch hot wixi. Mc kch hot ca mt neuron c xc nh bi sc mnh tch ly t cc tn hiu u vo ca n ni m mi tn hiu u vo c t l li bng trng s kt ni wi u vo . V vy, mc kch hat c tnh ton bng cch ly tng cc gi tr u vo sau khi c t l ha, wixi. Mt hm ngng (threshold function), f. Hm ny tnh kt qu u ra ca neuron bng cch xc nh xem mc kch hot nm di hay trn mt gi tr ngng l t hay nhiu. Hm ngng ny c khuynh hng to ra trng thi tt/m ca cc neuron.

III.2.2 Cc c trng ca mt mng Neuron

Ngoi cc tnh cht ca mt neuron n l, mt mng neuron cn c c trng bi cc tnh cht ton cc nh sau: Hnh thi mng (network topology): l m hnh hay mu kt ni gia cc neuron n l.
I1

w11 w12
H1 O1 H2 I1

w11
H1

I2

w12
I2

O1

I3

wi

Hnh 9.7 - Cc hnh thi mng neuron khc nhau. Gii thut hc (learning algorithm): l gii thut dng iu chnh cc trng s cc u vo ca cc neuron. Trong cc phn tip theo ca chng ny s trnh by mt s gii thut hc tiu biu. S m ha (encoding schema): Bao gm vic thng dch d liu thc t thnh cc gi tr u vo ca mng, v vic thng dch gi tr u ra ca mng thnh mt kt qu c ngha.

III.2.3 Mng neuron McCulloch-Pitts

V d u tin v tnh ton neural c MacCulloch v Pitts a ra vo 1943. u vo ca mt neuron McCulloch-Pitts l +1 (kch thch) hoc 1 (c ch). Hm kch hot nhn mi u vo vi gi tr trng s tng ng v cng chng li; nu tng ln hn hay bng khng, th neuron tr v 1, ngc li, l 1. McCulloch-Pitts cho thy cc neuron ny c th c xy dng tnh ton bt c hm logic no, chng minh rng cc h thng gm cc neuron ny cung cp mt m hnh tnh ton y . Hnh 9.8 minh ha cc neuron McCulloch-Pitts dng tnh hm logic and v or.

166

V Hunh Trm Trn Ngn Bnh

Chng 9: Hc my

Hnh 9.8 - Cc neuron McCulloch-Pitts dng tnh ton cc hm logic and v or. Cc neuron ny c 3 u vo: x v y l cc gi tr cn a vo, cn u vo th ba, i khi cn c gi l mt thin lch (bias), c gi tr hng l +1. Ba u vo ca neuron and c 3 trng s tng ng l +1, +1 v 2. V vy, vi cc gi tr bt k ca x, y, neuron tnh gi tr x+y-2; nu gi tr ny nh hn 0, n tr v 1. Ngc li tr v 1. Bng bn di minh ha cho tnh ton neuron x and y.

Bng 9.2 - M hnh McCulloch-Pitts cho logic and. Mc d McCulloch-Pitts chng minh cho sc mnh ca tnh ton neural, nhng sc hp dn ca tip cn ny ch thc s bt u khi cc gii thut hc thc t bt u pht trin. Phin bn u tin ca mng neuron c km gii thut hc c Frank Rosenblatt a ra vo cui thp nin 1950, c tn gi l perceptron.

III.3 Hc perceptron
III.3.1 Gii thut hc perceptron
Perceptron l mng neuron n tng. Cch lan truyn tn hiu ca perceptron tng t vi neuron McCulloch-Pitts. Cc gi tr u vo v cc mc kch hot ca perceptron l -1 hoc 1; trng s l cc s thc. Mc kch hot c xc nh qua tng w ixi. Perceptron s dng mt hm ngng gii hn cng, khi mt kch hot nm bn trn ngng, hm s cho kt qu l 1, v -1 nu ngc li. Cho trc cc gi tr u vo xi, cc trng s wi, v mt ngng, t, hm ngng f ca perceptron s tr v: 1 -1 nu w ixi >= t nu w ixi < t

Perceptron s dng mt hnh thc n gin ca hc c gim st (supervised learning). Sau khi perceptron c gng gii quyt mt mu bi ton (mu ny rt ra t tp d liu rn luyn

V Hunh Trm Trn Ngn Bnh

167

Gio Trnh Tr Tu Nhn To training data), chng trnh ng vai tr nh mt ngi thy gio s cung cp cho n kt qu ng ca mu bi ton (gi tr ny cng ly t tp d liu rn luyn). Da vo s khc bit gia kt qu ng c cung cp v kt qu m perceptron tnh ton c, n s t iu chnh cc trng s ca n lm thu hp khong cch li. Perceptron s dng lut nh sau: vi c l mt hng s cho trc, hng s ny th hin tc hc v d l gi tr u ra mong mun, perceptron s iu chnh trng s trn thnh phn th i ca vect u vo mt lng w i: w i = c(d f( w ixi)) xi f( w ixi) chnh l gi tr u ra ca perceptron, n c gi tr +1 hoc -1. V vy, hiu gia d v f( w ixi) l 0, 2 hoc -2. V vy, vi mi thnh phn ca vect u vo: Nu gi tr u ra mong mun v gi tr u ra tht bng nhau, th khng lm g c. Nu gi tr u ra th c l -1 v 1 l gi tr mong mun, th tng trng s ca ng th i ln 2cxi. Nu gi tr u ra th c l 1 v -1 l gi tr mong mun, th gim trng s ca ng th i -2cxi

S d c c gi l hng s th hin tc hc v nu c ln th cc gi tr iu chnh w i s ln, nh vy, y nhanh qu trnh w i hi t v gi tr ng ca n. Sau khi c hun luyn bng mt tp hp kh ln cc v d rn luyn cho trc, th tc ny s sinh ra mt tp cc trng s c tnh cht lm gim thiu trung bnh li trn ton tp v d rn luyn. Theo Minsky v Papert 1969, nu tn ti mt tp hp cc trng s em li u ra ng cho mi v d rn luyn, th th tc hc perceptron s hc c n.

III.3.2 S dng mng perceptron cho bi ton phn loi .III.3.2.1 Bi ton phn loi

Hnh 9.9 thng phn loi y .

- Mt h

Hnh trn a ra mt ci nhn khi qut v bi ton phn loi. D liu th t mt khng gian cc im c th c sau khi qua b chuyn i (transducer) s c chn v chuyn i thnh mt khng gian cc d liu hay mu mi. Trong khng gian mu mi ny, b trch lc c trng (feature extractor) s chn ra cc c trng ca d liu, v cui cng, d liu th hin

168

V Hunh Trm Trn Ngn Bnh

Chng 9: Hc my qua cc c trng s c a vo my phn loi (classifier) cho ra kt qu lp phn loi (class) ca d liu . Trong dy chuyn ny, mng perceptron ni ring v mng neuron ni chung ng vai tr nh mt my phn loi. Mt d liu u vo s c biu din nh mt vect gm n thnh phn (th hin cho n c trng) x1, x2, , xn. Cc d liu ny c th thuc 1 trong m lp class1, class2, classm. My phn loi s c nhim v xc nh xem d liu u vo thuc v lp no.

.III.3.2.2 V d perceptron
Trong v d n gin di y, b chuyn i v b trch lc c trng chuyn thng tin ca bi ton thnh cc tham s ca khng gian hai chiu. Bng 9.3 th hin d liu rn luyn ca perceptron gm c 2 c trng (x1 v x2) mang gi tr thc, v kt qu mong mun (output) gm hai gi tr 1 hoc -1, th hin cho hai lp phn loi ca d liu. Hnh 9.10 th hin hnh nh ca cc im d liu trong bng 9.3 cng vi ng phn cch hai lp d liu c to ra sau khi mng percpetron c hun luyn trn tp d liu ny.

Bng 9.3 - Tp d liu cho bi ton phn loi ca perceptron.

Hnh 9.10 - th hai chiu ca cc im d liu trong bng 9.3. Perceptron cung cp mt php tch tuyn tnh ca cc tp hp d liu.

Perceptron dng phn loi bi ton ny c thit k nh sau: Tn hiu u vo: 2 tham s x1 v x2, cng vi mt u vo thin lch (bias) lun c gi tr 1. Hnh 9.11- Mng perceptron cho v d ca bng 9.3. + w3 Hm ngng f(net) l mt hm du, hay cn gi l hm ngng hai cc tuyn tnh Mc kch hot: net = w1x1 + w2x2

V Hunh Trm Trn Ngn Bnh

169

Gio Trnh Tr Tu Nhn To

f(net) = +1, nu net>0 f(net) = -1, nu net<=0

Tn hiu thin lch phc v cho vic chuyn dch hm ngng trn trc tung. Phm vi ca chuyn dch ny s c hc bng cch iu chnh trng s w3 trong khi hun luyn mng. By gi chng ta s dng cc im d liu ca bng 9.3 luyn tp perceptron ca hnh 9.6. Gi thit ban u cc trng s wi c gi tr ngu nhin ln lt l: [0.75,0.5,-0.6] S dng gii thut hc perceptron trnh by trn vi tc hc c c chn l mt s dng nh 0.2 Chng ta bt u bng v d u tin trong bng 9.3: f(net)1 = f(0.75 * 1 + 0.5 * 1 0.6 *1 ) = f(0.65) = 1 Ta thy f(net)1 c gi tr ng vi gi tr u ra mong mun, nn ta khng iu chnh trng s. Cho nn W 2 = W 1 , W l vect trng s i din cho 3 trng s w1, w2,w3. n v d th 2: f(net)2 = f(0.75 * 9.4 + 0.5 * 6.4 0.6 *1 ) = f(9.65) = 1 Nhng gi tr mong i y l -1, v vy, ta cn iu chnh trng s theo lut hc: W t = Wt-1 + c( dt-1 f(net)t-1)Xt-1 trong : vo t l thi im Trong trng hp ny: c = 0.2, d2 = -1, v f(net)2 = 1 p dng lut hc trn, ta c:
0.75 9.4 3.01 W = W + 0.2(1 1)X = 0.5 0.4 6.4 = 2.06 0.6 1.0 1.00
3 2 2

c l hng s hc W, X: l vect trng s, v vect d liu u

By gi chng ta xt v d th 3: f(net)3 = f(-3.01 * 2.5 -2.06 * 2.1 1.0 *1 ) = f(-12.84) = -1 170 V Hunh Trm Trn Ngn Bnh

Chng 9: Hc my Trong khi gi tr mong i ca v d ny l 1, nn cc trng s tip tc c iu chnh

3.01 2.5 2.01 W = W + 0.2(1( 1))X = 2.06 + 0.4 2.1 = 1.22 1.0 1.0 0.60
4 3 3

C tip tc nh th, sau 10 ln lp, ng phn cch tuyn tnh nh trong hnh 9.10 xut hin. Sau khi lp li vic hun luyn perceptron trn tp d liu cho, khong 500 ln lp tng cng, vect trng s hi t v gi tr [-1.3, -1.1, 10.9]. V y chnh l cc h s ca phng trnh ng phn cch tuyn tnh trong hnh 9.10: -1.3 * x1 1.1 * x2 + 10.9 = 0.

III.3.3 Gii hn ca perceptron tnh tch ri tuyn tnh ca bi ton

Ban u, mng perceptron c cho n mt cch nhit tnh. Tuy nhin, Nils Nilsson (1965) v nhng ngi khc phn tch nhng gii hn ca m hnh perceptron. H chng minh rng perceptron khng th gii quyt mt lp cc bi ton kh, c th l cc bi ton m cc im d liu khng th tch ri tuyn tnh. Mt v d cho bi ton phn loi khng tuyn tnh l php ton ex-or. Ex-or c bng chn l nh sau: x1 1 1 0 0 x2 1 0 1 0 Output 0 1 1 0

Bng 9.4 - Bng chn l ca php ton logic ex-or.

Hnh 9.12 - th th hin cc im d liu ca bi ton Ex-Or

T th hnh 9.12, ta thy khng th tm c bt c mt ng thng no c th tch ri cc im d liu ca lp 0: {(0,0), (1,1)} ra khi cc im d liu ca lp 1: {(0,1),(1,0)}. Chng ta c th quan nim tp hp cc gi tr d liu i vi mt mng neuron nh dng nh ngha mt khng gian. Mi tham s ca d liu u vo tng ng vi mt chiu trong khng gian, v mi gi tr u vo s nh ngha mt im trong khng gian . Trong bi ton ex-or trn, bn gi tr u vo, c nh ch s theo cc ta x1, x2, to thnh cc im d liu trong hnh 9.12. Bi ton hc mt phn loi nh phn ca cc d liu rn luyn rt li thnh bi ton tch cc im ny thnh hai nhm. Nh vy, i vi mt khng gian n chiu, mt s phn loi l phn tch c mt cch tuyn tnh nu nh cc lp ca n c th c tch ra bi mt mt phng n-1 chiu. (Trong khng gian hai chiu th mt siu phng n1 chiu l mt ng thng, trong khng gian 3 chiu th n l mt mt phng, ).

V Hunh Trm Trn Ngn Bnh

171

Gio Trnh Tr Tu Nhn To

III.3.4 Lut Delta

Mt cch d dng tng qut ha mng perceptron l thay hm ngng gii hn cng bng mt hm kch hot kiu khc. Chng hn nh, hm kch hot lin tc ( c kh nng ly vi phn) to iu kin cho cc gii thut hc phc tp hn.

Hnh 9.13 - Cc hm ngng. Hnh 9.13 minh ha th ca mt s hm ngng: hnh 9.13a l mt hm ngng hai cc, hnh 9.13b minh ha mt hm kch hot sigmoidal thng dng (hm sigmoidal l hm c hnh cong nh ch S), c gi l hm logistic, hm ny c cng thc nh sau: f(net) = 1/(1 + e-*net) vi net = i wixi Cng nh cc hm nh ngha trc y, xi l u vo ca ng th i, wi l trng s trn ng vo i, v l tham s nn c s dng iu chnh cong ca ng. Khi cng ln, th ng cong cng tim cn vi hm ngng tuyn tnh (trong hnh 9.13a). Khi cng tin gn n 1, n cng gn nh l mt ng thng. Hm logistic c mt tnh cht c bit l, o hm ca hm ny c mt cng thc rt n gin: f (net) = f(net) * (1- f(net)) (1-1)

T hnh 9.13 ta thy vi cc hm ngng lin tc, neuron s cho kt qu chnh xc hn nh vo vic iu chnh tham s . Vic a ra cc hm kch hot lin tc lm xut cc tip cn mi lm gim li trong khi hc. Qui lut hc do Widrow-Hoff a ra vo khong 1960 c lp vi hm kch hot, ti thiu ha bnh phng ca li gia gi tr u ra mong mun v kch hot ca neuron, neti = WXi. Mt trong s lut hc quan trng cho cc hm kch hot lin tc l lut delta (Rumelhart et al. 1986). s dng lut delta, mng phi s dng mootj hm ngng lin tc c th ly vi phn. Hm logistic trnh by bn trn c c tnh cht ny. Khi cng thc hc theo lut delta cho vic iu chnh trng s u vo th j ca nt th i l: w = c(di Oi) f(neti)xj 172 V Hunh Trm Trn Ngn Bnh

Chng 9: Hc my trong , c l hng s iu khin tc hc, di v Oi l cc gi tr u ra thc s v mong mun ca nt th i. f(neti) l o hm ca hm kch hot cho nt th i, v xj l u vo th j ca nt th i. Thay th cng thc o hm (1-1) ca hm logistic f(net), ta c cng thc iu chnh trng s nh sau: w = c(di Oi) Oi( 1 Oi) xj (1-2)

T cng thc ny cho thy, cng thc iu chnh trng s ny ch c th p dng cho cc nt ca mng perceptron n tng, v ti ta mi c cc gi tr u ra mong mun di.

III.4 Hc Lan truyn ngc:

Nh phn tch trn, ta thy cc mng perceptron n tng c kh nng gii hn, chng khng th phn loi c cc bi ton khng tch ri tuyn tnh. Trong phn tip theo, chng ta s thy rng cc mng a tng c th gii quyt c cc bi ton ny.

Hnh 9.14 - Hc lan truyn ngc trong mng kt ni c mt tng n. Cc neuron trong mt mng a tng (xem hnh 9.14) c kt ni vi nhau theo tng lp, trong cc neuron tng k s truyn kch hot ca chng ch cho cc neuron tng k+1. X l tn hiu a tng c ngha l cc li nm su bn trong mng c th lan ra v pht trin mt cch phc tp thng qua cc tng lin tip. V vy, vic phn tch nguyn nhn gy ra li tng ra (output layer) l rt phc tp. Gii thut hc lan truyn ngc s cung cp mt phng php iu chnh trng s trong trng hp ny.

III.4.1 Gii thut hc lan truyn ngc

T lp lun cho rng ti cc nt ca mt mng a tng, li m mt nt phi chu trch nhim cng phi c chia phn cho cc nt tng n trc n v v vy cc trng s phi c iu chnh mt cch ph hp. Gii thut lan truyn ngc bt u ti tng ra v truyn cc li ngc v xuyn qua cc tng n (nh hnh 9.14).

V Hunh Trm Trn Ngn Bnh

173

Gio Trnh Tr Tu Nhn To

Hnh 9.15 - j deltaj *wij l tng ng gp ca nt i vo li tng ra. Lut delta tng qut iu chnh trng s ca u vo th k ca nt th i: wk = c(di Oi) Oi (1 Oi) xk cho nt tng ra

wk = c j (deltaj wij) Oi (1 Oi) xk cho nt tng n vi deltaj = (dj Oj) Oj (1 Oj) j chy trn cc nt ca tng k tip m ti nt i truyn cc u ra ca n. i vi mng c nhiu hn mt tng n, ta cng p dng th tc tng t mt cch quy truyn li t tng n th n vo tng n th n-1.

III.4.2 V d 1: Mng NetTalk

Mng NETtalk l mt v d hay cho vic s dng gii php mng neuron gii quyt mt vn hc kh. NETtalk hc c c vn bn ting Anh. y l mt nhim v kh khn i vi tip cn hc da trn k hiu, v pht m trong ting Anh mang tnh bt quy tc. Mc d c cc chng trnh da trn lut (rule-based) c vit gii quyt vn ny, nhng chng u phc tp v thc hin cha hon ho. NETtalk hc c mt chui vn bn v tr v mt m v cng vi trng m lin h cho mi ch ci trong chui. V pht m ca mt ch ci n nht ph thuc vo cc ch ci xung quanh n, ngi ta a vo NETtalk mt ca s gm 7 k t. Khi vn bn dch chuyn qua ca s ny, NETtalk tr v mt cp m v/trng m cho mi ch ci. Hnh 9.15 minh ha kin trc ca mng NETtalk. Mng gm c 3 tng neuron. Cc neuron u vo tng ng vi ca s 7 k t ca vn bn. Mi v tr trong ca s c biu din bi 29 neuron u vo, 26 neurons cho 26 k t alphabet, v 3 neurons cho du v khong trng. K t mi v tr trong ca s s kch hot neuron tng ng. Cc neuron u ra m ha m s dng 21 c im khc nhau ca cch pht m ca con ngi. 5 neurons cn li m ha du nhn v gii hn m tit. NETtalk c 80 neuron tng n, 26 gi tr u ra v 18.629 kt ni.

174

V Hunh Trm Trn Ngn Bnh

Chng 9: Hc my

Hnh 9.15 - Hnh thi mng ca NETtalk. Kt qu ca NETtalk l c th pht m ng 60% sau khi rn luyn vi mt tp d liu rn luyn gm 500 v d v lp li 100 lt. Ngoi kt qu t c trn, NETtalk cn cho thy mt s tnh cht ng ch ca mng neuron, c nhiu tnh cht trong s phn nh bn cht t nhin ca vic hc ngi. Chng hn nh, vic hc, khi c o bng phn trm cu tr li ng, s tin trin nhanh lc u, sau chm dn khi t l ng tng ln. V cng nh con ngi, khi neuron cng hc pht m c nhiu t, th n cng pht m ng cc t mi nhiu hn.

III.4.3 V d 2: Exclusiveor
Mt v d khc cho mng a tng l dng gii quyt bi ton Ex-or m mng n tng khng th phn loi c. Hnh 9.16 minh ha mng vi hai u vo, mt nt n v mt nt u ra. Mng cng c hai u vo thin lch (bias), mt i vo nt n v mt i vo nt u ra. Mt im c bit l cc u vo cng c ni trc tip vo nt u ra. Lin kt thm vo ny cho php nh thit k mng neuron t c mt mng vi t nt hn trn tng n v hi t nhanh hn. Gi tr net cho nt n v nt u ra cng c tnh nh cch thng thng, l tng ca cc tch gia gi tr u nhn vi trng s. Cc trng s c iu chnh theo gii thut hc lan truyn ngc v s dng hm kch hot sigmoidal. Tht ra, mng neuron trong hnh 9.16 khng phi l mt mng duy nht c th gii quyt bi ton ny.

Hnh 9.16 - Mt mng lan truyn ngc dng gii quyt bi ton exclusive-or.

V Hunh Trm Trn Ngn Bnh

175

Gio Trnh Tr Tu Nhn To Mng ny c rn luyn vi 4 v d: (0,0) 0; (1,0) 1; (0,1) 1; (1,1) 0 Sau khi c hun luyn 1400 lt vi 4 d liu trn, cc trng s hi t v cc gi tr nh sau: WH1 = -7.0 WOB = 7.0 WHB = 2.6 WO2 = -4.0 WO1 = -5.0 WHO = -11.0 WH2 = -7.0

Vi gi tr u vo l (0,0), gi tr u ra ca nt n s l: f(0 * (-7.0) + 0 * (-7.0) + 1* 2.6 ) = f(2.6) 1 Kt qu tr v ca nt u ra cho (0,0) s l: f(0 * (-5.0) + 0 * (-4.0) + 1 * (-11.0) + 1 * (7.0)) = f(-4.0) 0 Nh vy, ta thy rng mng lan truyn ngc phn loi c cc im d liu khng tuyn tnh.

III.5 Nhn xt chung v mng neuron

Ni chung cc mng a tng l y v mt tnh ton (computationally complete), c ngha l c th gii quyt c mi bi ton. Tuy nhin, thit k mt mng neuron a tng th nh thit k phi gii quyt c nhng vn sau: Lm sao chn s nt n v s tng n thch hp? Khi no s dng cc nt thin lch? Cch chn mt tp rn luyn? iu chnh cc trng s nh th no? Nn chn tc hc nh th no?

Ni chung, khng c mt quy lut no v tt c nhng iu ny, n ph thuc vo kinh nghim ca nh thit k, cng nh l kt qu ca qu trnh th-sai lp i lp li.

176

V Hunh Trm Trn Ngn Bnh

Chng 9: Hc my

IV TIP CN X HI V NI TRI: GII THUT DI TRUYN (GENETIC ALGORITHM - GA)

IV.1 Gii thiu
Cng nh cc mng neuron, cc thut ton di truyn cng da trn mt n d sinh hc: Cc thut ton ny xem vic hc nh l s cnh tranh trong mt qun th gm cc li gii ng vin ang tin ha ca bi ton. Mt hm thch nghi (fitness function) s nh gi mi li gii quyt nh liu n c ng gp cho th h cc li gii k tip hay khng. Sau , thng qua cc php ton tng t vi bin i gene trong sinh sn hu tnh, gii thut s to ra mt qun th cc li gii ng vin mi.

IV.2 Gii thut

Khi to qun th

Tha K dng

Gi hm thch nghi nh gi cc li gii

ng vin

Chn cc ng vin tt To con mi Thay th ng vin km bng cc con mi

Chn li gii tt nht t qun th

Hnh 9.17- Gii thut di truyn. Hnh 9.17 m t gii thut di truyn tng qut. Ty theo tng bi ton m nh thit k gii thut s phi m t chi tit hn v: Phng php biu din mt c th trong qun th cc li gii ng vin ca bi ton, hay ni khc hn l hnh thc biu din mt li gii tim nng ca bi ton. Khng phi li gii ca mi bi ton u c th c m ha mt cch d dng v t nhin di dng biu din mc bit nh trong bi ton tha mn CNF di y. ln ca qun th l s lng ng vin c trong qun th. Thng thng cc ng vin ca qun th ban u c chn mt cch ngu nhin. ln ca qun th l khng i qua cc th h, v vy, s c mt qu trnh chn lc v loi b mt s li gii ng vin c thch nghi thp.

V Hunh Trm Trn Ngn Bnh

177

Gio Trnh Tr Tu Nhn To iu kin dng ca vng lp: c th l chng trnh t ti mt s ln lp nht nh no , hay t ti trung bnh tt no ca qun th, Hm nh gi (fitness function): Dng nh gi mt ng vin c tt hay khng. Mt ng vin cng tt ngha l thch nghi ca n cng cao v tin n tr thnh li gii ng ca bi ton. Vic thit k mt hm nh gi tt l rt quan trng trong thut ton di truyn. Mt hm nh gi khng chnh xc c th lm mt i cc ng vin tt trong qun th. Chn la bao nhiu phn trm li gii tt gi li? Hay chn bao nhiu li gii ng vin kt hp vi nhau v sinh ra li gii con? Phng php to thnh vin mi t thnh vin hin c, cn gi l ton t di truyn (genetic operators): Cc ton t di truyn ph bin l o Lai ghp (cross-over): Ton t lai ghp ly hai li gii ng vin v chia tng li gii ra thnh hai phn, sau trao i cc phn vi nhau to ra ng vin mi. V d xem hnh 9.18. o t bin (mutation): t bin ly mt ng vin n l v thay i mt cch ngu nhin mt kha cnh no ca n. V d xem hnh 9.18. Th h n Ti sn xut
1101 1011 1011 1001
11 13 2 9

1101

1011

0100

1001

thch nghi (fitness)

Lai ghp
1 011 01
13

101

10 01 10 00
1

1 0 01 11 1 0 01 11
13

t bin
1 0 101
10

Th h n+1

Hnh 9.18 - V d minh ha gii thut v ton t di truyn. Trong v d minh ha bng hnh 9.18, ta thy ti th h th n ta c mt li gii c thch nghi rt thp (2), v v vy, n khng c s dng trong qu trnh ti sn xut. Thay vo , li gii c thch nghi cao nht (13) s c nhn i v a vo qu trnh ti sn xut. Hoc t ph bin hn l cc ton t di truyn: o o ngc (inversion): o ngc th t cc bit trong mu li gii. o Trao i (Exchange): Trao i hai bit bt k trong mu li gii vi nhau.
1101 o ngc Trao i 1101

178

V Hunh Trm Trn Ngn Bnh

Chng 9: Hc my

1011 0111 Mt ton t di truyn tt ng mt vai tr quan trng trong thut ton di truyn. Ton t di truyn phi bo ton nhng mi quan h ct yu trong qun th; v d, s c mt v s duy nht ca tt c cc thnh ph trong hnh trnh ca ngi bn hng trong bi ton ngi i bn hng.

Thay th thnh vin mi cho cc thnh vin hin c nh th no?

IV.3 V d 1: Bi ton tha CNF

Bi ton tha mn dng chun hi (Conjunctive normal form CNF) l mt bi ton n gin: Mt biu thc ca cc mnh (clause) dng chun hi hay CNF khi n l mt dy cc bin mnh c kt ni vi nhau bi ton t quan h and (). Mi mnh c dng l mt tuyn (disjunction), gm cc ton t quan h or () trn cc bin mnh (literal). V d : Nu ta c 6 bin mnh a, b, c, d, e v f, th biu thc sau y l mt CNF: (a c) (a c e) (b c d e) (a b c) (e f) (1-3)

Tha mn CNF c ngha rng chng ta phi tm ra mt php gn true hoc false (1 hoc 0) cho mi bin mnh a, b, c, d, e, f sao cho biu thc CNF c gi tr l TRUE. Mt cch biu din t nhin cho li gii ca bi ton ny l mt dy su bit, mi bit theo th t a, b, c, d, e, f biu din true (1) hoc false (0) cho mi bin mnh . Nh vy mu bit: 101010 cho bit a, c, v e l true v b, d, v f l false, v do khi thay cc gi tr ny vo biu thc (1-3), th cho gi tr false. Chng ta mun rng cc ton t di truyn sinh ra cc th h li gii sao cho biu thc CNF mang tr true, v vy mi ton t phi sinh ra mt mu 6-bit ca php gn true cho biu thc. Cch biu din li gii di dng mt mu cc bit nh trn mang li cho ta rt mt iu rt thun li l bt k ton t di truyn no (lai ghp, t bin, o ngc, hay trao i) u to ra mt mu bit mi l mt li gii kh d hp l. Vic chn la mt hm thch nghi cho qun th cc chui bit ny khng phi hon ton d dng. Thot nhn chui bit, ta kh c th xc nh mt hm thch nghi nh gi c cht lng ca n nh th no, ngha l kh on c tt ca n so vi p n ng. p n ng y chnh l chui bit sao cho biu thc CNF c gi tr true. Tuy nhin c mt s cch khc. Nu ta ch n biu thc CNF (1-3), th ta thy rng n c to thnh t hi ca 5 mnh . Do chng ta c th thit lp mt h phn hng cho php chng ta sp hng cc li gii (mu bit) tim nng trong khong gi tr t 0 n 5, ty thuc vo s mnh m mu tha mn. Do mu: 1 1 0 0 1 0 c thch nghi l 1

V Hunh Trm Trn Ngn Bnh

179

Gio Trnh Tr Tu Nhn To 0 1 0 0 1 0 c thch nghi l 2 0 1 0 0 1 1 c thch nghi l 3 1 0 1 0 1 1 c thch nghi l 5, v n chnh l mt li gii.

IV.4 V d 2: Bi ton ngi i bn hng TSP

Bi ton ngi bn hng (traveling salesperson problem TSP) l mt bi ton c in i vi AI v khoa hc my tnh 1 . Nh chng gii thiu chng III, ton b khng gian trng thi ca n i hi phi xem xt N! trng thi c th tm ra li gii ti u, trong N l s thnh ph cn i qua. Khi N kh ln th bi ton s b bng n t hp, v vy ngi ta t vn l c cn thit hay khng cho vic chy mt my trm lm vic t tin trong nhiu gi cho mt li gii ti u hay ch nn chy mt PC r tin trong vi pht c c nhng kt qu tt. Gii thut di truyn chnh l mt gii php cho la chn th hai. bi ton ny, dng mu bit biu din cho li gii ca bi ton khng phi l mt cch hay. Chng hn, ta c chn thnh ph cn gh thm 1, 2, 9, ta xem mi thnh ph nh mt mu 4 bit 0001, 0010, 1001. Khi mt li gii kh d s c hnh thc nh sau: 0001 0010 0011 0100 0101 0110 0111 1000 1001 Vi cch biu din nh vy, vic thit k cc ton t di truyn s tr nn rt kh khn. Ton t lai ghp nht nh l khng c, v chui mi c to t hai cha m khc nhau hu nh s khng biu din mt ng i trong gh thm mi thnh ph ng mt ln. Trong thc t, vi lai ghp, mt s thnh ph c th b xa b trong khi cc thnh ph khc c gh thm nhiu hn mt ln, v v vy khng phi l mt li gii hp l. Cn ton t t bin th th no? Gi s bit tri nht ca thnh ph th su, 0110, c t bin thnh 1? 1110, hay l 14, th n khng cn l mt thnh ph hp l. Mt cch tip cn khc l s b qua biu din dng mu bit v t cho mi thnh ph mt tn theo bng ch ci hoc s, v d 1, 2, 9; xem ng i qua cc thnh ph l mt s sp th t ca chn k s ny, v sau chn ton t di truyn thch hp to ra cc ng i mi. y ta thy php trao i (exchange) ngu nhin hai thnh ph trong ng i c th s dng c, cn php ton lai ghp (crossover) th khng. Vic trao i cc on ca mt ng i vi nhng on khc ca cng ng i , hoc bt c ton t no sp xp li cc ch ci ca ng i y (m khng xa b, thm, hay nhn i bt c thnh ph no) u c th s dng c. Tuy nhin, nhng phng php ny gy kh khn cho vic a vo th h con chu nhng thnh phn tt hn ca cc mu trong cc ng i qua ca cc thnh ph ca hai cha m khc nhau.

Pht biu ca bi ton TSP: Mt ngi bn hng c nhim v gh thm N thnh ph nh l mt phn ca l trnh bn hng. ng i gia mi cp thnh ph c mt chi ph (v d nh di on ng, gi v my bay). Hy tm ra ng i c chi ph thp nht cho ngi bn hng bt u ln ng ti mt thnh ph, thm tt c cc thnh ph khc ch ng mt ln ri quay li thnh ph xut pht.

180

V Hunh Trm Trn Ngn Bnh

Chng 9: Hc my Nhiu nh nghin cu a ra cc ton t lai ghp c kh nng khc phc nhng vn ny, trong c ton t lai ghp c th t (order crossover) do Davis a ra vo nm 1985. Lai ghp c th t xy dng con chu bng cch chn mt dy con cc thnh ph trong ng i ca mt mu cha m. N cng bo ton th t tng i cc thnh ph t cha m kia. u tin, chn hai im ct, biu th bi du |, im ct ny c chen vo mt cch ngu nhin vo cng mt v tr ca mi mu cha m. Nhng im ct ny l ngu nhin, nhng mt khi c chn, th nhng v tr nh nhau s c s dng cho c hai cha m. V d, c hai mu cho m p1 v p2, vi cc im ct sau thnh ph th ba v th by: p1 = (1 9 2 | 4 6 5 7 | 8 3) p2 = (4 5 9 | 1 8 7 6 | 2 3) Hai mu con c1 v c2 s c sinh ra theo cch sau. u tin, cc on gia hai im ct s c chp vo cc mu con: c1 = (x x x | 4 6 5 7 | x x) c2 = (x x x | 1 8 7 6 | x x) Bc k tip l bt u t im ct th hai ca mt trong hai mu cha m, nu ta ang mun hon tt mu c1, th ta s bt u t im ct th hai ca mu p2, ta chp cc thnh ph t im ct ny theo th t vo cc ch cn trng ca c1, b qua nhng thnh ph m c1 c (cc k s c in m v nghing trong s bn di). Khi n cui mu p2, th quay li u mu p2 tip tc chp sang c1 cho n khi c1 . p2 = (4 5 9 | 1 8 7 6 | 2 3) p1 = (1 9 2 | 4 6 5 7 | 8 3)

c1 = (2 3 9 | 4 6 5 7 | 1 8)

c2 = (3 9 2 | 1 8 7 6 | 4 5)

Vi gii thut lai ghp ny, cc ng i ca th h con s c m bo l cc ng i hp l, i qua mi thnh ph mt ln duy nht. Tm li, trong lai ghp th t, cc mnh ca mt ng i c truyn t mt cha m, p1, sang mt con, c1, trong khi sp xp ca cc thnh ph cn li ca con c1 c tha k t cha m kia, p2. iu ny ng h cho trc gic hin nhin l th t ca cc thnh ph ng vai tr quan trng trong vic to ra ng i vi chi ph thp nht, v v vy vic truyn li cc on thng tin c th t ny t cc cha m c thch nghi cao sang con ci l mt iu rt quan trng. Ngoi ton t lai ghp th t trn, cn c rt nhiu ton t lai ghp v t bin khc c a ra gii quyt bi ton ny. Bng 9.5 lit k cc ton t lai ghp, bng 9.6 lit k cc ton t t bin.

V Hunh Trm Trn Ngn Bnh

181

Gio Trnh Tr Tu Nhn To Tn ton t Alternating Position Crossover (AP) Murga Cycle Crossover (CX) Distance Preserving Crossover (DPX) Edge Assembly Crossover (EAX) Edge Recombination Crossover (ER) Heuristic Crossover (HEU) Inver-over Operator (IOO) Maximal Preservative Crossover (MPX) Krmer Position Based Crossover (POS) Order Crossover (OX1) Order Based Crossover (OX2) Partially mapped Crossover (PMX) Voting Recombination Crossover (VR) Nm (1999) (1987) (1996) (1997) (1989) (1987) (1998) (1988) (1991) (1985) (1991) (1985) (1989) Tc gi Larranaga, Kuijpers, Poza,

Oliver, Smith and Holland Freisbein and Merz Nagata and Kobayashi Whitley, Timothy, Fuquay Grefenstette Tao and Michalewicz Mhlenbein, Schleuter Syswerda Davis Syswerda Goldberg and Lingle Mhlenbein

and

Bng 9.5 - Danh sch cc ton t lai ghp cho bi ton TSP. Tn ton t gi Displacement Mutation (DM) Michalewicz Exchange Mutation (EM) Banzhaf Insertion Mutation (ISM) Inversion Mutation (IVM) Scramble Mutation (SM) Syswerda Simple Inversion Mutation (SIM) Holland Nm (1992) (1990) (1988) (1990) (1991) (1975) Fogel Fogel Tc

Bng 9.6 - Danh sch cc ton t t bin cho bi ton TSP.

IV.5 nh gi gii thut

Cc v d va nu trn lm ni bt nhng vn mang tnh duy nht ca thut ton di truyn v biu din tri thc, chn ton t di truyn, v thit k hm thch nghi. Biu din c chn phi h tr cho cc ton t di truyn. Mt im dng lu na l cc ton t di truyn phi c thit k sao cho bo lu c nhng mnh thng tin c ngha trong li gii tim nng t th h ny sang cc th h tip theo. Mt sc mnh quan trng ca thut ton di truyn l bn cht song song trong tm kim ca n. Cc thut ton ny thc hin mt dng mnh ca leo ni (hill climbing) trong duy tr nhiu li gii (trong qun th cc li gii), loi b nhng li gii khng c trin vng, v nng cao cht lng ca nhng li gii tt. Hnh 9.19 phng theo Holland (1986), cho thy nhiu li gii hi t v cc im ti u trong khng gian tm kim. Trong hnh ny, trc honh biu din cc im c th c trong khng gian li gii, trong khi trc tung phn nh 182 V Hunh Trm Trn Ngn Bnh

Chng 9: Hc my cht lng ca nh ng li gii . Cc im chm nm trn cung l cc thnh vin ca qun th hin ti. Khi u, nhng li gii c ri trong khng gian nhng li gii c th c. Sau mt vi th h, chng c khuynh hng cm li xung quanh nhng vng c cht lng li gii cao hn.

Cht lng li gii

Cht lng li gii

Khng gian tm kim a. Khng gian TK lc khi u

Khng gian tm kim b. Khng gian TK sau n th

Hnh 9.19 - Cc thut ton di truyn c xem l leo ni song song (theo Holland 1986) Tuy nhin, vi sc mnh nh vy, gii thut genetic cng khng th p dng cho tt c cc bi ton c th c. V nh ta thy qua hai v d trn, li gii ca bi ton phi c biu din di mt dng mu thch hp cho cc ton t di truyn hot ng. Trong thc t c nhiu bi ton khng th lm c iu ny. V vy, khi nghin cu v gii thut ny, c rt nhiu cu hi c a ra nhm hiu r hn na v bn cht hot ng ca n: 1. Liu chng ta c th a ra nhng c im v cc loi bi ton m gii thut di truyn (GA) c th thc hin tt 2. Cc loi bi ton no th khng thch hp vi GA. 3. Da vo u ta c th ni l GA thc hin tt hay khng tt i vi mt loi bi ton no ? 4. Liu c nhng qui lut no m t hnh vi ca GA mc v m? Hay c th hn, l liu c bt k s phn on no v s thay i ca thch nghi ca cc nhm con trong qun th theo thi gian? 5. C cch no m t cc hiu ng khc nhau ca cc ton t di truyn nh lai ghp, t bin, o ngc, v.v 6. Trong nhng trng hp no (bi ton no, ton t di truyn no) th GA s thc hin tt hn cc phng php nghin cu ca TTNT truyn thng. Nhng cu hi ny (v cn nhiu hn na) vn v ang c cc nh khoa hc nh Holland, Mitchell, Golderg, nghin cu.

V Hunh Trm Trn Ngn Bnh

183

Gio Trnh Tr Tu Nhn To

V TNG KT CHNG IX
Ni dung chnh ca chng ny bao gm: Gii thiu tng qut v mt nhnh nghin cu mi ca Tr Tu Nhn To, l Hc my. Hc c nh ngha nh l bt c s thay i no trong mt h thng cho php n tin hnh tt hn trong ln th hai khi lp li cng mt nhim v hoc vi mt nhim v khc rt ra t cng mt qun th cc nhim v . C ba tip cn hc: Tip cn th nht l tip cn k hiu, hai l tip cn mng neuron hay kt ni v tip cn th ba l tip cn ni tri hay di truyn v tin ha. Cc chng trnh hc theo tip cn k hiu s biu din vn di dng cc k hiu. Chng ny trnh by mt gii thut c s dng rng ri ca tip cn ny, l ID3. ID3 s hc t tp d liu rn luyn bao gm rt nhiu v d, mi v d bao gm mt tp cc cp thuc tnh gi tr. Thuc tnh v gi tr y l cc k hiu. Sau khi hc xong, ID3 biu din khi nim hc c bng mt cy quyt nh. Tip cn kt ni hay mng neuron m phng h thn kinh ca con ngi hc c cc khi nim m khng s dng k hiu biu din vn . Mng n tng perceptron cho thy sc mnh ca mng neuron, tuy nhin kh nng p dng ca chng ch hn ch cho cc bi ton c tnh tch ri tuyn tnh. Mng a tng p dng gii thut hc lan truyn ngc vt qua nhng hn ch ca mng perceptron, chng t c sc mnh thc s ca tip cn ny. Tng t nh tip cn kt ni, tip cn di truyn v tin ha c cm hng bt ngun t tri thc ca con ngi v s tin ha ca sinh vt: ch c nhng c th c kh nng thch nghi vi s thay i ca mi trng th mi tn ti v pht trin. Thut ton di truyn m phng theo nguyn l .

VI BI TP CHNG IX
IX.1. Cho mt tp hp cc v d rn luyn nh sau: STT 1 2 3 4 5 6 Phn loi + + + + A1 T T F F F F A2 T F T F T F A3 F T T T F F

184

V Hunh Trm Trn Ngn Bnh

Chng 9: Hc my An mun p dng gii thut ID3 xy dng cy quyt nh vi tp d liu rn luyn trn. p dng cc cng thc tnh entropy v gain, hy gip An xc nh thuc tnh no (A1, A2, hay A3) l thuc tnh tt nht hi u tin nhm to ra mt cy quyt nh n gin nht. (Lu : phi trnh by cc tnh ton entropy v gain i n kt lun).

IX.2. ng dng gii thut di truyn tm gi tr ca cc bin x, y, z sao cho hm f(x,y,z) = ysin(zcos(x)) xcos(zsin(y)) t gi tr ln nht. Bit rng 0 < x < 10, 0< y < 10, 0 <z < 10.

IX.3. Cho mt tp hp gm 10 v d rn luyn nh sau:

STT 1 2 3 4 5 6 7 8 9 10 Cui-tun (A1) ng Sai Sai ng ng Sai Sai Sai ng ng ang-i (A2) C C Khng C Khng C Khng C Khng C TG-ch (pht) (A3) 0-10 >30 0-10 10-30 10-30 0-10 10-30 10-30 >30 >30 S-ch-bn C Khng C C Khng C khng C Khng Khng

Tp d liu trn th hin quyt nh s ch bn hay khng ca mt ngi khi bc vo mt nh hng ng khch khng cn bn trng. Quyt nh ca anh ta s ph thuc vo mt s yu t nh hm c phi l ngy cui tun khng (cui-tun) A1, anh ta c ang i khng (ang-i) A2, thi gian ch bn (TG-ch) A3: di 10 pht (0-10), t 10 n 30 pht (10-30) hay trn 30 pht (>30). p dng cc cng thc tnh entropy v gain, xc nh thuc tnh tt nht hi k tip nhm to ra mt cy quyt nh n gin nht theo gii thut ID3. Trnh by cc tnh ton entropy v gain mi bc.

V Hunh Trm Trn Ngn Bnh

185

Gio Trnh Tr Tu Nhn To

Chng IX ............................................................................................................................153 HC MY............................................................................................................................153 (MACHINE LEARNING)....................................................................................................153 I. GII THIU: ...........................................................................................................153 I.1. nh ngha hc..............................................................................................154 I.2. Cc tip cn hc: .............................................................................................155 II. TIP CN K HIU: gII THUT QUY NP CY QUYT NH ID3 .........155 II.1. Gii thiu.........................................................................................................155 II.2. Gii thut ID3 xy dng cy quyt nh t trnxung..................................157 II.3. Thuc tnh no l thuc tnh dng phn loi tt nht? ...............................160 II.4. Tm kim khng gian gi thuyt trong ID3.....................................................162 II.5. nh gi hiu sut ca cy quyt nh:...........................................................163 II.6. Chuyn cy v cc lut....................................................................................163 II.7. Khi no nn s dng ID3 ................................................................................164 III. TIP CN KT NI: MNG NEURON...........................................................164 III.1. Gii thiu.........................................................................................................164 III.2. C bn v mng kt ni:..................................................................................165 III.3. Hc perceptron ................................................................................................167 III.4. Hc Lan truyn ngc:....................................................................................173 III.5. Nhn xt chung v mng neuron.....................................................................176 IV. TIP CN X HI V NI TRI: GII THUT DI TRUYN (GENETIC ALGORITHM - GA)........................................................................................................177 IV.1. Gii thiu.........................................................................................................177 IV.2. Gii thut .........................................................................................................177 IV.3. V d 1: Bi ton tha CNF.............................................................................179 IV.4. V d 2: Bi ton ngi i bn hng TSP .......................................................180 IV.5. nh gi gii thut ..........................................................................................182 TNG KT CHNG IX ..............................................................................................184 BI TP CHNG IX ........................................................................................184

186

V Hunh Trm Trn Ngn Bnh