Grid Resource Management

Bo co Seminar
Pham The Anh Pham Thanh Toan Trinh Quoc Huy 6/1/2009

Grid Resource Management June 1, 2009

Contents
1 Gii thiu............................................................................................................................3 2 M hnh phn cp (Hierachical Model)..............................................................................4 3 M hnh Abstract Owner.....................................................................................................7 3.1 Cu trc tng qut ca m hnh AO.............................................................................8 3.2 Ti nguyn trong Grid................................................................................................11 3.2.1 Instrument...........................................................................................................11 3.2.2 Channel...............................................................................................................11 3.2.3 Complex..............................................................................................................12 3.3 m phn vi mt AO...............................................................................................12 3.4 Job Shops...................................................................................................................13 3.5 Tm tt.......................................................................................................................14 4 M hnh Computational Market/Economy........................................................................14 4.1 Grid Resource Broker (Mi gii ti nguyn mng li - GRB)................................15 4.2 Grid Middleware........................................................................................................16 4.3 Domain resource manager..........................................................................................17 4.4 Mt s nh gi:.........................................................................................................17 Ti liu tham kho...............................................................................................................18

Grid Resource Management June 1, 2009

1 Gii thiu
Resource management l tc v quan trng trong grid computing. Resource management iu khin cc kh nng cung cp resource hay service cho cc thc th khc trong grid nh user, ng dng hoc cc service khc. Trong grid computing, resource c th phn chia lm 2 loi khc nhau: Physical resource: computer, disk, database, networks, cc cng c khoa hc,.. Hay Logical resource: thc thi cc ng dng hay cc workflow phc tp. Mi mt ng dng c th s dng mt hoc nhiu loi resource khc nhau, mi resource owner li c yu cu khc nhau v vic s dng resource. T vic qun l resource trong grid phi tho mn cc yu cu sau: Phi thng nht c gia ngi cung cp resource vi ngi s dng resource. Phi m bo c cht lng ca dch v trong qu trnh thc thi job. C kh nng x l cc ng cnh khc nhau trong qu trnh s dng resource: task submission, workload management, on-demand access, co-scheduling, Khi grid cng pht trin th vic qun l resource trn li cng tr nn phc tp v cc l do sau: Tnh phn phi ca resource: mi mt resource trn grid khng tp trung mt ni m c th c phn phi u khp, hoc rt xa so vi ngi s dng. Tnh khng ng nht: mi loi resource c cc tnh cht ring, khng loi resource no c th ging nhau hon ton. Cc resource c qun l bi cc c nhn hoc t chc khc nhau. Mi mt owner ny li c cc chnh sch ring bit trong vic qun l resource ca mnh v s dng cc m hnh khc nhau cho vic qun l resource. Bi bo ny gii thiu 3 m hnh qun l ti nguyn trong grid: M hnh phn cp (Hierachical Model)

Grid Resource Management June 1, 2009 M hnh s hu tru tng (Abstract Owner Model) M hnh tnh ton kinh t/th trng (Computational Market/Economy Model)

2 M hnh phn cp (Hierachical Model)

L mt m hnh qun l resource c a ra ti hi ngh ln th 2 v Grid Computing. c s dng trong cc phn mm tnh ton li nh Globus, Legion, Ninf, Netsolve Trong m hnh Hierarchical, cc thnh phn c chia lm 2 loi l passive v active components. Passive components bao gm: Resource: o Tt c mi th c th c s dng trong mt khong thi gian. o C th phc hi c hoc khng th phc hi sau khi s dng o Mi mt resource u c ngi owner. Ngi owner ny c th tnh ph trn vic s dng resource. o C th c t tn v c th c cc thng s miu t cho resource. o V d nh: Computer, SMP, network bandwidth, cc device c bit Task: o Bt c thnh phn no tiu th resource. o Cc chng trnh tnh ton, file staging, hoc communication. Job: o L cc thc th c nh ngha qui trn task nh sau: Mi job c th c cu thnh t nhiu subjob, mi subjob l mt task hoc c cu thnh t nhiu subjob khc. o Cng vic n gin nht l ch gm 1 task. Schedule: o L s nh x gia task vi resource theo thi gian.

Grid Resource Management June 1, 2009 Active components bao gm: Scheduler: o Tnh ton ra cc schedule cho cc job, sao cho tho mn cc rng buc c nh trc trong thi gian runtime. o Scheduler s nh thi tt c cc task ca mt cng vic cng mt lc ch khng tch bit cc task ca cng mt job ra nh thi ring l. o Cc scheduler nm ngoi control domain khng th cung cp resource, cc scheduler ny thng c gi l super scheduler hay l metascheduler. Information Services: o C th coi nh l mt database miu t cc item trong h thng qun l resource: resource, jobs, schedulers, agents, o C th hin thc n gin bng mt LDAP (Lightweight Directory Access Protocol) server. Domain Control Agent (local resource manager, box) o L mt tp hp cc resource c qun l bi mt agent. o C th commit resource vo trong grid. Deployment agent: o Hin thc cc schedule bng cch thng tho vi cc control agent t c resource cho vic thc thi cc job. Users: o Cc i tng submit job vo trong grid thc thi. Admission Control Agent: o iu tit vic vic nhn job thc thi trong li. o Admission Control Agent c th t chi nhn mt job hoc tin hnh tr hon vic thc thi job. Monitors: o Theo di qu trnh thc thi ca job bng cc thng tin ly c t cc Domain Control Agent ang thc thi job.

Grid Resource Management June 1, 2009 o Tu theo trng thi ca cng vic m monitors c th thng bo n Job Control Agent v Scheduler tin hnh vic nh thi li. Job Control Agent: o Chu trch nhim iu phi mt job qua h thng grid. iu phi vic giao tip gia cc thnh phn trong h thng qun l resource. V d nh gia Scheduler v gia Monitors. Vic phn chia cc thnh phn trong m hnh Hierarchical ch mang tnh logic.V d nh, chng ta phn chia scheduler, deployment, v monitor nhng trong thc t 3 thnh phn ny c th kt hp li thnh mt thc th duy nht. Hoc l information service c th c tch hp vo trong Domain Control Agent hoc tch ring bit. Qu trnh tng tc gia cc thnh phn trong m hnh hierarchical c miu t nh hnh:

Qu trnh thc thi ca mt job s thng qua cc bc sau: 1. u tin, mt user s submit mt job vo grid thng qua mt Job Control Agent. 2. c th quyt nh xem job ny s c thc thi hay tr hon hoc b t chi, Job Control Agent s gi mt Admission Control Agent. Admission Control Agent xem xt tnh trng ca cc resource trong h thng (c th thng qua cc Information Service) ri a ra quyt nh.

Grid Resource Management June 1, 2009 3. Nu Admission Control chp nhn thc thi job, s a job vo mt Scheduler tin hnh discovery resource. 4. Scheduler s xem xt cc Domain Agent bit c tnh trng resource hin thi. 5. Scheduler tin hnh tnh ton ra cc schedule v a cc schedule ny cho cc deployment thc thi. 6. Deployment tin hnh thng tho vi cc Domain Control Agent to ra cc reservation cho cc resource cn thc thi job. 7. Sau qu trnh negotiate, cc reservation s c a li cho Job Control Agent. 8. mt thi im nht nh, Job Control Agent s lm vic vi nhiu deployment khc nhau, cc deployment ny s lm vic vi cc Domain Control Agent tng ng v bt u thc thi job. 9. Trong qu trnh thc thi, Monitor s thng xuyn theo di qu trnh thc thi job. Monitor s yu cu nh thi li nu nh vic thc thi qu chm so vi yu cu c t ra.

3 M hnh Abstract Owner

M hnh Abstract Owner c a ra nh mt m hnh l tng nhm tru tng ha cc ti nguyn trong Grid. c th hiu tng ca vic xut ra m hnh AO, ta xem xt trng hp l mt ngi cn thc hin mt cuc gi in thoi ng di t Vit Nam i Singapore. cuc
7

Grid Resource Management June 1, 2009 gi c thc hin th cn phi c nhng ti nguyn: nh cung cp ng truyn, cung cp in, cc tng i chuyn mch v.v. Vy th ai l ngi s hu nhng ti nguyn ? Thc t ngi gi in thoi khng cn phi quan tm n vic ny. iu h cn bit l c mt i tng no cung cp dch v gi in thoi, v h s tha thun vi i tng thc hin cuc gi. Tr li vi m hnh AO, trong m hnh ny, mi mt ti nguyn trong grid s c i din bi mt hoc nhiu s hu tru tng abstract owners. Ti nguyn ny v ln c th ch l mt b x l hay l c mt h thng grid. Vi nhng ti nguyn phc hp, th s hu tru tng ny chnh xc s lm cng vic ca mt mi gii broker trong grid m phn ti nguyn vi nhng s hu tru tng khc; ngi s dng ti nguyn (grid user) s ch cn bit mt knh giao tip duy nht l vi s hu tru tng cung cp ti nguyn m h cn m khng cn quan tm ti nguyn c t chc nh th no. Cc phn tip theo ca mc ny s miu t chi tit hn v s hu tru tng, ti nguyn, phng thc m ngi s dng ti nguyn s m phn vi s hu tru tng s dng ti nguyn, cch thc ngi s dng s dng ti nguyn. K t phn ny tr i, ngi s dng ti nguyn s c gi l client c th l mt chng trnh my tnh s dng ti nguyn hoc cng c th l con ngi. s hu tru tng c gi l abstract owner AO, ng theo nguyn bn ca bi bo gc trnh nhm ln. 3.1 Cu trc tng qut ca m hnh AO M hnh ca mt AO tng t nh c cu ca mt ca hng thc n nhanh: c mt u vo nhn t hng ca khch hng v mt u ra tr kt qu. yu cu s dng mt ti nguyn no , client s dng Order Window (ca s t hng) thc hin vic m phn vi AO. Qu trnh m phn nhm xc nh khong thi gian m client c th s dng c ti nguyn, hay chi ph s dng ti nguyn Sau khi qu trnh m phn kt thc, client c th thc hin vic t hng ti nguyn hoc hy b qu trnh m phn do iu kin m AO a ra khng tha mn vi yu cu ca client (VD: chi ph cao, khng tha mn thi im s dng ti nguyn). Nu vic t hng c thc hin th sau client c th nhn ti nguyn t AO thng qua Pickup Window (ca s giao hng). Vic nhn ti nguyn c thc hin bng nhng giao thc c tha thun trc gia client v AO chng hn nh client c th n v nhn ti nguyn vo mt thi im nh trc, hoc AO phi pht thng ip thng bo cho client bit khi m ti nguyn sn sng. Cc giao thc ny s c xc nh trong qu trnh m phn.

Grid Resource Management June 1, 2009

Hnh 3.1 M hnh Abstrac Owner

Qu trnh m phn gia client v AO c miu t qua lu hnh 3.2. Nh vy, cc ca s Order v Pickup l cc interface ca AO client c th access v thc hin cc bc t hng cng nh nhn v ti nguyn. Cc ca s ny phi h tr nhng giao thc chun client c th trao i thng tin. Cc lung thng tin trao i gia client v AO thng thng khng i hi lng d liu ln, do vic ti u ha hiu nng truyn nhn khng tht s cn thit. y, ta c th s dng nhng phng thc iu khin t xa nh CORBA hin thc. Khi nim ti nguyn trong m hnh ny s c xem nh l mt i tng, vi cc thuc tnh, cc phng thc v gi tr nh danh i tng ti nguyn. Thuc tnh ca ti nguyn cho php client c th ty bin ti nguyn cn phng thc ca ti nguyn c s dng khi to cng nh iu khin ti nguyn . Thng thng, cc thuc tnh ca ti nguyn s c gn trong qu trnh t hng v s c truy vn (ch c) sau khi ti nguyn c chuyn giao.

Hnh 3.2- Qu trnh m phn v nhn ti nguyn

Grid Resource Management June 1, 2009 Nhn t bn ngoi, mt AO s c cu trc chnh l 2 interface Order Window v Pickup Window nh cp trn, d AO l trc tip iu khin ti nguyn hay l mt broker s dng nhng ngun ti nguyn khc. Tuy nhin, vi gc nhn t bn trong, cu trc ca mt AO s c phn khc bit ty theo AO trc tip qun l ti nguyn hay l broker. Nu AO trc tip qun l ti nguyn th cu trc kh n gin, n cn c them mt b qun l ti nguyn (Resource Manager) c chc nng m phn vi client, iu phi ti nguyn v giao ti nguyn cho client s dng (Hnh 3.3).

Hnh 3.3 AO trc tip s hu ti nguyn

Trong trng hp AO l mt broker, ngha l AO cn s dng ti nguyn ca cc AO khc cung cp th ngoi thnh phn Manager, AO cn c them 2 thnh phn na: Sale Representative: Giao tip vi cc Order Window ca cc AO khc m phn xin s dng ti nguyn Delivery Representative: Giao tip vi cc Pickup Window ca cc AO khc nhn ti nguyn (Hnh 3.4)

10

Grid Resource Management June 1, 2009

Hnh 3.4 AO l mt broker

3.2 Ti nguyn trong Grid Trong m hnh AO, cc ngun ti nguyn c chia lm 3 loi: Instrument, Channel v Complex. 3.2.1 Instrument Ti nguyn instrument c nh ngha l mt ti nguyn hin hu ti mt v tr hoc trong mt khong thi gian m ti nguyn to ra, tiu th hoc chuyn ti d liu, thng tin. Ti nguyn instrument c chia ra lm 3 loi: Compute Instrument: bao gm cc b x l n hoc a x l vi b nh v cc c ch lu tr tng ng. Archival Instrument: bao gm cc i tng lu tr thng tin m n v nh nht l file. Personal Instrument: bao gm cc interface m cn c s tham gia trc tip ca con ngi 3.2.2 Channel Ti nguyn dng Channel l cc ti nguyn nhm phc v vic truyn ti d liu hoc thng tin gia hai hay nhiu instrument gia nhng a im khc nhau hay cng mt a im nhng thi im khc nhau. Mt Channel kt ni ti mt Instrument thng qua khi nim cng (port) trn instrument.

11

Grid Resource Management June 1, 2009 3.2.3 Complex Ti nguyn dng Complex n thun l tp hp nhng ti nguyn Instrument c kt ni vi nhau bi cc Channel. 3.3 m phn vi mt AO Khi thc hin m phn vi AO, client trc tin s to ra mt i tng ti nguyn mu vi cu trc thch hp v gn cho cc thuc tnh ca i tng ny Mt gi tr xc nh HOC Gi tr cho bit thuc tnh khng cn thit (dont care value) HOC Ga tr i din bi tn bin. Cc gi tr c gn bi tn bin s c tham chiu bi bng gi tr. Ngoi ra, client cng c th t thm cc rng buc c bn ln cc bin.

Gi tr xc nh (constrant) c client gn vo thuc tnh gi n AO khi client yu cu ti nguyn vi nhng thuc tnh c nh. Gi tr dont care ch nhng thuc tnh m i vi client l khng quan trng. Cn nhng gi tr bin l nhng gi tr m phn. AO sau khi nhn c i tng ti nguyn mu s tr v li cho client vi nhng thuc tnh bin c gn gi tr, l nhng gi tr m AO c th p ng cho client. AO c th gn cho nhng thuc tnh ny gi tr c th m phn (negotiable), khi AO s cung cp mt tp cc gi tr m AO c th p ng cho thuc tnh . Cn c vo client s chn. Khi thuc tnh c chn gi tr th n s tr thnh hng s. Mt cch tng qut th mt yu cu t client gi n AO s dng ti nguyn s bao gm: Cc thuc tnh ca i tng ti nguyn mu Cc rng buc trn cc thuc tnh Kiu m phn (Negotiation style), bao gm: o Immediate: AO cung cp ti nguyn cho client ngay lp tc o Pending: AO cn m phn thm ly thng tin cho ti nguyn o Confirmation: AO v client m phn xong v sn sang cung cp ti nguyn theo kt qu phin m phn o Cancel: Hy b kt qu ca phin m phn.

12

Grid Resource Management June 1, 2009 Phng thc nhn ti nguyn (Pickup Approach), nh AO s thng bo cho client khi ti nguyn sn sang, hay client s kt ni n AO nhn ti nguyn vo mt thi im nh trc. Chng thc: dung chng thc trong trng hp yu cu Chi ph chp nhn s dng ti nguyn (Bid): chi ph cao nht m client c th chp nhn s dng ti nguyn. nh danh ca phin m phn (Negotiation ID): s dng nh mt session ID xc nh khi ti nguyn sn sang. Khi client nhn ti nguyn t AO, client s phi cung cp nh danh ca phin m phn nhn ti nguyn.

3.4 Job Shops Trong m hnh AO va c cp trn, mi AO thng l mt i din ca mt ti nguyn ring r, do so vi cc m hnh khc, AO cha c c chc nng phn chia cng vic vi nhng tc v ln. Khi nim Job Shop c a ra gii quyt vn ny.

Hnh 3.5 Job Shob

Job Shop nhn vo mt tc v phc hp v nhim v ca n l phn chia tc v ny thnh nhng tc v con nh hn v ln lch thc hin nhng tc v ny (Job Scheduling). Job Shop c 2 thnh phn chnh: Estimator, c nhim v: o Nhn tc v t client, xc nh khong thi gian sm nht tc v c th c hon thnh. o Giao tip vi cc AO yu cu ti nguyn

13

Grid Resource Management June 1, 2009 o Ln lch thc hin cc tc v nh khi m ti nguyn sn sang t cc AO. Executor, c nhim v: o Nhn ti nguyn t cc AO o Chun b mi trng cn thit thc hin cc tc v nh o Khi ng cc tc v nh khi ti nguyn sn sang v thu thp kt qu o Dn dp v gii phng ti nguyn khi tc v kt thc o Tr kt qu cui cng v cho client. 3.5 Tm tt M hnh ngh AO c trnh by trong bi bo ch mc l mt m hnh xut, n cn rt nhiu khim khuyt cn c khc phc trc khi c th tr thnh mt m hnh Grid trn thc t. Mt trong nhng khim khuyt ny l m hnh cha a ra c cch thc mt client khi cn s dng ti nguyn c th tm kim ra ngun ti nguyn hin ang c qun l bi AO no.

4 M hnh Computational Market/Economy

Ti nguyn trn mi trng tnh ton li nm trn nhiu vng a l khc nhau, mi thnh phn c b phn qun l ti nguyn ring, cc quy nh ring ca n, v c th ly gi khc nhau cho cc nhu cu khc nhau t th trng tnh ton. Thc t i hi phi c mt m hnh hng th trng nhng t chc s hu ti nguyn c th ti a ha li nhun, ng thi gip ngi dung gim thiu chi ph m vn t c kt qu nh mong i. M hnh ny phi c cc cng c v dch v gip ngi dung nh ngha ra cht lng dch v h cn; gip pha qun l ti nguyn c kh nng thch ng cao vi nhng thay i theo tng thi im. M hnh dng ny mang c hai m hnh, phn cp v AO.

14

Grid Resource Management June 1, 2009

Mt s c th thc hin theo m hnh hng th trng Nhng thnh phn chnh: User Applications (sequential, parametric, parallel, or collaborative applications) Grid Resource Broker (mi gii ti nguyn li - a.k.a., Super/Global/Meta Scheduler) Grid Middleware Domain Resource Manager (Local Scheduler or Queuing system) 4.1 Grid Resource Broker (Mi gii ti nguyn mng li - GRB) ng vai nh thnh phn trung gian gia ngi dng v cc ti nguyn li, da vo cc dch v middleware. Chu trch nhim: Pht hin ti nguyn, chn ti nguyn, kt ni cc phn mm ng dng, d liu v cc ti nguyn phn cng Khi ng cc qu trnh tnh ton, ph hp theo s thay i ti nguyn trn mng li, v trnh by mng lui vi ngi dnh nh mt th thng nht.

Cc thnh phn khi ca mt resource broker gm:

15

Grid Resource Management June 1, 2009 Job Control Agent (JCA): l thnh phn trung tm thng nht, chu trch nhim : o lun chuyn job i qua cc thnh phn ca h thng, o trng coi qu trnh lp lch, l n v thc hin to ra cc jobs, o Theo di tnh trng ca mt job o Giao tip vi khch hang/ngi dung , scheduler adviser, dispatcher Schedule Advisor (Scheduler): c trch nhim pht hin ti nguyn (da vo grid explorer), chn ti nguyn v phn task. Chc nng chnh ca n l chn cc ti nguyn khi vi yu cu ngi dng nh thoi mn vic cho ra kt qu ng thi hn v ti thiu ha chi ph tnh ton. Grid Explorer: c trch nhim pht hin ra cc ti nguyn bng cch giao tip vi grid-information server (my ch cha thng tin dch v) v xc nh danh sch cc my tnh c xc thc, v theo di thng tin tnh trng ca ti nguyn. Trade manager( TM): lm vic theo hng ca thut ton chn ti nguyn xc nh chi ph s dng ti nguyn. Giao tip vi trade server v thng lng s dng cc dch v vi gi thp. N c th tm gi dch v t my ch thng tin dch v nu ch s hu ti nguyn c ng. Deployment Agent: trch nhim cho vic kch hot cc task trn nhng thnh phn ti nguyn c chn nh ch th ca Scheduler. N nh thi cp nht tnh trng thc hin task ti JCA

4.2 Grid Middleware Cung cp cc dch v gip cho qu trnh ni gia ngi dng v cc ti nguyn t xa, thng qua mi gii ti nguyn (GRB) hoc cc ng dng c th chy trn grid. N cung cp cc dch v li nh qun l tin trnh t xa, phn b ti nguyn, truy cpd liu, thng tin, dch v an ton, chng thc, v cht lng dch v nh t trc dch v m bo s kh dng ca ti nguyn, v cho gi ti thiu chi ph.

16

Grid Resource Management June 1, 2009 o Trade server (TS) l mt tc nhn qun l ti nguyn, m phn vi ngi dng v bn cc truy xut ti ti nguyn, hng ti vic vic x dng ti a ti nguyn v li nhun cho cc t chc s hu (kim nhiu tin nht c th) N thm d cc thut ton tnh gi nh ngha bi ngi dng trong qu trnh m phn. V hng h thng k ton ghi li s s dng ti nguyn o Pricing algorithms/Methods: quyt nh gi cho cc dch v. N c th theo mt s quy nh khc nhau ti a ha li nhun. Cho php nhi dng thay i gi dch v theo thi gian o Accounting system: c trch nhim ghi nhn li vic s dng ti nguyn v tnh chi ph cho ngi dng da vo b phn qun l ti nguyn gia ResourceBroker v trade derver. 4.3 Domain resource manager Cc b qunl ti nguyn ni b c trch nhim qun l v lp lch tnh ton trn tt c cc ti nguyn ni b nh my trm hoc cluster. Chng c th c trch nhim cung cp truy cp ti thit b lu tr, c s d liu, v cc thit b khoa hc, nh knh vin vng 4.4 Mt s nh gi: Cc dch v cung cp bi TradeServer c th truy xut t hoc cung cp bi cc Information server. Trong trng hp ny, mt Trade Manager hoc Broker c th truy xut trc tip vo Information server, Trade Manager c th dng gi c nim yt hoc mi nhng biu gi cnh tranh hoc u gi chn ra cc ti nguyn tha mn nhu cu ngi dng. T cc tho lun trn, c th thy c nhiu cch khc nhau xc nh hoc bit chi ph truy xut. S v d cung cp l mt trong nhng la chn c th thc hin cho m hnh hng th trng, v n c th khc nhau da vo giao thc trao i thng tin nh th gii thc. Nhng h thng li khc nhau c th theo nhiu nh hng khc nhau hot ng, v n s rt hu ch nu nhng h thng ny lin thng . S lin thng ny c th c pht trin t cc cng ng li hoc cc t chc chun ha nh GF v eGrid.

17

Grid Resource Management June 1, 2009

Ti liu tham kho

[1] Rajkumar Buyya, Steve Chapin*, and David DiNucc Architectural Models for Resource Management in the Grid - School of Computer Science and Software Engineering, Monash University. ---HT---

18