3.

ANALYSIS TECHNIQUES
Trong phn ny chng ta bt u vi nhng php o thc t thc y cc loi cu hi chng ta mun tr li v gii thch phng php phn tch ca chng ta hng ti vc gii thch nhng cu hi trn. 3.1.Tng quan a cng vic ca chng ta vo gc nhn, Hnh 2 cho thy hm phn phi (phn b) tch ly (CDF) ca bn s liu cht lng cho tp d liu LvodA (datasheet LvodA ). Nh d on, hu ht cc phin xem tri nghim cht lng rt tt, th d, c BufRatio rt thp, JoinTime thp, v RendQual tng i cao. Tuy nhin s tm nhn b cc vn cht lng l khng nh. c bit, 7% trong s nhng tm nhn tri qua BufRatio ln hn 10%, 5% trong s nhng views c JoinTime ln hn 10s, v 37% trong s nhng views c RendQual thp hn 90%. Cui cng, ch c mt phn tng i mh cc views nhn c bit rate cao nht. a ra mt s lng khng ng k nhng views tri nghim nhng vn cht lng, iu l rt quan trng cho cc nh cung cp ni dung hiu rng nu nng cao cht lng ca nhng phin ny c th c kh nng tng the user engagement. hiu lm th no m cht lng c th nh hng n the engagement, chng ta xem xt mt i tng video t LiveA v LvodA. i vi video ny, chng ta bin nhng sessions khc nhau da trn nhng gi tr ca nhng tiu chun cht lng v tnh ton thi gian chi trung bnh cho mi bin. Hnh 3 v 4 th hin cch bn tiu chun cht lng tng tc vi thi gian chi. Nhn vo nhng xu hng trc quan khng nh vn cht lng . ng thi, nhng trc quan ha ban u ny chm ngi cho mt s cu hi:

Lm th no chng ta xc nh s liu (metrics) no quan trng nht ? Nhng s liu cht lng ny (qyality metrics) l c lp hay chng l nhng biu hin ca cng mt hin tng c bn? Ni cch khc, l quan st mi quan h gia the engagement v quality metrics M do M hoc do mt mi quan h tim n gia M v mt metric M quan trng hn khc. Lm th no chng ta c th nh lng c mt quality metric quan trng nh th no? Chng ta c th gii thch c nhng hnh vi dng nh phn trc gic khng? V d, RendQual l ph nhn s tng quan vi video LiveA (Hnh 4(d)), trong khi AvgBitrate cho thy mt xu hng khng n iu bt ng i vi LvodA (Hnh 3(c)).

gii quyt hai cu hi u tin, chng ta s dng khi nim ni ting ca s tng quan v thng tin ly c t khai thc d liu cc ti liu m chng ta m t tip theo. nh gi nh hng, chng ta cng s dng nhng m hnh hi quy tuyn tnh da trn nhng metric(s) quan trng nht. Cui cng chng ta s dng nhng hiu bit lnh vc c th v cc th nghim trong cc thit lp iu khin gii thch nhng quan st bt thng.

3.2.Tng quan Cch tip cn t nhin nh lng s tng tc gia mt cp bin s l cc mi tng quan (correlation). y chng ta quan tm n vic nh lng ln v hng ca mi quan h gia the engagement metric v the quality metrics. trnh a ra cc gi nh v bn cht ca cc mi quan h ca cc bin, chng ta la chn tng quan Kendall, thay v tng quan Pearson. Tng quan Kendall l mt th hng tng quan m khng thc hin bt k gi nh v s phn b underflying, noise, hoc tnh cht ca cc mi quan h. (Tng quan Pearson gi nh rng the noise trong d liu l Gaussian v rng mi quan h l xp x tuyn tnh). Vi d liu th mt vector (x,y) m mi x l the measured quality metric v y the engagement metric (thi gian chi hoc s views). Chng ta bin n da trn gi tr ca the quality metric. Chng ta chn bin sizes ph hp vi mi quality metric quan tm: Vi JoinTime, chng ta s dng 0.5s intervals, vi Bufratio v RendQual chng ta s dng 1% bin, vi RateBuf chng ta s dng 0:01/min sized bins, v vi AvgBirate chng ta s dng 20 kbps-sized bins. Vi mi bins, chng ta tnh ton gi tr trung bnh da trn kinh nghim ca the engagement metric qua the sessions/viewers m ri vo trong bin. Chng ta tnh ton tng quan Kendall gia the mean-per-bin vector v gi tr ca nhng ch s bin. Chng ta s dng metric tng quan binned ny vi hai l do. Th nht, chng ta quan st thy rng h s tng quan b sai lch bi mt lng ln users m c cht lng cao nhng thi gian chi rt thp, c th bi v li ch (s ch tm) user thp. Mc tiu chnh ca chng ta, trong bi bo ny, khng phi l nghin cu quan tm (li ch) user trong nhng ni dung c th. Thay vo , chng ta mun hiu v nh hng va cht lng n user engagement nh th no. t c mc ch ny, chng ta nhn vo gi tr trung bnh cho mi bin v tnh ton tng quan trn the binned data. L do th hai l quy m. Tnh ton xp hng tng quan l tnh ton tn km quy m phn tch chng ta nhm n. Tng quan binned gi li nhng thuc tnh cht lng m chng ta mun lm ni bt vi chi ph thp hn 3.3.Information Gain Tng quan c ch cho vic nh lng tng tc gia cc bin khi mi quan h l gn nh khng thay i (roughly monotone) (hoc tng hoc gim). Nh hnh 3(c) cho thy, ddiieeuf ny khng phi lc no cng l trng hp. Hn na chng ta mun tin xa hn vic phn tch metric ring r. Th nht, chng ta mun hiu nu mt cp (hoc mt tp hp) quality metrics c b sung hoc nu chng nm bt c cng tc dng (hiu ng). Mt v d, xem xt RendQual trong hnh 3; RendQual c th phn nh mt vn network hay mt vn CPU pha client. Bi v BufRatio cng tng quan vi PlayTime, chng ta nghi ng rng RendQual phn nh cng tc dng. Xc nh v pht hin ra cc mi quan h n ny, tuy nhin, l t nht (thiu hp dn). Th hai, cc nh cung cp dch v ni dung c th mun bit the top k metrics m chng cn ti u ha ci thin user engagement. Phn tch correlation-based khng th tr li nhng cu hi nh vy.

 gii quyt nhng thch th trn, chng ta tng cng phn tc tng quan bng cch s dng khi nim ca information gain, da trn khi nim ca entropy. Entropy ca bin ngu nhin Y l

Trong P[Y = yi] l xc xut m Y = yi. Entropy c iu kin ca Y khi xc nh mt bin ngu nhin X khc c nh ngha l H(Y|X) = j P[X = Xj] H[Y|X = xj] v khi y the informatin gain l H(Y) H(Y|X), v the relative information gain l

Theo trc gic, metric quanties ny bng cch no s hiu bit v X ca chng ta lm gim s khng chc chn trong Y. C th, chng ta mun nh lng iu m mt quality metric thng tin cho chng ta v the engagement ; v d, what does knowing the AvgBitrate or BufRatio tell us about the play time distribution? Nh vi tng quan, chng ta bin data vo nhng bns ring bit vi cng c im bin (thng s k thut bin). i vi thi gian chi (play time), chng ta chn nhng kch thc bin (bin sizes) khc nhau ph thuc vo khong thi gian ca ni dung. T binned data ny, chng ta tnh ton H(Y|X1,...,XN), trong Y l the discretized play time v X1,...,XN l quality metrics. T c tnh ny, chng ta tnh ton the relative information gain. Lu rng hai lp k thut phn tch ny l b sung. Tng quan cung cp mt bn tm lc first-order ca mi quan h khng thay i (n iu - monotone) gia engagement v quality. The information gain c th chng thc cc mi tng quan hoc tng cng n khi mi quan h l khng n iu (monotone). Hn na, n cung cp s hiu bit ca s tng tc su sc hn gia the quality metrics bng cch m rng cc trng hp a bin. 3.4.Hi quy Xp hng tng quan v informatin gain ch yu l phn tch cht lng. N cng hu ch hiu tc ng nh lng ca mt quality metric trn user engagement. C th chng ta mun tr li cu hi dng: S ci tin mong i trong the engagement l g nu chng ta ti u ha mt quality metric c th bi mt s lng (amount) nht nh. i vi phn tch nh lng, chng ta nh vo hi quy. Tuy nhin, nh trc quan cho thy, mi quan h gia the quality metrics v the engagement l khng phi lun lun r rng v mt s the metrics c tnh ph thuc ni ti (bn cht). V vy, trc tip p dng cc k thut hi quy vi cc thng s phi tuyn tnh phc tp c th dn n cc m hnh thiu gii thch ngha v mt vt l. Trong khi mc tiu cui cng ca chng ta l rt ra nh hng nh lng tng i ca nhng metrics khc nhau, lm nh vy l rt nghim ngt bn ngoi phm vi bi bo ny. Nh mt gii php thay th n gin, chng ta s dng hi quy tuyn tnh da trn ng cong ph hp nh lng nh hng ca phm vi c th ca quality metric quan trng nht. Tuy

nhin, chng ta lm vy ch sau khi trc quan xc nhn rng mi quan h l khong tuyn tnh trn phm vi quan tm. Ddiieeuf ny cho php chng ta s dng n gin cc m hnh tuyn tnh ph hp m cng d dng gii thch.

4. VIEW LEVEL ENGAGEMENT

The engagementmetric ca s quan tm mc view l PlayTime. Chng ta bt u vi ni dung VoD di, sau tin hnh n ni dung live v ngn. Trong mi trng hp, chng ta bt u vi s tng quan c bn da trn vic phn tch v tng n ln vi information gain da trn vic phn tch. Lu rng chng ta tnh ton s tng quan c binned (the binned correlation) v cc h s information gain trn mt per-video-object c s. Sau chng ta nhn vo s phn b cu cc h s trn tt c cc i tng video. Sau khi xc nh nhng s liu quan trng nht, chng ta nh lng tc ng ca vic ci thin cht lng bng cch s dng mt m hnh hi quy tuyn tnh trn mt phm vi c th ca s liu cht lng. Tm li, chng ta thy rng BufRatio lun c tc ng cao nht n user engagement trong s tt c cc s liu cht lng. V d, 90 pht live event, mc tng BufRatio 1% c th lm gim PlayTime hn 3 pht. Tht th v, tc ng tng i ca nhng s liu khc ph thuc v loi ni dung. i vi live video, RateBuf l hi tiu cc hn tng quan vi PlayTime so vi VoD di (long VoD); bi v cc b m my nghe nhc nh c t thi gian phc hi khi bng thng thay i bt thng. Phn tch ca chng ta cng nhng bitrates cao hn c nhiu kh nng ci thin user engagement cho ni dung live. Ngc li vi nhng videos VoD live v di, nhng videos RendQual ngn by t tng quan tng t BufRatio. Chng ta cng thy rng cc s liu khc nhau l khng c lp. Cui cng, chng ta gii thch mt s cc quan st bt thng t mc 3 su hn. 4.1 Long VoD Content (Ni dung VoD di)

Hnh 5 cho thy s phn b ca cc h s tng quan cho cc s liu cht lng i vi tp d liu LvodA. Chng ta tnh n c gi tr tuyt i v gi tr c du o lng mc v tnh cht (v d, tng hoc gim) ca s tng quan. Chng ta tng hp cc gi tr trung bnh cho c hai b d liu trong bng 2. Kt qu l nht qun trn c hai tp d liu i vi nhng s liu cht lng ph bin BufRatio, JoinTime, v RendQual. Nh li rng hai b d liu tng ng vi hai nh cung cp ni dung khc nhau; nhng kt qu ny xc nhn rng cc quan st ca chng ta khng phi l duy nht cho b d liu LvodA. Kt qu cho thy BufRatio c tng quan PlayTime mnh nht.Bng trc gic, chng ta mong i mt BufRatio cao hn lm gim PlayTime (v d, mt tng quan tiu cc) v mt RdQual cao hn gia tng PlayTime (v d, mt tng quan tch cc). Hnh 5(b) xc nhn trc gic ny v bn cht ca cc mi quan h. Chng ta nhn thy rng c t tc ng n khong thi gian chi. ng ngc nhin l, AvgBitrate c tng quan rt thp.

Tip theo, chng ta tin hnh kim tra nu phn tch the univariate information gain chng thc hoc b sung cc kt qu tng quan trong hnh 6. Tht th v, th t tng i gia RateBuf v BufRatio o ngc so vi hnh 5. L do (xem hnh 7) l hu ht khi xc xut l trong bin u tin (0-1% BufRatio) v entropy y l tng t nh phn phi tng th. Do , the information gain i vi BufRatio thp; RateBuf khng b vn ny (khng c trnh by) v c information gain cao hn. Chng ta cng thy rng AvgBitrate c information gain cao ngay c khi tng quan ca n l rt thp. Chng ta xem li quan st ny ti mc 4.1.1.

Cho n nay chng ta xem mi s liu cht lng mt cch c lp. Mt cu hi t nhin l: Vic kt hp hai s liu cht lng li c cung cp nhiu hiu bit su sc hn khng? V d, BufRatio v RendQual c th c c tng quan vi nhau. Trong trng hp ny bit rng c hai tng quan vi PlayTime khng thm thng tin mi. nh gi iu ny, chng ta cho thy s phn b ca hai bin relative information gain trong hnh 8. r rng hn, thay v hin th tt c cc cp kt hp, vi mi s liu chng ta tnh n hai bin kt hp vi relative information gain cao nht. i vi tt c cc s liu, s kt hp vi AvgBitrate cung cp hai bin information gain cao nht. Ngoi ra, mc d BufRatio, RateBuf, v RendQual c tng quan mnh m trong hnh 5(a), s kt hp ca chng khng thm nhiu thng tin mi bi v chng vn d tng quan.

4.1.1 Strange behavior in AvgBitrate (Hnh vi k l trong AvgBirate) Gia hnh 5 v 6, chng ta nhn thy rng AvgBirate l s liu vi tng quan km nht nhng ng th hai v information gain cao nht. Quan st ny lin quan n hnh 3 ca muc 3. Mi quan h gia PlayTime v AvgBirate l khng n iu; n cho thy mt cao im gia 800-1000 Kbps, thp hai bn ca vng ny, v tng nh mc cao nht. Do mi quan h khng n iu ny, s tng quan thp. Tuy nhin bit gi tr ca AvgBirate cho php chng ta d on PlayTime; c c information gain khng tm thng (non-trivial). By gi iu ny gii thch l do ti sao the information gain cao v mi tng quan thp, nhng khng cho chng ta bit ti sao PlayTime thp i vi di 1000-1600 Kbps. L do l cc gi tr ca birate trong phm vi ny tng ng vi nhng client phi chuyn i birate v b m gy ra bi cc iu kin mng ngho. Nhu vy, PlayTime thp y ch yu l do hu qu ca b m, m chng ta quan st l yu t quan trng nht. iu ny cng ch ra s cn thit phi la chn birate mnh v cc thut ton thch ng. 4.2 Live Content Hnh 9 cho thy s phn b ca cc h s tng quan cho tp d liu LiveA. Cc gi tr trung bnh cho hai tp d liu c tm tt trong bng 3. Chng ta nhn thy mt s khc bit quan trng i vi kt qu LvodA:P AvgBitrate l tng quan mnh hn i vi ni dung trc tip. Tng t nh s liu LvodA, BufRatio tng quan mnh, trong khi JoinTime l tng quan km.

i vi c ni dung VoD di v ni dung trc tip, BufRatio l mt s liu quan trng. Tht th v, i vi trc tip, chng ta thy rng RateBuf c mt tng quan tiu cc vi PlayTime mnh hn rt nhiu. iu ny cho thy the Live users l nhy cm hn vi mi s kin m so vi cc i tng the Long VoD. Kho st thm vn ny, chng ta thy rng thi gian m trung bnh l nh hn nhiu i vi long VoD (3 giy) so vi trc tip (live) (7s), v d, mi s kin m trong trng hp ni dung trc tip l t ph hn. Bi v kch thc b m trong long VoD l ln hn, h thng l trong tnh trng tt hn trong mt bin ng trong lin kt bng thng. Hn na h thng c th ch ng hn trong vic d on m v do ngn chn n bng cch chuyn sang my ch khc, hoc chuyn i bitrate. Do c nhng s kin m t hn v ngn hn i vi long VoD. Vi trc tip (live), ngc li, b m l ngn hn m bo rng lung l mi nht. Kt qu l, h thng t c kh nng ch ng d on qua cc bin ng, lm tng c s lng v thi gian ca cc s kin m. Hnh 10 tip tc khng nh rng AvgBitrate l mt s liu quan trng v JoinTime l t quan trng hn i vi ni dung trc tip. Hai bin kt qu (khng c trnh by cho ngn gn) m phng nhng tc dng tng t t hnh 8, ni m s kt hp vi AvgBitrate cung cp nhng the information gain tt nht.

4.2.1 Why is RendQual negatively correlated? (Ti sao RenQual tng quan tiu cc) Chng ta nhn thy mt hnh vi bt thng PlayTime vi RendQual i vi ni dung trc tip trong hnh 4(d). Cc kt qu trc t c b d liu LiveA v LiveB tip tc khng nh y khng phi l mt bt thng c trng cho cc video c hin th trc , nhng l mt hin tng ph bin trong ni dung trc tip. minh ha ti sao tn quan tiu cc ny pht sinh, chng ta tp trung vo mi quan h gia RenQual v PlayTime i vi mt video trc tip (live video) c th trong hnh 11. Tht ngc nhin chng ta thy mt phn ln nhng viewers vi cht lng dng hnh thp v thi gian chi cao. Hn na gi tr BufRatio i vi nhng ngi s dng ny cng rt thp. Ni cch khc, nhng ngi s dng ny khng c vn v mng, nhng thy mt s st gim trong RenQual, nhng vn tip tc xem video trong mt khong thi gian di bt chp t l khung hnh km.

Chng ta suy on rng mi tng quan tiu cc truy cp trc quan ny gia RendQual v PlayTime pht sinh mt s kt hp ca hai hiu ng. Hiu ng u tin lm vi hnh vi ngi dng. Khng ging nh nhng ngi xem long VoD (vis d, TV episodes), nhng ngi xem video trc tip cng c kh nng chy cc the video player ch nn (v d, nghe nhng li bnh lun th thao). Trong tnh hung nh vy trnh duyt c thu nh hoc the player c th nm trong mt tab trnh duyt n. Hiu ng th hai l mt ti u ha bi the player gim tiu th CPU khi video ang c chi ch nn. Trong nhng trng hp ny, the player gim t l dng khung hnh gim s dng CPU. Chng ta lp li trn nhng tnh hung gim thiu trnh duyt (scenariosminimizing the browser) hoc chi video trong mt c s nn trong thit lp iu khin v thy rng the player thc s gim RendQual n 20% (v d, dng 6-7 trong 30 khung hnh mi giy). Tht k l, nh PlayTime trong hnh 4(d) cng xut hin tai RenQual 20%. Nhng th nghim c kim sot ny xc nh gi thuyt ca chng ta rng mi quan h bt thng trn thc t do nhng ti u ha player ny i vi nhng nhi s dng chi video ch nn. 4.2.2 Case study with high impact events (Trng hp nghin cu vi nhng s kin tc ng cao) Mt mi quan tm c bit i vi nhng nh cung cp ni dung trc tip l liu cc quan st t cc s kin in hnh c th c p dng cho cc s kin tc ng cao [22]. gii quyt mi quan tm ny, chng ta xem xt b d liu LiveH. Bi v d liu c thu thp trong thi gian tng ng vi thi gian khng cung cp RenQual v RateBuf, chng ta ch tp trung vo BufRatio v AvgBitrate, m chng ta quan st cc s liu quan trng nht cho ni dung trc tip trong cc cuc tho lun trc. Hnh 12(a) v hnh 12(b) cho thy rng cc xu hng v nhng h s tng quan i vi LiveH1 ph hp cht ch vi cc kt qu vi nhng b d liu LiveA v LiveB. Chng ta cng xc nhn rng cc gi tr cho LiveH2 v LiveH3 gn nh ging ht vi LiveH1; chng ta khng trnh by nhng iu ny cho ngn gn. Nhng kt qu ny, mc d s b, cho thy rng cc quan st ca chng ta p dng cho nhng s kin k d nh vy.

Di vi birate trung bnh, nhng nh thi gian chi xung quanh mt birate ca 1:2 Mbps. Ngoi gi tr , tuy nhin, the engagement gim xung. L do cho hnh vi ny tng t nh quan st trc trong mc 4.1.1. Hu ht nhng ngi dng cui (end-users) (v d, DSL, nhng ngi s dng cap bng thng rng) khng th duy tr mt lung bng thng cao. Kt qu l, the player ng vng m hoc cng c th chuyn sang gia lung bitrate thp hn. Nh chng ta thy, buffering tc ng bt li n tri nghim ngi dng.

Metric: S liu Dataset: B d liu Negative: Tiu cc Positive: Tch cc probability mass: khi xc xut live content: ni dung trc tip anomalous behavior: hnh vi bt thng rendering quality: cht lng dng hnh play time: thi gian chi rate: t l frame rate: t l khung hnh counter-intuitive: truy cp trc quan effect: hiu ng viewer: ngi xem live: trc tip observation: quan st buffering: vng m