2017 IEEE International Conference on Big Data (BIGDATA)

$5HYLHZRQ&\EHU6HFXULW\'DWDVHWVIRU0DFKLQH/HDUQLQJ
$OJRULWKPV


2]OHP<DYDQRJOX0XUDW$\GRV
+DFHWWHSH8QLYHUVLW\   +DFHWWHSH8QLYHUVLW\
'HSDUWPHQWRI&RPSXWHU(QJLQHHULQJ'HSDUWPHQWRI&RPSXWHU(QJLQHHULQJ
$QNDUD7XUNH\$QNDUD7XUNH\
milletseverozlem@gmail.com maydos@hacettepe.edu.tr
 

Abstract²,WLVDQXQGHQLDEOHIDFWWKDWFXUUHQWO\LQIRUPDWLRQLVD 1DWLRQDO ,QVWLWXWH RI 6WDQGDUGV DQG 7HFKQRORJ\ 1,67 

SUHWW\ VLJQLILFDQW SUHVHQFH IRU DOO FRPSDQLHV RU RUJDQL]DWLRQV $PHULFDQFRPSDQLHVDVHDUO\DVVXIIHUHGORVVHVRIXSWR
7KHUHIRUHSURWHFWLQJLWVVHFXULW\LVFUXFLDODQGWKHVHFXULW\PRGHOV ELOOLRQGROODUVIROORZLQJ,7DWWDFNV>@
GULYHQ E\ UHDO GDWDVHWV KDV EHFRPH TXLWH LPSRUWDQW 7KH 
RSHUDWLRQV EDVHG RQ PLOLWDU\ JRYHUQPHQW FRPPHUFLDO DQG
FLYLOLDQV DUH OLQNHG WR WKH VHFXULW\ DQG DYDLODELOLW\ RI FRPSXWHU
V\VWHPV DQG QHWZRUN )URP WKLV SRLQW RI VHFXULW\ WKH QHWZRUN
VHFXULW\ LV D VLJQLILFDQW LVVXH EHFDXVH WKH FDSDFLW\ RI DWWDFNV LV
XQFHDVLQJO\ ULVLQJ RYHU WKH \HDUV DQG WKH\ WXUQ LQWR EH PRUH
VRSKLVWLFDWHG DQG GLVWULEXWHG 7KH REMHFWLYH RI WKLV UHYLHZ LV WR
H[SODLQDQGFRPSDUHWKHPRVWFRPPRQO\XVHGGDWDVHWV7KLVSDSHU
IRFXVHV RQ WKH GDWDVHWV XVHG LQ DUWLILFLDOLQWHOOLJHQW DQG PDFKLQH
OHDUQLQJ WHFKQLTXHV ZKLFK DUH WKH SULPDU\ WRROV IRU DQDO\]LQJ
QHWZRUNWUDIILFDQGGHWHFWLQJDEQRUPDOLWLHV

Keywords— Cyber Security, Data Mining, Artifical Intelligent,

Machine Learning, Benchmarking.

, ,1752'8&7,21
&\EHUVHFXULW\LVWKHVHWRIDSSO\LQJVHFXULW\SUHYHQWLRQVWR
SURYLGH FRQILGHQWLDOLW\ LQWHJULW\ DQG DYDLODELOLW\ RI GDWD >@
1XPHURXV GHVFULSWLRQV DUH PDGH DERXW F\EHU VHFXULW\ LQ WKH 
Fig 1. Activities of unidentified as potential attack and threat
OLWHUDWXUH$FFRUGLQJWR&DQRQJLDDQG0DQGDULQR³7KHDUWRI
to organization [2]
HQVXULQJWKHH[LVWHQFHDQGFRQWLQXLW\RIWKHLQIRUPDWLRQVRFLHW\
7KH KXJH LQFUHDVH LQ WKH SHUFHQWDJH RI F\EHUDWWDFNV KDV
RI D QDWLRQ JXDUDQWHHLQJ DQG SURWHFWLQJ LQ &\EHUVSDFH LWV
FDXVHG DUWLILFLDO LQWHOOLJHQW DQG PDFKLQH OHDUQLQJ EDVHG
LQIRUPDWLRQ DVVHWV DQG FULWLFDO LQIUDVWUXFWXUH´ >@ &\EHU
PHWKRGRORJLHV D FUXFLDO SDUW LQ GHWHFWLQJ VHFXULW\ WKUHDWV  ,Q
VHFXULW\ LV D VLJQLILFDQW UHVHDUFK DUHD EHFDXVH DOO RI WKH
RUGHUWRSURYLGHWKHEHVWVHFXULW\DSSOLFDWLRQVEHDFFHSWHGDQG
RSHUDWLRQV EDVHG RQ JRYHUQPHQW PLOLWDU\ FRPPHUFLDO
DSSURSULDWH OHYHO RI VHFXULW\ EH REWDLQHG VHFXULW\UHODWHG
ILQDQFLDO DQG FLYLOLDQV JDWKHU SURFHVV DQG VWRUH WUHPHQGRXV
EHQFKPDUNVDUHYHU\LPSRUWDQW)URPWKLVSRLQWRIYLHZWKH\
YROXPHRIGDWDRQFRPSXWHUVDQGRWKHUV>@,QRUGHUWREHRQ
DUHHVVHQWLDOIRUVHYHUDOW\SHVRIF\EHUVHFXULW\UHVHDUFKVXFKDV
WKH GHIHQVLYH VLGH RQ F\EHU VHFXULW\ FRPSDQLHV UHTXLUH
LQWUXVLRQGHWHFWLRQV\VWHP:KLOHWKHUHKDVEHHQDIHZVWXGLHV
RUJDQL]DWLRQ RI LWV HIIRUWV WKURXJKRXW LWV ZKROH LQIRUPDWLRQ
DERXW SDUWLFXODU GDWDVHWV WKHUH KDV EHHQ OHVV DERXW WKH
V\VWHP7KHFRPSRQHQWVRIF\EHUVHFXULW\FRQVLVWRIQHWZRUN
FRPSUHKHQVLYHVWDWHRIVHFXULW\UHODWHGGDWDVHWV,QOLWHUDWXUH
VHFXULW\ DSSOLFDWLRQ VHFXULW\ PRELOH VHFXULW\ GDWD VHFXULW\
WKHUHDUHQXPHURXVVWXGLHVLQWKHILHOGRIF\EHUVHFXULW\XVLQJ
HQGSRLQWVHFXULW\DQGVRRQ>@
YDULRXVGDWDVHWV>@,QWKLVVWXG\DFRPSUHKHQVLYHUHYLHZ
2YHUWKHODVWIHZ\HDUVWKHXVHRIWKH,QWHUQHWDQGFRPSXWHU
RI WKH FXUUHQW SXEOLFO\ DYDLODEOH GDWDVHWV LV JLYHQ :H DOVR
DSSOLFDWLRQVKDVVHHQDQLPPHQVHH[SDQVLRQDQGWKH\KDYHWXUQ
SURYLGH D JHQHUDO DVVHVVPHQW RI DUWLILFLDO LQWHOOLJHQW DQG
LQWRWKHLQWHJUDOSDUWRIWRGD\¶VJHQHUDWLRQRISHRSOH:LWKWKH
PDFKLQHOHDUQLQJWHFKQLTXHVXVLQJWKHVHGDWDVHWV
H[SRQHQWLDO LQFUHDVH RI FRPSXWHU DSSOLFDWLRQV DQG FRPSXWHU
7KHUHVWRIWKHSDSHULVRUJDQL]HGDVIROORZV3DUWH[SODLQV
QHWZRUNV XVDJH VHFXULW\ LV EHFRPLQJ LQFUHDVLQJO\ PRUH
WKHHVVHQWLDOVHFXULW\FRQFHSWV3DUWSUHVHQWVVXPPDU\RIWKH
VLJQLILFDQW>@$WWDFNHUV DUHDEOH WR SRWHQWLDOO\ XVH VHYHUDO
SUHYLRXV VWXGLHV 3DUW  GHVFULEHV WHFKQLTXHV WKDW EHORQJ WR
SDWKVE\PHDQVRIDSSOLFDWLRQWRGRKDYRFWR\RXUEXVLQHVVRU
PDFKLQH OHDUQLQJ DOJRULWKP DQG DUWLILFLDO LQWHOOLJHQW 3DUW 
RUJDQL]DWLRQ )LJXUH  LOOXVWUDWHV VRPH SRWHQWLDO DWWDFNV DQG
GLVFXVVHVPDMRUGDWDVHWVDQGWKHLUFKDUDFWHULVWLF)LQDOO\SDUW
WKUHDWVWRRUJDQL]DWLRQV
SUHVHQWVREVHUYDWLRQVDQGFRQFOXGLQJUHPDUNV
$OO RI WKHVH SDWKV V\PEROL]HV D ULVN WKDW PD\ RU PD\ QRW EH
VHULRXV HQRXJK WR ZDUUDQW DWWHQWLRQ >@ $FFRUGLQJ WR WKH

978-1-5386-2715-0/17/$31.00 ©2017 IEEE 2186


,, %$6,&6(&85,7<&21&(376
&\EHU VHFXULW\ LV WKH VHW RI DSSO\LQJ VHFXULW\ SUHYHQWLRQV WR
SURYLGH FRQILGHQWLDOLW\ LQWHJULW\ DQG DYDLODELOLW\ RI GDWD ,Q
WKLVVHFWLRQZHH[SODLQWKHZHOONQRZQWULDGRIFRQILGHQWLDOLW\
LQWHJULW\DQGDYDLODELOLW\ &,$ RILQIRUPDWLRQVHFXULW\>@
&RQILGHQWLDOLW\DLPVWRUHVWULFWGLVFORVXUHVDQGWRJUDQWDFFHVV
RI LQIRUPDWLRQ WR RQO\ WKH DXWKRUL]HG SHRSOH 7KDQNV WR
FRQILGHQWLDOLW\FRPSDQLHVDUHDEOHWRSURWHFWWKHLUVHQVLWLYHDQG
SULYDWHDVVHWVIURPXQDXWKRUL]HGKDQGV7KHUHDUHYDULRXVZD\V
RIHQVXULQJFRQILGHQWLDOLW\VXFKDVHQFU\SWLRQDFFHVVFRQWUROV
DQGVWHJDQRJUDSK\>@
,QWHJULW\ UHTXLUHVSURWHFWLQJGDWD LQ D FRQVLVWHQWSUHFLVHDQG
UHOLDEOHPDQQHU7KLVKDVWRJXDUDQWHHWKDWGDWDLVQRWDOWHUHGLQ
WKHFRXUVHRIDVSHFLILFSHULRG,QRUGHUWRSUHYHQWXQDXWKRUL]HG
XVHUV PDNLQJ PRGLILFDWLRQV WKH ULJKW SURFHVVHV DQG DFWLRQV
KDYHWREHWDNHQ+DVKLQJGLJLWDOVLJQDWXUHVFHUWLILFDWHVQRQ
UHSXGLDWLRQDUHWKHWRROVDQGDOJRULWKPVSURYLGLQJLQWHJULW\>
@
$YDLODELOLW\ LV DQRWKHU VHFXULW\ FRQFHSW WKDW WKH GDWD DQG
UHVRXUFHV VKRXOG EH DYDLODEOH ZKHQ SHRSOH QHHG WR DFFHVV LW
SDUWLFXODUO\GXULQJHPHUJHQFLHVRUGLVDVWHUV7KHF\EHUVHFXULW\
VSHFLDOLVWV VKRXOG KDQGOH WKH WKUHH FRPPRQ FKDOOHQJHV IRU
DYDLODELOLW\GHQLDORIVHUYLFH 'R6 ORVVRILQIRUPDWLRQV\VWHP
FDSDELOLWLHVEHFDXVHRIQDWXUDOGLVDVWHUVDQGHTXLSPHQWIDLOXUHV
GXULQJDQRUPDORSHUDWLRQ>@
,,, 5(/$7('678',(6
,QWKHOLWHUDWXUHWKHUHKDVEHHQFRQVLGHUDEOHDPRXQWRIVWXGLHV
RQWKHSUREOHPRIF\EHUVHFXULW\7KHUHDUHYDULRXVZLGHVSUHDG
DSSURDFKHVLQJHQHUDOF\EHUVHFXULW\VROXWLRQV,QWKLVVHFWLRQ
ZH KDYH IRFXV RQ XVLQJ PDFKLQH OHDUQLQJ DQG DUWLILFLDO
LQWHOOLJHQWDSSURDFKHVIRUF\EHUVHFXULW\LVVXHV
&KRZGKXU\ HW DO SURSRVHG D QHZ ERWQHW GHWHFWLRQ PHWKRG
EDVHG RQ WRSRORJLFDO IHDWXUH RI QRGHV ZLWKLQ D JUDSK 7KH
SURSRVHGPHWKRGRORJ\LVDEOHWRGHWHFWDQRPDO\E\VHDUFKLQJD
OLPLWHGQXPEHURIQRGHV7KLV PHWKRGRORJ\LVEDVHGRQVHOI
RUJDQL]LQJ PDS 620  FOXVWHULQJ WKDW EHORQJV WR D FODVV RI
XQVXSHUYLVHG V\VWHP 7KLV VWXG\ XVHG &78 GDWDVHWV WKH
ODUJHVW GDWDVHW WKDW FRQWDLQV ERW ODEHOHG QRGHV )XUWKHUPRUH
WKLV VWXG\ XVHG DQRWKHU GHWHFWLRQ DOJRULWKP VXSSRUW YHFWRU
PDFKLQH 690  IRU FRPSDULVRQ ([SHULPHQWDO UHVXOWV VKRZ
WKDWSURSRVHGPHWKRGRORJ\FRXOGEHDEOHWRVWLOOGHWHFWERWZLWK
DFFHSWDEOHDFFXUDF\E\VHDUFKLQJIHZQXPEHURIQRGHV>@
+XVH\QRYHWDOSURSRVHGDELRLQVSLUHGFRPSXWLQJWHFKQLTXH
DOVR NQRZQ DV DQW FRORQ\ FOXVWHULQJ IRU GHWHFWLRQ RI ERWQHW
DWWDFNV 7KLV SURSRVHG PRGHO LV DEOH WR H[SORUH ERWQHW KRVWV
TXLFNO\ DQG SUHFLVHO\ ZKLOH QRW GHSHQGLQJ RQ LWV WUDIILF
SD\ORDG$WWKHVDPHWLPHWKHLUDSSURDFKZDVWHVWHGXVLQJWZR
GLIIHUHQW FOXVWHULQJ DOJRULWKPV WKDW LV $77$& DQG .PHDQV
IRU FRPSDULVRQ ,627 GDWDVHW ZDV SUHIHUUHG EHFDXVH RI LWV
YROXPH>@
1HHWKX % UHSUHVHQWV D IUDPHZRUN WKDW LV 3&$ IRU IHDWXUH
VHOHFWLRQ ZLWK 1DLYH %D\HV LQ RUGHU WR GHYHORS D QHWZRUN
LQWUXVLRQ GHWHFWLRQ V\VWHP ,Q WKLV VWXG\ .''&XS 
LQWUXVLRQ GHWHFWLRQ EHQFKPDUN GDWDVHW LV SUHIHUUHG IRU
2187
H[SHULPHQWV 7KH UHVXOWV VKRZ WKDW WKH SHUIRUPDQFH RI WKLV
PHWKRGDFKLHYHVKLJKHUGHWHFWLRQUDWHOHVVWLPHFRQVXPLQJDQG
KDV ORZ FRVW IDFWRU FRPSDUHG WR WKH QHXUDO QHWZRUN DQG WUHH
DOJRULWKP EDVHG DSSURDFK ,Q DGGLWLRQ SURSRVHG V\VWHP
SURYLGHVDERXWDFFXUDF\>@
5DIDODQGHWDOSUHVHQWHGDQRYHOPHWKRGIRUGHWHFWLQJF\EHU
DWWDFNVWDUJHWLQJZHEDSSOLFDWLRQV7KLVPHWKRGZDVFRPSDUHG
ZLWK1DLYH%D\HV$GD%RRVW3DUWDQG-ZKLFKDUHPDFKLQH
OHDUQLQJDOJRULWKPV,QDGGLWLRQ&6,&+773'DWDVHWLV
XVHGIRUDVVHVVPHQWRISURSRVHGPRGHO7KLVVWXG\VSHFLILFDOO\
IRFXVHG RQ VROXWLRQV WKDW DUH XVLQJ +773 SURWRFROV WR
FRPPXQLFDWHFOLHQWVZLWKWKHVHUYHUV7KHDXWKRUVFODLPHGWKDW
WKLV PRGHO LV DEOH WR REWDLQ WKH KLJKHU GHWHFWLRQ SHUFHQWDJH
ZKLOH KDYLQJ ORZHU IDOVH SRVLWLYH UDWH $W WKH VDPH WLPH WKH
UHVXOWV VKRZ WKDW - PHWKRG LV WKH EHVW DSSURDFK IRU WKLV
SUREOHPDQGWUXHSRVLWLYHYDOXHLVDURXQG>@
1JX\HQDQG)UDQNHSURSRVHGDQDGDSWLYHLQWUXVLRQGHWHFWLRQ
V\VWHP $,'6 7KLVV\VWHPLVDEOHWRGHWHFW PDQ\GLIIHUHQW
W\SHVRIDWWDFNVLQWKHKHWHURJHQHRXVDQGDGYHUVDULDOQHWZRUN
HQYLURQPHQWV $XWKRUV FRQGXFW WKH H[SHULPHQWV RQ WZR
GLIIHUHQW GDWDVHWV IRU EHQFKPDUNLQJ :HE $SSOLFDWLRQ
)LUHZDOOVWKH(&0/3.''+773GDWDVHWDQGWKH&,6,&
+773$WWKHVDPHWLPH1DwYH%D\HV%D\HVQHWZRUN
GHFLVLRQ VWXPS DQG 5%) QHWZRUN WKDW DUH PDFKLQH OHDUQLQJ
DOJRULWKPVDUHXVHGIRUFRPSDULVRQZLWKWKHSURSRVHGPHWKRG
7KHH[SHULPHQWDOUHVXOWVLOOXVWUDWHGWKDWLQWKHFDVHRIWKH&6,&
GDWDVHWLWSURYLGHVDOPRVWDQGKLJKHUDFFXUDFLHV
WKDQWKHEHVW,'6ZKLFKLVWKH%D\HV1HWZRUNEDVHG,'6DQG
WKH+HGJH%RRVWLQJDOJRULWKPUHVSHFWLYHO\>@
;LH DQG HW DO IRFXVHG RQ GHWHFWLQJ DQRPDOLHV ZLWK D VKRUW
VHTXHQFH PRGHO ,Q WKLV VWXG\ D QRYHO DQRPDO\ GHWHFWLRQ
V\VWHP LV SURSRVHG XVLQJ 6XSSRUW 9HFWRU 0DFKLQH 690 
$')$/' LV XVHG IRU FRQGXFWLQJ H[SHULPHQWV )RU WKLV
H[SHULPHQWVNYDOXHVZHUHVHOHFWHGN DQGWKHEHVW
DFKLHYHPHQWLVREWDLQHGZLWKN ZKHUHDYHUDJH$&&RI
LVDFKLHYHGDWD)35RIDURXQG7KHH[SHULPHQWDOUHVXOW
UHSUHVHQWVWKDWLWQRWRQO\SURYLGHDVDWLVIDFWRU\DFKLHYHPHQW
EXWDOVRGHFUHDVHWKHFRPSXWDWLRQDOFRVWODUJHO\>@
=DPDQLDQG0RYDKHGLUHSUHVHQWVHYHUDOPRGHOVIRUGHWHFWLQJ
LQWUXVLRQ ,Q WKLV VWXG\ WKHVH PRGHOV DUH GLYLGHG EDVHG RQ
FODVVLFDODUWLILFLDOLQWHOOLJHQFH $, DQGEDVHGRQFRPSXWDWLRQDO
LQWHOOLJHQFH &,  VXFK DV JHQHWLF DOJRULWKPV DQG IX]]\ ORJLF
7KH\ FRQGXFWHG YDULRXV H[SHULPHQWV DQG FRPSDUHG WKHLU
DOJRULWKPV¶ SHUIRUPDQFH ([SHULPHQWDO UHVXOWV VKRZV WKDW
GHFLVLRQ WUHH DOJRULWKP KDV DFKLHYHG WKH EHVW UHVXOWV 2Q WKH
RWKHU KDQG WKLV VWXG\ H[SODLQHG KRZ GLIIHUHQW IHDWXUHV RI &,
PRGHOVFRXOGEHXVHGWREXLOGHIIHFWLYH,'6>@
,Q RUGHU WR HIILFLHQWO\ GHWHFW YDULRXV W\SHV RI QHWZRUN
LQWUXVLRQV +RTXH HW DO  SURSRVHG DQ LQWUXVLRQ GHWHFWLRQ
V\VWHP ,'6  EDVHG RQ JHQHWLF DOJRULWKPV ,Q WKLV VWXG\
SDUDPHWHUV DQG HYROXWLRQ SURFHVVHV RI *$ ZHUH H[SODLQHG LQ
GHWDLOV3URSRVHGPRGHOXVHGHYROXWLRQWKHRU\IRULQIRUPDWLRQ
HYROXWLRQLQRUGHUWRILOWHUWKHWUDIILFGDWDDQGWKXVGHFUHDVHWKH
FRPSOH[LW\ ,Q DGGLWLRQ .'' EHQFKPDUN GDWDVHW XVHG LQ
RUGHU WR HYDOXDWH WKH SHUIRUPDQFH RI WKH PRGHO 7KH
H[SHULPHQWDO UHVXOWV VKRZ WKDW WKLV PRGHO KDV DFKLHYHG
UHDVRQDEOHGHWHFWLRQUDWH>@

:DQJ DQG 3DVFKDOLGLV SURSRVHG D QRYHO DSSURDFK WKDW KDV
WZRVWDJHLQRUGHUWRGHWHFWWKHSUHVHQFHRIDERWQHWDQGWR
LGHQWLI\WKHERWV)LUVWVWDJHLVUHOHYDQWWREHFRPLQJDZDUHRI
DQRPDOLHV E\ OHYHUDJLQJ ODUJH GHYLDWLRQV RI DQ HPSLULFDO
GLVWULEXWLRQ,QDGGLWLRQWKLVVWDJHVXJJHVWVWZRWHFKQLTXHVIRU
FUHDWLQJ WKH HPSLULFDO GLVWULEXWLRQ )LUVW WHFKQLTXH LV D IORZ
EDVHG DSSURDFK HVWLPDWLQJ WKH KLVWRJUDP RI TXDQWL]HG IORZV
DQG ODWWHU LV D JUDSK EDVHG DSSURDFK HVWLPDWLQJ WKH GHJUHH
GLVWULEXWLRQ RI QRGH LQWHUDFWLRQ JUDSKV ,Q RUGHU WR GHWHFW WKH
ERWVVHFRQGVWDJHXVHVVRFLDOQHWZRUNFRPPXQLW\LQDJUDSK
WKDW FDSWXUHV FRUUHODWLRQV RI LQWHUDFWLRQV DPRQJ QRGHV RYHU
WLPH)RUWKHH[SHULPHQWVWKH\XVHGUHDOZRUOGERWQHWWUDIILF
WKDWLV&78GDWDVHW>@
%KX\DQHWDOLQWURGXFHGDQHZDSSURDFKWRFUHDWHXQELDVHG
IXOO IHDWXUH UHDOOLIH QHWZRUN LQWUXVLRQ GDWDVHWV LQ RUGHU WR
FRPSHQVDWHIRUWKHFUXFLDOODFNRIWKHDYDLODEOHGDWDVHWV7KH\
FUHDWHG D VLJQLILFDQW DPRXQW RI DQ LQWUXVLRQ GDWDVHW LQ WKH
GHYHORSPHQWDQGYDOLGDWLRQRSHUDWLRQRIGHWHFWLRQV\VWHPV,Q
DGGLWLRQWKLVVWXG\H[SODLQVDVHWRIUHTXLUHPHQWVIRUFUHDWLQJ
DQHIILFLHQWGDWDVHW)LQDOO\VL[GLIIHUHQWDWWDFNVFHQDULRVZHUH
FUHDWHGDQGGLVFXVVHGLQWKLVVWXG\>@
:LMHVLQJKHHWDOIRFXVRQGHWHFWLQJDUDQJHRIERWQHWIDPLOLHV
E\ DQDO\]LQJ QHWZRUN WUDIILF IORZV 7KHLU SURSRVHG PHWKRG
FRQVLVWVRIWZRSDUWV)LUVWSDUWVLVWKDWWKH\GHILQHDSSURSULDWH
GDWDVHWWHPSODWHVZLWKPRUHUHOHYDQWIHDWXUHVLQRUGHUWRGHWHFW
ERWQHW IURP ,3 IORZV 6HFRQG SDUW XVHG ,3 IORZ GDWD IRU
GHWHFWLQJ ERWQHW EHKDYLRUV LQ XQODEHOHG WUDIILF ,Q WKLV VWXG\
WKH\ XVHG SXEOLF DYDLODEOH ,3),; GDWDVHW 7KLV DSSURDFK LV D
QHZPHWKRGRORJ\DQGLWFRQWULEXWHGWRDYDLODEOH,3IORZEDVHG
ERWQHWGHWHFWLRQVWXGLHV>@
+DGGDGL HW DO KDYH DQDO\]HG YDULRXV ERWQHW GHWHFWLRQ
DSSURDFKHVEDVHGRQWKHPRGHOXVHGDQGW\SHRIGDWDHPSOR\HG
%RW+XQWHU DQG 6QRUW EDVHG RQ SXEOLF UXOH EDVHG V\VWHPV DUH
WZRRIDSSURDFKHV2WKHUDSSURDFKHVDUHEDVHGRQGDWDPLQLQJ
WHFKQLTXHV OLNH SDFNHW SD\ORDG EDVHG DQG WUDIILF IORZ EDVHG
7KLVVWXG\PDNHVXVHRIILYHSXEOLFO\DYDLODEOHERWQHWGDWDVHWV
VXFK DV &$,'$ ,627 HWF 7KH\ FRQGXFWHG VHYHUDO
H[SHULPHQWV XVLQJ & .QQ NQHDUHVW QHLJKERUV  690
%D\HVLDQ 1HWZRUNV ([SHULPHQWDO UHVXOWV VKRZ WKDW WKH
SHUIRUPDQFHRIWKHIORZEDVHGV\VWHPLVKLJKHURUVLPLODUWRWKH
UHVXOWVUHSRUWHGLQWKHOLWHUDWXUH>@
%KX\DQDHWDOSURSRVHGDQHPSLULFDOVWXG\XVLQJGLIIHUHQW
LQIRUPDWLRQ PHWULFV LQ RUGHU WR KDQGOH LPSRUWDQW VHFXULW\
SUREOHPV VXFK DV GHWHFWLRQ RI ERWK ORZUDWH DQG KLJKUDWH
''R6DWWDFNV7KH\FRQGXFWHGVHYHUDOH[SHULPHQWVXVLQJIRXU
VLJQLILFDQW LQIRUPDWLRQ HQWURS\ PHDVXUHV +DUWOH\ HQWURS\
6KDQQRQ HQWURS\ 5HQ\LQV HQWURS\ DQG 5HQ\LQV
JHQHUDOL]HG HQWURS\ IRU GHWHFWLQJ ''R6 DWWDFNV RI YDULRXV
W\SHV&$,'$DQG78,'6''R6GDWDVHWVDUHXVHGIRUVKRZLQJ
HIILFLHQF\DQGHIIHFWLYHQHVVRIHDFKPHWULFIRU''R6GHWHFWLRQ
>@
+RTXH HW DO SURSRVHG D QRYHO VWDWLVWLFDO PHWKRGRORJ\ LQ
RUGHU WR DQDO\]H ''R6 DWWDFN IURP QRUPDO WUDIILF 7KLV
PHWKRGRORJ\ FDOOHG DV )HDWXUH )HDWXUH 6FRUH ))6F  7KLV
VWXG\XVHGWKUHHIHDWXUHVIURPQHWZRUNWUDIILF7KHVHIHDWXUHV
DUH HQWURS\ RI VRXUFH ,3V YDULDWLRQ RI VRXUFH ,3V DQG SDFNHW
UDWH  7KH VXFFHVV RI WKH SURSRVHG PRGHO LV HYDOXDWHG ZLWK
2188
&$,'$ ''R6  DQG 0,7 '$53$ GDWDVHWV 7KH
H[SHULPHQWDO UHVXOWV VKRZ WKDW SURSRVHG PRGHO \LHOGV 
GHWHFWLRQDFFXUDF\RQWKHQRUPDOL]HG&$,'$GDWDVHW>@
.DWR DQG .O\XHY KDYH GHYHORSHG DQ ''R6 DWWDFN GHWHFWLRQ
V\VWHP$OVRWKLVVWXG\DQDO\]HGWKHFKDUDFWHULVWLFVRI''R6
DWWDFNV7KLVV\VWHPXVHG690ZLWKDQ5%) *DXVVLDQ NHUQHO
IURP PDFKLQH OHDUQLQJ 7R FRPSDUH WKH SHUIRUPDQFH RI WKH
SURSRVHG V\VWHP WKUHH W\SHV RI WUDLQLQJ DQG WHVW GDWDVHWV
LQFOXGLQJ GLIIHUHQW SDWWHUQV DQG GLIIHUHQW QXPEHU RI SDWWHUQV
ZHUH FUHDWHG )RU HYDOXDWLQJ VXFFHVV RI V\VWHP SUHFLVLRQ
UHFDOO QHJDWLYH SUHGLFWLYH YDOXH 139  ZHUH FDOFXODWHG
'HYHORSPHQWV\VWHPKDVDFKLHYHGVXFFHVVIXOUHVXOWVZLWKPRUH
WKDQDFFXUDF\ZLWKDOOW\SHVRIGDWDVHW>@
6DDG HW DO IRFXVHG RQ GHWHFWLQJ 33 ERWV WKDW UHSUHVHQWV WKH
QHZHVW DQG PRVW FKDOOHQJLQJ W\SHV RI ERWQHWV FXUUHQWO\
DYDLODEOH,QRUGHUWRGHWHFW33ERWQHWFRPPDQGDQGFRQWURO
& &  SKDVH WKH\ SURSRVHG WKH FKDUDFWHUL]DWLRQ RI QHWZRUN
WUDIILFEHKDYLRUV
,QWKLVVWXG\WKH\XVHGILYHPDFKLQHOHDUQLQJDOJRULWKPVWKDW
DUHQHDUHVWQHLJKERUVOLQHDUVXSSRUWYHFWRUPDFKLQHDUWLILFLDO
QHXUDOQHWZRUNDQGQDwYHED\HV7KHH[SHULPHQWDOUHVXOWVVKRZ
WKDWWUXHGHWHFWLRQUDWHRIWKH33%RWQHW& &LVDERYH
IRUWKH6XSSRUW9HFWRU0DFKLQH$UWLILFLDO1HXUDO1HWZRUNDQG
WKH1HDUHVW1HLJKERUV&ODVVLILHUDQGWKHWRWDOHUURUUDWHLVOHVV
WKDQ>@
$ QXPEHU RI UHODWHG GHWHFWLRQ V\VWHPV DUH FRPSDUHG DQG WKH
UHVXOWVVKRZQLQ7DEOH,QSDUWLFXODUZHFRPSDUHWKHPDFKLQH
OHDUQLQJWHFKQLTXHVXVHGIRUGHYHORSLQJWKHGHWHFWLRQV\VWHPV
GDWDVHWVXVHGIRUH[SHULPHQWVHYDOXDWLRQPHWKRGVFRQVLGHUHG
EDVHOLQHFODVVLILHUVIRUFRPSDULVRQVHWFLQUHOHYDQWVWXGLHV
,9 7(&+1,48(6
A. Machine Learning
$FFRUGLQJ WR 6WDQIRUG FRPSXWHU VFLHQFH SURIHVVRU $QGUHZ
1J0DFKLQHOHDUQLQJ 0/ LV³WKHVFLHQFHRIJHWWLQJFRPSXWHUV
WRDFWZLWKRXWEHLQJH[SOLFLWO\SURJUDPPHG´>@7KHSULPDU\
DLPRI0/LVWREXLOGPRGHOVWKDWFDQWDNHLQSXWGDWDDQGXWLOL]H
VWDWLVWLFDODQDO\VLVLQRUGHUWRIRUHFDVWDQRXWSXWYDOXHZLWKLQDQ
VXLWDEOHUDQJH,QWKHILHOGRIFRPSXWHUVFLHQFH0/LVRQHRI
WKH IDVWHVW H[SDQGLQJ DUHDV ZLWK FRPSUHKHQVLYH DSSOLFDWLRQV
0/ DOJRULWKPV DUH RIWHQ FODVVLILFDWLHG DV VXSHUYLVHG
XQVXSHUYLVHG DQG 5HLQIRUFHPHQW /HDUQLQJ 6XSHUYLVHG
DOJRULWKPVDUHWKHPRVWFRPPRQO\XVHGLQWKHPDFKLQHOHDUQLQJ
DOJRULWKPV ,Q DGGLWLRQ VXSHUYLVHG DOJRULWKPV FDQ EH IXUWKHU
JURXSHGLQWRUHJUHVVLRQDQGFODVVLILFDWLRQ,QOLWHUDWXUHVHYHUDO
PDFKLQH OHDUQLQJ DOJRULWKPV XVHG >@ &RPPRQO\ XVHG
PDFKLQHOHDUQLQJDOJRULWKPVDUH

 /LQHDU5HJUHVVLRQ
 /RJLVWLF5HJUHVVLRQ
 'HFLVLRQ7UHH
 690
 1DLYH%D\HV
 .11
 .0HDQV
 5DQGRP)RUHVW

B. Artifical Intelligence
$, LV D ILHOG RI VFLHQWLILF UHVHDUFK WR LQFUHDVH FRPSXWLQJ
SRZHU WR GHYHORS SURGXFWLYH DOJRULWKPV DQG ZHOO RUJDQL]HG
NQRZOHGJH $, DSSOLHV IRU VROYLQJ FRPSOLFDWHGSUREOHPV WKDW
FDQQRWEHVROYHGZLWKRXWFRPELQLQJLQWHOOLJHQFHGLVFRYHULQJ
WKH KLGGHQ SDWWHUQV IURP GDWD DQG GHYHORSLQJ LQWHOOLJHQW
PDFKLQHV>@
$,KDVQXPHURXVDSSOLFDWLRQVRQNQRZOHGJHUHSUHVHQWDWLRQ
LQIRUPDWLRQ UHWULHYDO VSHHFK UHFRJQLWLRQ XQGHUVWDQGLQJ
QDWXUDO ODQJXDJH FRPSXWHU YLVLRQ ELRLQIRUPDWLFV H[SHUW
V\VWHPV URERWLFV JDPH SOD\LQJ DQG F\EHU GHIHQVH ZLWK WKH
KHOSRIYDULRXVDOJRULWKPVOLNHDUWLILFLDOQHXUDOQHWZRUNJHQHWLF
DOJRULWKPV DUWLILFLDO LPPXQH V\VWHPV SDUWLFOHVZDUP
LQWHOOLJHQFHVWRFKDVWLFDOJRULWKPVDQGIX]]\ORJLF>@
$UWLILFLDO1HXUDO1HWZRUNV $11V ZKLFKLVDWHFKQLTXHRI
$,DUHVHWRIFRPSXWHUDOJRULWKPVWKDWDUHELRORJLFDOO\LQVSLUHG
WRVLPXODWHWKHZD\LQZKLFKWKHKXPDQEUDLQQHXURQSURFHVVHV
LQIRUPDWLRQ >@ $11V JDWKHU WKHLU NQRZOHGJH E\ GHWHFWLQJ
WKH SDWWHUQV DQG UHODWLRQVKLSV DPRQJ GDWD DQG OHDUQ WKURXJK
WKHLU DUFKLWHFWXUHV WUDQVIHU IXQFWLRQV DQG OHDUQLQJ DOJRULWKPV
>@
7KHUH DUH PDQ\ W\SHV RI QHXUDO QHWZRUNV IRU YDULRXV
DSSOLFDWLRQV DYDLODEOH LQ WKH OLWHUDWXUH >@ 0XOWLOD\HUHG
SHUFHSWURQ 0/3  W\SH QHXUDO QHWZRUNV DUH WKH VLPSOHVW DQG
PRVWFRPPRQO\XVHGQHXUDOQHWZRUNDUFKLWHFWXUHV>@
0/3VDUHWUDLQHGZLWKPDQ\OHDUQLQJDOJRULWKPV/HYHQEHUJ
0DUTXDUGW /0  LV RQH RI PRVW SUHIHUUHG WUDLQLQJ DOJRULWKPV
IRU0/3V



7DEOH2XWOLQHRIWKH6WXGLHV3UHVHQWHGLQWKH/LWHUDWXUH

)HDWXUH
6WXG\ 7HFKQLTXH 'DWDVHW 3UREOHP'RPDLQ (YDOXDWLRQ0HWKRG 6HOHFWLRQ
&$,'$''R60,7'$53$
>@ 6WDWLVWLFDO0HWKRG GDWDVHWV ''R6DWWDFNGHWHFWLRQ $FFXUDF\ <HV
=HXV 6QRUW =HXV 1(75(6(& 
&690.11%D\HVLDQ =HXV 1,06 &RQILFNHU 'HWHFWLRQ5DWH)DOVH
>@ 1HWZRUNV &$,'$ DQG,6278YLF %RWQHWGHWHFWLRQ 3RVLWLYH5DWH <HV
>@ 620 &78 ''R6DWWDFNGHWHFWLRQ $FFXUDF\ <HV

1DwYH%D\HV3&$
>@ DOJRULWKP .''&XS ,QWUXVLRQ'HWHFWLRQ )DOVH3RVLWLYH5DWH <HV
3UHFLVLRQ5HFDOO
1HJDWLYH3UHGLFWLYH
>@ 690 &$,'$''R6 ''R6DWWDFNGHWHFWLRQ 9DOXH <HV
1DLYH%D\HV$GD%RRVW
>@ 3DUWDQG- &6,&+773'DWDVHW :HE$SSOLFDWLRQV$WWDFN )DOVH3RVLWLYH5DWH 1R
1DwYHED\HVED\HVQHWZRUN
GHFLVLRQVWXPS5%) (&0/3.''+773&6,&
>@ QHWZRUN +773 :HE$SSOLFDWLRQV$WWDFN )DOVH3RVLWLYH5DWH 1R
+RVWEDVHG$QRPDO\
>@ NQHDUHVWQHLJKERXU N11  $')$/LQX[GDWD 'HWHFWLRQ $FFXUDF\ 1R
>@ *HQHWLF$OJRULWKP .''&XS ,QWUXVLRQ'HWHFWLRQ 'HWHFWLRQUDWH '5  <HV
.''&XS&$,'$78,'6
>@ ,QIRUPDWLRQ0HWULFV ''R6 ''R6$WWDFN'HWHFWLRQ 1$ 1R
WUXHGHWHFWLRQUDWH(UURU
>@ 11&$116901%&*%& ,627 %RWQHW'HWHFWLRQ 5DWH <HV
690-1DLYH%D\HV 7UXH3RVLWLYH)DOVH
>@ /RJLVWLF5HJUHVVLRQ ,627816:1% &ORXG6HFXULW\ 1HJDWLYH 1R
'HFLVLRQ7UHHV/DQJXDJH
>@ 0RGHOLQJ7)%DVHG (&0/3.'''DWDVHW +773$WWDFNV SUHFLVLRQUHFDOO <HV

7UXH3RVLWLYH5DWH)DOVH
>@ .11690 .'' ''R6DWWDFNGHWHFWLRQ 3RVLWLYH5DWH <HV
$GDSWLYH1HXUR)X]]\
>@ ,QIHUHQFH6\VWHP .''&$,'$ ''R6DWWDFNGHWHFWLRQ $FFXUDF\ 1R
*HQHULF)HDWXUH6HOHFWLRQ
>@ *H)6  &6,&+773'DWDVHW )HDXWXUH6HOHFWLRQ $FFXUDF\ <HV

>@ 5DQGRP)RUHVW .'' )HDXWXUH6HOHFWLRQ $FFXUDF\ <HV

7UXH3RVLWLYH)DOVH
>@ 5%)690 .'' 1HWZRUN,QWUXVLRQ'HWHFWLRQ 1HJDWLYH <HV
$GDSWLYH7LPH'HSHQGHQW
7UDQVSRUWHU$QWV&OXVWHULQJ
>@  ,627 %RWQHW'HWHFWLRQ $FFXUDF\ 1R


2189

WHFKQRORJ\XQGHUVWRRGE\WKHVHUYHU,VWKHUHDQ/'$3
9 &<%(56(&85,7<'$7$6(76
GDWDEDVHRQWKH:HE6HUYHU",VWKHUHDQ64/GDWDEDVH
1RZDGD\VVHYHUDOUHVHDUFKJURXSVSXWWRJHWKHUPDQ\W\SH RQWKH:HE6HUYHU"
RIGDWDERWKIRUWKHLURZQVWXG\SXUSRVHVDQGWRSURYLGHGDWDWR
FRPPXQLW\ UHSRVLWRULHV 7KLV VHFWLRQ H[SODLQV WKH H[LVWLQJ 4XHU\ SDUWV LQFOXGH IHDWXUHV WKDW DUH PHWKRG SURWRFRO XUL
VHFXULW\UHODWHG GDWDVHWV XVLQJ PDFKLQH OHDUQLQJ DQG DUWLILFLDO TXHU\KHDGHUVDQGERG\
LQWHOOLJHQWUHVHDUFK
C. ISOT (Information Security and Object Technology)
A. KDD Cup 1999 Dataset (DARPA1998) Dataset
'$53$KDVJDWKHUHGDQGGHDORXWWKHILUVWVWDQGDUG ,627 ,QIRUPDWLRQ 6HFXULW\ DQG 2EMHFW 7HFKQRORJ\ 
GDWD E\ 0,7 /LQFROQ /DERUDWRU\ XQGHU 'HIHQFH $GYDQFHG GDWDVHWLVDFRPELQDWLRQRIRSHQO\DYDLODEOHYDULRXVERWQHWVDQG
5HVHDUFK3URMHFWV$JHQF\ '$53$ DQG$LU)RUFH5HVHDUFK QRUPDOGDWDVHWVWKDWFRQWDLQVWRWDOWUDIILFIORZ)RU
/DERUDWRU\ $)5/ VSRQVRUVKLSWRHYDOXDWHFRPSXWHUQHWZRUN PDOLFLRXVWUDIILFLQ,627LWZDVFROOHFWHGIURP)UHQFKFKDSWHU
LQWUXVLRQGHWHFWLRQV\VWHPV.''&XSLVSDUWRIWKHGDWD RIKRQH\QHWSURMHFWWKDWFRQVLVWRI6WRUPDQG:DOHGDFERWQHWV
FROOHFWHGIURP0,7/LQFROQ/DEVLQFOXGHVWFSGXPSDQG%60 1RQPDOLFLRXV WUDIILF ZDV REWDLQHG IURP 7UDIILF /DE (ULFVRQ
OLVW ILOHV 7KLV GDWDVHW LV EDVHG RQ WKH GDWD FDSWXUHG LQ 5HVHDUFKLQ+XQJDU\$IWHUWKDWWKLVWUDIILFZDVFRPELQHGZLWK
'$53$¶,'6HYDOXDWLRQSURJUDPDQGSUHSDUHGE\6WROIRHW DQRWKHUGDWDVHWWKDWLVFUHDWHGE\/DZUHQFH%HUNHOH\1DWLRQDO
DO >@ $OVR WKLV GDWDVHW LV FRQVLGHUHG EHQFKPDUN GDWD IRU /DE /%1/  7KLV FRPSLODWLRQ FRQWDLQV JHQHUDO WUDIILF IURP
DVVHVVPHQW RI LQWUXVLRQ GHWHFWLRQ V\VWHPV 7KH GDWD LQFOXGHV QXPHURXV W\SH RI DSSOLFDWLRQV EHVLGHV WKDW +773 ZHE
IRXU PDLQ FDWHJRULHV RI DWWDFNV WKDW DUH 'HQLDORI6HUYLFH EURZVLQJ:RUOGRI:DUFUDIWWUDIILFDQGWUDIILFIURP$]XUHXV
'R6 XVHUWRURRW 85 5HPRWHWR/RFDO$WWDFN 5/ DQG ELWWRUHQWFOLHQW7KXVWKLVWUDIILFLVFRQVLGHUDEOHELJGDWDVHWIRU
3URELQJ$WWDFN$OVRWKHUHDUHWKUHHFRQWHQWIHDWXUHVDQGWKLUW\ (ULFVRQ /DE /%1/ QHWZRUN WUDFH FRYHUHG  VXEQHWV IURP
HLJKWQXPHULFDOIHDWXUHVLQWKHGDWDVHW7KHIHDWXUHVFRQVLVWRI WR0RUHRYHU/1%/WUDIILFFRQVLVWVRIDPHGLXP
EDVLFIHDWXUHVRILQGLYLGXDO7&3FRQQHFWLRQVFRQWHQWIHDWXUHV VL]HGHQWHUSULVHQHWZRUNDQGLQYROYHVILYHKXJHGDWDVHWV>@
ZLWKLQDFRQQHFWLRQVXJJHVWHGE\GRPDLQNQRZOHGJHDQGWUDIILF
IHDWXUHVFRPSXWHGXVLQJDWZRVHFRQGWLPHZLQGRZ.''¶ D. HTTP CSIC 2010 Dataset
LV RQH RI WKH PRVW ZLOGO\ SRSXODU XVHG GDWD VHW WR HYDOXDWH 7KH+773&6,&GDWDVHWLQYROYHVVHYHUDOWKRXVDQGV
SHUIRUPDQFHRIDQRPDO\GHWHFWLRQPHWKRGV$VRIWRGD\WKHUH RIZHEUHTXHVWVWKDWJHQHUDWHGDXWRPDWLFDOO\DQGGHYHORSHGDW
DUHWKLUW\UHVHDUFKHVXVLQJ.''GDWDVHW>@ ,QIRUPDWLRQ 6HFXULW\ ,QVWLWXWH RI &6,& 6SDQLVK 5HVHDUFK
1DWLRQDO &RXQFLO  7KH GDWDVHW FDQ EH XVHG IRU WHVWLQJ ZHE
B. ECML-PKDD 2007 Dataset
DWWDFN SURWHFWLRQ V\VWHPV 7KLV GDWD FRQVLVW RI  QRUPDO
7KH (&0/3.''  GDWDVHW ZDV FUHDWHG IRU WKH UHTXHVWVDQGPRUHWKDQDQRPDORXVUHTXHVWVDQG+773
(XURSHDQ &RQIHUHQFH RQ 0DFKLQH /HDUQLQJ DQG .QRZOHGJH UHTXHVWVDUHODEHOHGDVQRUPDORUDQRPDORXV)RUFRQYHQLHQFH
'LVFRYHU\ LQ  7KH (&0/3.'' 'LVFRYHU\ &KDOOHQJH WKHGDWDVHWDUHVSOLWLQWRWKUHHGLIIHUHQWVXEVHWVWKDWDUHWUDLQLQJ
ZDVDGDWDPLQLQJFRPSHWLWLRQKHOGLQFRQMXQFWLRQZLWKWKHWK DQRPDORXV DQG WHVWLQJ 7KH DQRPDORXV UHTXHVWV UHIHU WR D
(XURSHDQ&RQIHUHQFHRQ0DFKLQH/HDUQLQJ (&0/ 7DEOH,, FRPSUHKHQVLYHILHOGRIDSSOLFDWLRQOD\HUDWWDFNV,QWKLVGDWDVHW
VKRZVFKDUDFWHULVWLFVRI(&0/3.'' WKHUH DUH WKUHH W\SHV RI DWWDFNV WKDW DUH VWDWLF G\QDPLF DQG
 XQLQWHQWLRQDO LOOHJDO UHTXHVWV )RU H[DPSOH 64/ LQMHFWLRQ
7DEOH,)HDWXUHVRI(&0/3.'''DWDVHW &5/)LQMHFWLRQFURVVVLWHVFULSWLQJEXIIHURYHUIORZVHWFDUH
 G\QDPLFDWWDFNV6WDWLFDWWDFNVWU\WRUHTXHVWKLGGHQUHVRXUFHV
 7UDLQLQJ6HW 7HVW6HW
7KHVH UHTXHVWV LQFOXGH REVROHWH ILOHV VHVVLRQ ,' LQ 85/
7RWDO5HTXHVW  
9DOLG5HTXHVW       UHZULWH FRQILJXUDWLRQ ILOHV GHIDXOW ILOHV HWF 8QLQWHQWLRQDO
$WWDFNV       LOOHJDOUHTXHVWVGRQRWKDYHPDOLFLRXVLQWHQWLRQKRZHYHUWKH\
&URVV6LWH6FULSWLQJ   GRQRWIROORZWKHQRUPDOEHKDYLRURIWKHZHEDSSOLFDWLRQDQG
64/,QMHFWLRQ   GRQRWKDYHWKHVDPHVWUXFWXUHDVQRUPDOSDUDPHWHUYDOXHV IRU
/'$3,QMHFWLRQ   H[DPSOHDWHOHSKRQHQXPEHUFRPSRVHGRIOHWWHUV 7KLVGDWDVHW
;3$7+,QMHFWLRQ  
3DWKWUDYHUVDO  
KDVEHHQVXFFHVVIXOO\XVHGIRUZHEGHWHFWLRQLQSUHYLRXVZRUNV
&RPPDQG([HFXWLRQ   >@
66,  
E. CTU-13 (Czech Technical University) Dataset

7KH GDWDVHW LV GHVFULEHG LQ H[WHQVLEOH PDUNXS ODQJXDJH &78 &]HFK 7HFKQLFDO 8QLYHUVLW\  GDWDVHW LV WKH
;0/  $OO RI WKH VDPSOH LV UHSUHVHQWHG E\ D XQLTXH LG DQG FRPELQDWLRQ RI VHL]XUHV RI  GLIIHUHQW PDOZDUH LQ D
FRQVLVWVRIWKHWKUHHPDLQSDUWVWKDWDUHFRQWH[WFODVVDQGTXHU\ QRQILFWLRQDOQHWZRUNHQYLURQPHQW7KHDLPRIWKLVGDWDVHWLVWR
>@ FDSWXUH UHDO PL[HG ERWQHW WUDIILF ,QIHFWHG KRVWV JHQHUDWHG
&RQWH[WSDUWVLQFOXGHIROORZLQJIHDWXUHV ERWQHW WUDIILF DQG YHULILHG QRUPDO KRVWV JHQHUDWHG QRUPDO
 2SHUDWLQJ V\VWHP UXQQLQJ RQ WKH ZHE VHUYHU +773 WUDIILF/DVWO\%DFNJURXQGWUDIILFLVDUHPDLQGHURIWKHWUDIILF
6HUYHU WDUJHWHG E\ WKH UHTXHVW ,V WKH ;3$7+ WKDWZHGRQRWNQRZZKDWLWLVIRUVXUH7KH&78GDWDVHW
LQFOXGHV WKLUWHHQ FDSWXUHV RI GLIIHUHQW ERWQHW VDPSOHV DOVR

2190

NQRZQDVVFHQDULRV(DFKRIDOOVFHQDULRVZDVH[HFXWHGZLWKD )73+\GUD66+$GGXVHU-DYD0HWHUSUHWHU0HWHUSUHWHUDQG
SDUWLFXODUPDOZDUHWKDWXVHGYDULRXVSURWRFROVDQGFDUULHGRXW :HEVKHOODUHODXQFKHGLQWXUQDJDLQVWWKHKRVWHDFKRIZKLFK
VHYHUDO DFWLRQV 7KLV GDWDVHW LV RQH RI WKH ODUJHVW DQG PRUH UHVXOWV LQ  DEQRUPDO WUDFHV 7DEOH ,,, KDV VKRZQ WKH
ODEHOHGLQWRH[LVWLQJGDWDVHWVDQGFUHDWHGE\&788QLYHUVLW\RI FRPSRVLWLRQRI$')'/'
3UDJXHLQ&]HFK5HSXEOLFLQ)LUVWO\*ULOOHWDOKDYHXVHG 
WKH &78 GDWDVHW 7KLV VWXG\ FRPSDUHG YDULRXV ERWQHW 7DEOH,,,7KHFRPSRVLWLRQRI$')'/'
GHWHFWLRQPHWKRGRORJLHVXVLQJ&78GDWDVHWDQGSURSRVHGD 
QRYHOHUURUPHWULF>@,QWKLVVWXG\WRHYDOXDWHSHUIRUPDQFH 7UDFH7\SH 1XPEHU /DEHO
RI ERWQHW GHWHFWLRQ %&OXV DQG 7KH &RRSHUDWLYH $GDSWLYH 7UDLQLQJ  QRUPDO
9DOLGDWLRQ  QRUPDO
0HFKDQLVPIRU1HWZRUN3URWHFWLRQ &$01(3 DQG%RW+XQWHU
+\GUD)73  DWWDFN
DOJRULWKPV ZHUH XVHG 7KLV GDWDVHW KDV EHHQ XVHG LQ ORWV RI +\GUD66+  DWWDFN
VWXGLHV,Q*ULOOHWDOXVHGWKLVGDWDVHWWRPHDVXUHUHVXOWV $GGXVHU  DWWDFN
RIORFDODGDSWLYHPXOWLYDULDWHVPRRWKLQJ /$06 PRGHORQWKH -DYD0HWHUSUHWHU  DWWDFN
1HW)ORZ DQRPDO\ GHWHFWLRQ  )DOVH DODUP UDWH RI DQRPDO\ 0HWHUSUHWHU  DWWDFN
GHWHFWLRQ RQ LQWUXVLRQ GHWHFWLRQ V\VWHPV KDV EHHQ DEOH WR EH :HEVKHOO  DWWDFN
UHGXFHG WKDQNV WR SURSRVHG PRGHO >@  7KH GHWDLOV RI WKH 
VFHQDULRLVVKRZQLQ7DEOH,ZLWKSURSHUWLHV7KHDGYDQWDJHRI 7KH DLP RI $')$ GDWDVHW LV WR WDNH WKH SODFH RI H[LVWLQJ
XVLQJ WKLV GDWDVHW LV WKDW LW LV FDUHIXOO\ ODEHOHG GDWDVHW DQG EHQFKPDUN GDWD VHWV EHFDXVH WKHVH EHQFKPDUN GDWDVHWV KDYH
FDSWXULQJSURFHVVFRQGXFWHGLQFRQWUROOHGHQYLURQPHQW>@ IDLOHGWRUHIOHFWWKHFKDUDFWHULVWLFVRIPRGHUQFRPSXWHUV\VWHPV
 G. UNSW-NB15 Dataset
7DEOH,,$PRXQWRIGDWDRQHDFKERWQHWVFHQDULR
816:1%GDWDVHWZDVFUHDWHGE\WKH,;,$3HUIHFW6WRUP
'XUDWLRQ 6L]H %RW 1XPEHU WRROLQWKH&\EHU5DQJH/DERIWKH$XVWUDOLDQ&HQWUHIRU&\EHU
'DWDVHW K 1HW)ORZV *%  QDPH RIERWV %RWQHWIORZ 6HFXULW\ $&&6 7KLVGDWDVHWFRQWDLQVDSSUR[LPDWHO\RQHKRXU
 RIDQRQ\PL]HGWUDIILFWUDFHVIURPD''R6DWWDFNLQ>
    1HULV   
 @
    1HULV    7KLV GDWDVHW UHSUHVHQW QLQH W\SHV RI PDMRU DWWDFNV WKDW DUH
 )X]]HUV $QDO\VLV %DFNGRRUV 'R6 ([SORLWV *HQHULF
    5ERW    5HFRQQDLVVDQFH6KHOOFRGHDQG:RUPV,QRUGHUWRFODVVLI\WKLV
    5ERW   
GDWDVHW,;,$3HUIHFW6WRUPWRROKDVDFKLHYHGUHSRUWIURPWKH
    9LUXW    
    0HQWL    DWWDFNGDWD7DEOH,9LOOXVWUDWHVW\SHVRIPRGHUQDWWDFNLQWKLV
    6RJRX     GDWDVHW
    0XUOR    
 7DEOH,9)HDWXUHVRI816:1%'DWDVHW
    1HULV   
 &DWHJRU\ 7UDQLQJVHW 7HVWLQJVHW
    5ERW    1RUPDO  
    5ERW     $QDO\VLV  
    16,6D\   
%DFNGRRU  

    9LUXW    'R6  
 ([SORLWV  
)X]]HUV  
F. The ADFA Datasets *HQHULF  
,Q WKH ILHOG RI KRVWEDVHG DQRPDO\ GHWHFWLRQ PRVW RI WKH 5HFRQQDLVVDQFH  
H[LVWLQJEHQFKPDUNGDWDVHWVVXFKDV801>@DQG'$53$>@
6KHOOFRGH  
LQWUXVLRQGHWHFWLRQGDWDVHWVZHUHFRPSLOHGDGHFDGHDJRDQG
:RUPV  
KDYH IDLOHG WR UHIOHFW WKH FKDUDFWHULVWLFV RI PRGHUQ FRPSXWHU
7RWDO5HFRUGV  
V\VWHPV ,Q  $XVWUDOLDQ 'HIHQFH )RUFH $FDGHP\ /LQX[
'DWDVHW KDV EHHQ UHOHDVHG E\ WKH $XVWUDOLDQ 'HIHQFH )RUFH 
$FDGHP\LQ8QLYHUVLW\RI1HZ6RXWK:DOH,QRUGHUWRHYDOXDWH 7KHUH DUH  IHDWXUHV LQ WKLV GDWDVHW ,Q RUGHU WR H[WUDFW
KRVW EDVHG LQWUXVLRQ GHWHFWLRQ V\VWHP $')$ GDWDVHW /LQX[ IHDWXUHV$UJXV%UR,'6WRROVZHUHXVHGDQGPRGHOVZHUH
GDWDVHW ZDVJHQHUDWHGRQD8EXQWX/LQX[KRVW26ZLWK GHYHORSHG)HDWXUHVDUHFDWHJRUL]HGLQWRRQO\ILYHJURXSVWKDW
$SDFKHUXQQLQJ3+3)7366+0\64/ DUHIORZIHDWXUHVEDVLFIHDWXUHVFRQWHQWIHDWXUHVWLPHIHDWXUHV
DQG 7LNL:LNL ZHUH VWDUWHG 7KLV GDWDVHW LQYROYHV QRUPDO DQG DQGDGGLWLRQDOJHQHUDWHGIHDWXUHV&RPSDUHGWRH[LVWLQJGDWDVHW
DWWDFN/LQX[EDVHGV\VWHPFDOOVWUDFHV:KHQDVDPSOLQJVWDJH WKLV GDWDVHW KDV VHYHUDO DWWDFN IDPLOLHV WKDW XOWLPDWHO\ UHIOHFW
WKHKRVWWKDWLVFRQILJXUHGWRUHSUHVHQWDPRGHUQ/LQX[VHUYHU PRGHUQORZIRRWSULQWDWWDFNV>@
FDSWXUHVWKHV\VWHPFDOOWUDFHVZKHUHOHJLWLPDWHSURJUDPVDUH
RSHUDWHGDVXVXDO6XEVHTXHQWO\WKHF\EHUDWWDFNVLH+\GUD

2191

9, &21&/86,21
7KHSURWHFWLRQRIFRPSXWHUV\VWHPVIURPF\EHUDWWDFNVLVRQH
RI WKH PDLQ LVVXHV IRU QDWLRQDO DQG LQWHUQDWLRQDO VHFXULW\
9DULRXVUHVHDUFKHVKDYHEHHQFRQGXFWHGXVLQJVHYHUDOGDWDVHWV
DQGDUWLILFLDOLQWHOOLJHQWDQGPDFKLQHOHDUQLQJSOD\DVLJQLILFDQW
UROHLQSURWHFWLRQRIFRPSXWHUV\VWHPV,QWKLVSDSHUZHKDYH
RXWOLQHGDFRPSUHKHQVLYHFODVVHVRIYDULRXVGDWDVHWVDORQJZLWK
WKHLUDGYDQWDJHVDQGGLVDGYDQWDJHV,QWKHIXWXUHZHDUHJRLQJ
WRSODQJHQHUDWLQJDQHZGDWDVHWDQGPDNHLWSXEOLFO\DYDLODEOH
 
$&.12:/('*0(17
7KHDXWKRUVZRXOGOLNHWRWKDQNDQRQ\PRXVUHYLHZHUVIRUWKHLU
FRQVWUXFWLYHFRPPHQWVDQGYDOXDEOHVXJJHVWLRQV
5()(5(1&(6
>@ 6XPHHW ' ;LDQ ' ³'DWD 0LQLQJ DQG 0DFKLQH /HDUQLQJ LQ
&\EHUVHFXULW\´&5&SUHVV
>@ &ODXGLD & 0DQGDULQR 5 -U &\EHUVHFXULW\ 7KH 1HZ &KDOOHQJH RI
WKH +DQGERRN RI 5HVHDUFK RQ %XVLQHVV 6RFLDO 1HWZRUNLQJ
2UJDQL]DWLRQDO 0DQDJHULDO DQG 7HFKQRORJLFDO 'LPHQVLRQV
2UJDQL]DWLRQDO0DQDJHULDODQG7HFKQRORJLFDO'LPHQVLRQV
>@ 3DXO7´&\EHU6HFXULW\7KUHDWV´
>@ 5RVVRXZ 9 1LHNHUN - 9 )URP LQIRUPDWLRQ VHFXULW\ WR F\EHU
VHFXULW\FRPSXWHUV VHFXULW\
>@ )UDOH\ - % &DQQDG\ - 7KH SURPLVH RI PDFKLQH OHDUQLQJ LQ
F\EHUVHFXULW\6RXWKHDVW&RQ,(((
>@ KWWSVZZZV\PDQWHFFRPFRQWHQWGDPV\PDQWHFGRFVRWKHU
UHVRXUFHVZHEDSSOLFDWLRQILUHZDOORZDVSWRSFRYHUDJHHQSGI
 DFFHVVHG
>@ %XF]DN$/*XYHQ($VXUYH\RIGDWDPLQLQJDQGPDFKLQHOHDUQLQJ
PHWKRGVIRUF\EHUVHFXULW\LQWUXVLRQGHWHFWLRQ,((( &RPPXQLFDWLRQV
6XUYH\V 7XWRULDOV
>@ 7KXUDLVLQJKDP % 'DWD PLQLQJ IRU VHFXULW\ DSSOLFDWLRQV (PEHGGHG
DQG 8ELTXLWRXV &RPSXWLQJ  (8&  ,(((,),3 ,QWHUQDWLRQDO
&RQIHUHQFHRQ9RO,(((
>@ 0HVKUDP$+DDV&$QRPDO\GHWHFWLRQLQLQGXVWULDOQHWZRUNVXVLQJ
PDFKLQH OHDUQLQJ D URDGPDS 0DFKLQH /HDUQLQJ IRU &\EHU 3K\VLFDO
6\VWHPV6SULQJHU%HUOLQ+HLGHOEHUJ
>@ )HLO\ 0 $OLUH]D 6 6XUHVZDUDQ 5 $ VXUYH\ RI ERWQHW DQG ERWQHW
GHWHFWLRQ (PHUJLQJ 6HFXULW\ ,QIRUPDWLRQ 6\VWHPV DQG 7HFKQRORJLHV
6(&85:$5( 7KLUG,QWHUQDWLRQDO&RQIHUHQFHRQ,(((
>@ 6DOHP 0% 6KORPR + 6DOYDWRUH - 6 $ VXUYH\ RI LQVLGHU DWWDFN
GHWHFWLRQUHVHDUFK,QVLGHU$WWDFNDQG&\EHU6HFXULW\
>@ &KRZGKXU\6%RWQHWGHWHFWLRQXVLQJJUDSKEDVHGIHDWXUHFOXVWHULQJ
-RXUQDORI%LJ'DWD
>@ 1HHWKX % ³$GDSWLYH ,QWUXVLRQ 'HWHFWLRQ 8VLQJ 0DFKLQH /HDUQLQJ´
,QWHUQDWLRQDO -RXUQDO RI &RPSXWHU 6FLHQFH DQG 1HWZRUN 6HFXULW\
,-&616   
>@ .R]LN 5 &KRUDĞ 0 5HQN 5 +RáXERZLF] : ³$ 3URSRVDO RI
$OJRULWKP IRU :HE $SSOLFDWLRQV &\EHU $WWDFN 'HWHFWLRQ´ ,Q ,),3
,QWHUQDWLRQDO &RQIHUHQFH RQ &RPSXWHU ,QIRUPDWLRQ 6\VWHPV DQG
,QGXVWULDO0DQDJHPHQW SS 6SULQJHU%HUOLQ+HLGHOEHUJ
>@ 1JX\HQ+7)5$1.(.³$GDSWLYH,QWUXVLRQ'HWHFWLRQ6\VWHPYLD
RQOLQH PDFKLQH OHDUQLQJ´ ,Q +\EULG ,QWHOOLJHQW 6\VWHPV +,6 WK
,QWHUQDWLRQDO&RQIHUHQFHRQ,(((
>@ ;LH0-LDQNXQ+-LOO6(YDOXDWLQJKRVWEDVHGDQRPDO\GHWHFWLRQ
V\VWHPV$SSOLFDWLRQRIWKHRQHFODVVVYPDOJRULWKPWRDGIDOG)X]]\
6\VWHPV DQG .QRZOHGJH 'LVFRYHU\ )6.'  WK ,QWHUQDWLRQDO
&RQIHUHQFHRQ,(((
>@ =DPDQL 0 0DKQXVK 0 0DFKLQH OHDUQLQJ WHFKQLTXHV IRU LQWUXVLRQ
GHWHFWLRQ´DU;LYSUHSULQWDU;LY
2192
>@ +RTXH060XNLW0%LNDV01DVHU$³$QLPSOHPHQWDWLRQRI
LQWUXVLRQ GHWHFWLRQ V\VWHP XVLQJ JHQHWLF DOJRULWKP´ DU;LY SUHSULQW
DU;LY
>@ :DQJ-,RDQQLV&K3DVFKDOLGLV%RWQHWGHWHFWLRQEDVHGRQDQRPDO\DQG
FRPPXQLW\ GHWHFWLRQ ,((( 7UDQVDFWLRQV RQ &RQWURO RI 1HWZRUN
6\VWHPV
>@ %KX\DQ0+'KUXED.%-XJDO..7RZDUGV*HQHUDWLQJ5HDO
OLIH'DWDVHWVIRU1HWZRUN,QWUXVLRQ'HWHFWLRQ,-1HWZRUN6HFXULW\

>@ :LMHVLQJKH88GD\D79LMD\9$QHQKDQFHGPRGHOIRUQHWZRUNIORZ
EDVHGERWQHWGHWHFWLRQ3URFHHGLQJVRIWKHWK$XVWUDODVLDQ&RPSXWHU
6FLHQFH&RQIHUHQFH $&6& 9RO
>@ +DGGDGL)/H/'3RUWHU=LQFLU+H\ZRRG/³2QWKH(IIHFWLYHQHVV
RI 'LIIHUHQW %RWQHW 'HWHFWLRQ $SSURDFKHV´ ,Q ,63(& SS  

>@ %KX\DQ 0 + %KDWWDFKDU\\D ' . -XJDO . .DOLWD $Q HPSLULFDO
HYDOXDWLRQRILQIRUPDWLRQPHWULFVIRUORZUDWHDQGKLJKUDWH''R6DWWDFN
GHWHFWLRQ3DWWHUQ5HFRJQLWLRQ/HWWHUV
>@ +RTXH1'KUXED.%-XJDO..DOLWD$QRYHOPHDVXUHIRUORZUDWH
DQG KLJKUDWH ''R6 DWWDFN GHWHFWLRQ XVLQJ PXOWLYDULDWH GDWD DQDO\VLV
&RPPXQLFDWLRQ6\VWHPVDQG1HWZRUNV &2061(76 WK,QWHUQDWLRQDO
&RQIHUHQFHRQ,(((
>@ .DWR.9LWDO\.$Q,QWHOOLJHQW''R6$WWDFN'HWHFWLRQ6\VWHP8VLQJ
3DFNHW$QDO\VLVDQG6XSSRUW9HFWRU0DFKLQH,-,&5
>@ 6KHULI6,VVD7$OL$*KRUEDQL%6'DYLG=:HL/X-RKQ)HOL[
3D\PDQ +DNLPLDQ 'HWHFWLQJ 33 ERWQHWV WKURXJK QHWZRUN EHKDYLRU
DQDO\VLVDQGPDFKLQHOHDUQLQJ3URFHHGLQJVRIWK$QQXDO&RQIHUHQFH
RQ 3ULYDF\ 6HFXULW\ DQG 7UXVW 367  0RQWUHDO 4XHEHF &DQDGD

>@ %KDPDUH'6DOPDQ76DPDND0(UEDG$-DLQ5³)HDVLELOLW\RI
6XSHUYLVHG 0DFKLQH /HDUQLQJ IRU &ORXG 6HFXULW\´ ,Q ,QIRUPDWLRQ
6FLHQFHDQG6HFXULW\ ,&,66 ,QWHUQDWLRQDO&RQIHUHQFHRQ SS
 
>@ *DOOODJKHU%(OLDVVÕUDG7³&ODVVLILFDWLRQRIKWWSDWWDFNVDVWXG\RQ
WKH (&0/3.''  GLVFRYHU\ FKDOOHQJH´ /DZUHQFH /LYHUPRUH
1DWLRQDO/DERUDWRU\ //1/ /LYHUPRUH&$
>@ $KPDG<81XU,$OL6³$Q(YDOXDWLRQRQ.11690$OJRULWKP
IRU 'HWHFWLRQ DQG 3UHGLFWLRQ RI ''R6 $WWDFN´ ,Q ,QWHUQDWLRQDO
&RQIHUHQFHRQ,QGXVWULDO(QJLQHHULQJDQG2WKHU$SSOLFDWLRQVRI$SSOLHG
,QWHOOLJHQW6\VWHPV6SULQJHU,QWHUQDWLRQDO3XEOLVKLQJ
>@ .XPDU.$UXQ5DM36HOYDNXPDU6'HWHFWLRQRIGLVWULEXWHGGHQLDO
RIVHUYLFHDWWDFNVXVLQJDQHQVHPEOHRIDGDSWLYHDQGK\EULGQHXURIX]]\
V\VWHPV&RPSXWHU&RPPXQLFDWLRQV
>@ 7RUUDQR*LPHQH]&1JX\HQ+ÈOYDUH]*3HWURYLF6)UDQNH.
³$SSO\LQJ )HDWXUH 6HOHFWLRQ WR 3D\ORDG%DVHG :HE $SSOLFDWLRQ
)LUHZDOOV´ ,Q 3URF RI ,QWHUQDWLRQDO :RUNVKRS RQ 6HFXULW\ DQG
&RPPXQLFDWLRQ1HWZRUNV ,:6&1 SS(GLWRU3DWULF%RXUV
*M¡YLF 1RUXHJD ,6%10D\R
>@ +DVDQ 0 1DVVHU 0 $KPDG 6 0ROOD . ³)HDWXUH 6HOHFWLRQ IRU
,QWUXVLRQ 'HWHFWLRQ 8VLQJ 5DQGRP )RUHVW´ -RXUQDO RI ,QIRUPDWLRQ
6HFXULW\
>@ 0UXW\XQMD\D 3 $EUDKDP $ 5DQMDQ 3DWUD 0 $ K\EULG LQWHOOLJHQW
DSSURDFK IRU QHWZRUN LQWUXVLRQ GHWHFWLRQ 3URFHGLD (QJLQHHULQJ 

>@ /LSSPDQQ53)ULHG'-*UDI,-:+DLQHV.'0F&OXQJ'
:HEEHU6:\VFKRJUDG'&XQQLQJKDQ5=LVVPDQ0³(YDOXDWLQJ
,QWUXVLRQ 'HWHFWLRQ 6\VWHPV 7KH  '$53$ RIIOLQH LQWUXVLRQ
GHWHFWLRQ HYDOXDWLRQ´ ,Q 3URF RI '$53$ ,QIRUPDWLRQ 6XUYLYDELOLW\
&RQIHUHQFHDQG([SRVLWLRQ ',6&(; +LOWRQ+HDG6RXWK&DUROLQD
-DQXDU\,(((&RPSXWHU6RFLHW\3UHVV/RV$ODPLWRV&$

>@ /LSSPDQQ5+DLQHV-:)ULHG'-.RUED-.³7KH'$53$
2II/LQH,QWUXVLRQ'HWHFWLRQ(YDOXDWLRQ´,Q3URF5HFHQW$GYDQFHVLQ
,QWUXVLRQ'HWHFWLRQ 5$,' +'HEDU/0HDQG6):X(GV
6SULQJHU9HUODJ1HZ<RUN1<
>@ 0F+XJK - ³7HVWLQJ ,QWUXVLRQ 'HWHFWLRQ 6\VWHPV $ &ULWLTXH RI WKH
 DQG  '$53$ ,QWUXVLRQ 'HWHFWLRQ 6\VWHP (YDOXDWLRQV DV
3HUIRUPHG E\ /LQFROQ /DERUDWRU\´ ,Q 3URF RI $&0 7UDQVDFWLRQV RQ
,QIRUPDWLRQDQG6\VWHP6HFXULW\ 7,66(&   SS

>@ 3HUH]9LOOHJDV$7RUUDQR*LPHQH]&$OYDUH]*³$SSO\LQJ0DUNRY
&KDLQVWR:HE,QWUXVLRQ'HWHFWLRQ´,Q3URFRI5HXQLyQ(VSDxRODVREUH
&ULSWRORJtD\6HJXULGDGGHOD,QIRUPDFLyQ 5(&6, SS
3XEOLFDFLRQHVXUY7DUUDJRQD (VSDxD 6HSWLHPEUH
>@ 7RUUDQR*LPHQH]&3HUH]9LOOHJDV$$OYDUH]*³$QDQRPDO\EDVHG
DSSURDFKIRULQWUXVLRQGHWHFWLRQLQZHEWUDIILF´-RXUQDORI,QIRUPDWLRQ
$VVXUDQFHDQG6HFXULW\YROLVVXHSS,661

>@ 7RUUDQR*LPHQH]&3HUH]9LOOHJDV$$OYDUH]*³$6HOI/HDUQLQJ
$QRPDO\%DVHG:HE$SSOLFDWLRQ)LUHZDOO´,Q3URFRIQG,QWHUQDWLRQDO
:RUNVKRS LQ &RPSXWDWLRQDO ,QWHOOLJHQFH LQ 6HFXULW\ IRU ,QIRUPDWLRQ
6\VWHPV &,6,6 $GYDQFHVLQ,QWHOOLJHQWDQG6RIW&RPSXWLQJYRO
SS  6SULQJHU9HUODJ $ +HUUHUR 3 *DVWDOGR 5 =XQLQR (
&RUFKDGRHGLWRUHV%XUJRV (VSDxD 6HSWLHPEUH
>@ 7RUUDQR*LPHQH] & 3HUH]9LOOHJDV $ $OYDUH] * ³$Q $QRPDO\
EDVHG:HE$SSOLFDWLRQ)LUHZDOO´,Q3URFRI,QWHUQDWLRQDO&RQIHUHQFHRQ
6HFXULW\DQG&U\SWRJUDSK\ 6(&5<37 SS,167,&&3UHVV
()HUQiQGH]0HGLQD00DOHN-+HUQDQGRHGLWRUHV0LOiQ ,WDOLD 
-XOLR





2193