International Journal of Artificial Intelligence,

ISSN 0974-0635; Int. J. Artif. Intell.

2016, Volume 14, Issue Number 1
Copyright 2016 CESER PUBLICATIONS


1(8&2*$5$1HXURPRGXODWLQJ&RJQLWLYH$UFKLWHFWXUH
IRU%LRPLPHWLF(PRWLRQDO$,

-RUGL9DOOYHUGX0D[7DODQRY6DOYDWRUH'LVWHIDQR

0DQXHO0D]]DUD0DUFR0DQFDDQG$OH[DQGHU7FKLWFKLJLQ

8QLYHUVLWDW$XWzQRPDGH%DUFHORQD&DWDORQLD6SDLQ
MRUGLYDOOYHUGX#XDEFDW

.D]DQ)HGHUDO8QLYHUVLW\5XVVLD
^PBWDODQRYVBGLVWHIDQRDBWFKLFKLJLQ`#LWNIXUX

3ROLWHFQLFRGL0LODQR,WDO\
VDOYDWRUHGLVWHIDQR#SROLPLLW

,QQRSROLV8QLYHUVLW\5XVVLD
^PPD]]DUDDFKLFKLJLQ`#LQQRSROLVUX

'HSDUWPHQWRI'HYHORSPHQWDQG,QQRYDWLRQ&(516ZLW]HUODQG
PDUFRPDQFD#FHUQFK


$%675$&7

This paper introduces a new model of artificial cognitive architecture for intelligent systems, the
Neuromodulating Cognitive Architecture (NEUCOGAR). The model is biomimetically inspired and
adapts the neuromodulators role of human brains into computational environments. This way we
aim at achieving more efficient Artificial Intelligence solutions based on the biological inspiration
of the deep functioning of human brain, which is highly emotional. The analysis of new data
obtained from neurology, psychology philosophy and anthropology allows us to generate a
mapping of monoamine neuromodulators and to apply it to computational system parameters.
Artificial cognitive systems can then better perform complex tasks (regarding information selection
and discrimination, attention, innovation, creativity,) as well as engaging in affordable emotional
relationships with human users.


.H\ZRUGV $IIHFWLYH &RPSXWLQJ &RJQLWLYH 0RGHOLQJ &RPSXWLQJ (PRWLRQV 0DFKLQH 7KLQNLQJ
(PRWLRQDO7KLQNLQJ1HXURPRGXODWLRQ1HXURWUDQVPLVVLRQ

0DWKHPDWLFV6XEMHFW&ODVVLILFDWLRQ%57



,1752'8&7,21

+XPDQPLQGLQLWVFRPSOH[LW\LVDSRZHUIXODQGHIIHFWLYHPDFKLQH'HVSLWHEHLQJODUJHO\XQH[SORUHG
VHYHUDOKXPDQELRLQVSLUHGDSSURDFKHVHPHUJHGLQ&RPSXWHU6FLHQFHWRFRSHZLWKRSHQSUREOHPVLQ
$UWLILFLDO ,QWHOOLJHQFH +XPDQELRLQVSLUHG V\VWHPV LPSOHPHQW FRJQLWLYH SURFHVVHV DQG UHODWHG
PHFKDQLVPVWROHDUQIURPH[SHULHQFHDQGFRQVHTXHQWO\WDNHGHFLVLRQV+RZHYHUWKHVHDSSURDFKHV
FDQQRW WDNH LQWR DFFRXQW DOO RI WKH PLQGV G\QDPLF DQG FUHDWLYH DVSHFWV E\ MXVW PRGHOLQJ FKHPLFDO
DQGSK\VLRORJLFDOUHODWLRQVKLSVDPRQJQHXURQV

www.ceser.in/ceserp
www.ceserp.com/cp-jour

www.ceserpublications.com
 International Journal of Artificial Intelligence

7R RYHUFRPH WKLV LVVXH VHYHUDO ZRUNV LQ OLWHUDWXUH DGRSWHG DQ HPRWLRQGULYHQ FRJQLWLYH DSSURDFK
.DQVHL HQJLQHHULQJ LPSOHPHQWHG DQ HPRWLRQDORULHQWHG DSSURDFK 1DJDPDFKL  DQ HPRWLRQDO
FRJQLWLYHDUFKLWHFWXUHLVIRXQGLQ&RJ$II6ORPDQDQG&KULVOH\DQGWKH2&&PRGHORI2UWRQ\
&ORUH DQG &ROOLQV  LV EDVHG RQ WKH FRJQLWLYH DSSUDLVDO WKHRU\ RI FRPSXWDWLRQDO PRGHOV IRU
HPRWLRQJHQHUDWLRQ$VLJQLILFDQWVWHSLQVXFKGLUHFWLRQLVWDNHQE\DIIHFWLYHFRPSXWLQJ3LFDUG
DQG VRFLDO URERWLFV %UHD]HDO  ZKLFK LQWURGXFH HPRWLRQEDVHG PDFKLQHU\ $QRWKHU LPSRUWDQW
FRQWULEXWLRQZKLFKUHDIILUPVWKHLQH[WULFDEOHFRQQHFWLRQEHWZHHQHPRWLRQVDQGFRJQLWLRQLVSURSRVHG
E\0LQVN\ZKRDJDLQPDNHVWKHFDVHIRUWKHQHHGIRUHPRWLRQDOPDFKLQHV0RUHUHFHQWO\DQ
LQWHUHVWLQJ FKDUDFWHUL]DWLRQ RI HPRWLRQV LV SURYLGHG E\ /|YKHLP  ZKHUH HPRWLRQV DUH
FDWHJRUL]HG RQ D WKUHHGLPHQVLRQDO PRGHO RI PRQRDPLQH QHXURWUDQVPLWWHUV VHURWRQLQ GRSDPLQH
QRUDGUHQDOLQH 7KHVH ZRUNV KLJKOLJKW WKH LPSRUWDQFH RI HPRWLRQV LQ $, DQG URERWLFV +RZHYHU D
FRPSUHKHQVLYHFRJQLWLYHIUDPHZRUNWKDWWDNHVHPRWLRQVLQWRDFFRXQWLVVWLOOODFNLQJLQOLWHUDWXUH

,QWKLVSDSHUZHILOOWKLVJDSE\SURYLGLQJDQLQQRYDWLYHELRLQVSLUHGFRJQLWLYHDUFKLWHFWXUH%XLOGLQJXS
IURP /|YKHLPV WKUHHGLPHQVLRQDO PRGHO RI HPRWLRQV ZH SURSRVH WKH 1(8URPRGXODWLQJ &2JQLWLYH
$5FKLWHFWXUH 1(8&2*$5 WR FDWHJRUL]H FRPSXWLQJ V\VWHP FKDUDFWHULVWLFV DQG SURSHUWLHV
SURFHVVLQJ VWRUDJH DQG PHPRU\ PHWULFV YLD HPRWLRQOLNH VWDWHV 7KLV KHOSV LQ EXLOGLQJ QHZ
FODVVLILHUVDQGSUHGLFWLRQWHFKQLTXHVWRLQIHUWKHEHKDYLRURIFRPSXWLQJV\VWHPV

7KLV SDSHU H[WHQGV RXU SUHYLRXV UHVXOWV DV SUHVHQWHG LQ 7DODQRY HW DO  ,Q SDUWLFXODU IXUWKHU
DUWLIDFWV DQG GHWDLOV KDYH EHHQ DGGHG WR WKH 1(8&2*$5 PRGHO WR EHWWHU GHVFULEH DQG MXVWLI\ WKH
RYHUDOODSSURDFK$SUHOLPLQDU\LPSOHPHQWDWLRQZLWKUHVXOWVIURPH[SHULPHQWVKDVEHHQGRFXPHQWHG
DVZHOODVDQLQGHSWKFRPSDULVRQZLWKH[LVWLQJDUFKLWHFWXUHVDQGVROXWLRQV

7KH UHPDLQGHU RI WKH SDSHU LV RUJDQL]HG DV IROORZV DQ RYHUYLHZ RI UHODWHG ZRUN DQG EDFNJURXQG
FRQFHSWV LV SURYLGHG LQ 6HFWLRQ  ZKLOH RXU HPRWLRQEDVHG DSSURDFK 1(8&2*$5 LV GHWDLOHG LQ
6HFWLRQ  6HFWLRQ  UHSRUWV RQ D SUHOLPLQDU\ LPSOHPHQWDWLRQ DQG YDOLGDWLRQ RI WKH SURSRVHG
DUFKLWHFWXUH6HFWLRQFORVHVWKHSDSHUZLWKVRPHGLVFXVVLRQDQGFRQFOXVLRQV


%$&.*5281'

5HFHQWO\ D QHZ JHQHUDWLRQ RI FRJQLWLYH VFLHQWLVWV DQG URERWLFLVWV SURSRVHG D SDUDGLJP VKLIW ILUVWO\
HPRWLRQVZHUHDIXQGDPHQWDOSDUWRIWKHFRJQLWLYHSURFHVVHVDWWHQWLRQPRWLYDWLRQVWUDWHJ\VHOHFWLRQ
PRRG GLVSRVDO UHDFWLRQ LQYHQWLRQ DPRQJ D ORQJ OLVW VHFRQGO\ WKH LQWULQVLF UHODWLRQVKLS EHWZHHQ
PLQG DQG ERG\ OHG WR WKH ELUWK RI HPERGLHG FRJQLWLRQ RU JURXQGHG FRJQLWLRQ %DUVDORX  7KLV
DOORZHG WKH HPHUJHQFH RI D QHZ JHQHUDWLRQ RI FRJQLWLYH DQG URERWLFV WKHRU\ OHG E\ SHRSOH OLNH
%URRNV  %URRNV DQG 6WHLQ  RU 'H/DQFH\  5RERWLFV &ODUN  3KLORVRSK\
'DPDVLR5DPDFKDQGUDQRU5L]]RODWWL1HXURORJ\


28

 International Journal of Artificial Intelligence

,Q WKH PLGGOH RI WKLV FRJQLWLYH UHYROXWLRQ WKDW OHG WR HPERGLHG URERWLFV HQDFWLYH FRJQLWLRQ RU
PRUSKRORJLFDO FRPSXWDWLRQ DSSURDFKHV DQ LPSRUWDQW GLVFLSOLQH KDV EHHQ UHFRJQL]HG DV D PDLQ
UHIHUHQFHQHXURORJ\1HXURORJ\UHYROXWLRQL]HGDUWLILFLDOLQWHOOLJHQFHVFLHQFHZLWKGDWDFRPLQJIURPLQ
YLYR VFDQQHG EUDLQV E\ HOHFWURHQFHSKDORJUDPV DQG PDJQHWLF UHVRQDQFH LPDJLQJ DQG KLJKOLJKWHG
WKHLPSRUWDQFHRIHPRWLRQDOSURFHVVHVLQFRJQLWLRQG\QDPLFV

+DYLQJ PDGH WKH FDVH IRU WKH FUXFLDO UROH RI JURXQGHG DQG HPRWLRQDO DVSHFWV LQ FRJQLWLYH
DUFKLWHFWXUHVZHDVVXPHWKH/|YKHLPPRGHODVDUHIHUHQFHIRURXURZQDUFKLWHFWXUH/|YKHLP
EDVHG KLV WKHRU\ RQ D WKUHHGLPHQVLRQDO PRGHO RI HPRWLRQV DQG PRQRDPLQH QHXURWUDQVPLWWHUV
VHURWRQLQGRSDPLQHQRUDGUHQDOLQH,QKLVWKHRU\WKHPRQRDPLQHVDUHPXWXDOO\RUWKRJRQDODQGWKLV
PD[LPL]HVWKHDPRXQWRILQIRUPDWLRQWKDWFDQEHHQFRGHG+RZHYHUWKLVLVDKHDYLO\VLPSOLILHGPRGHO
WKDWGRHVQRWWUXO\UHIOHFWWKHWUXHFRPSOH[LW\RIELRORJLFDOV\VWHPVEXWQHYHUWKHOHVVLVDFRQYHQLHQW
PRGHOIRUDILUVWDWWHPSWDWDQHPRWLRQDO$,V\VWHPVXFKDVWKHRQHZHDLPWRGHVFULEH

7R GHVFULEH WKLV PRGHO LQ PRUH GHWDLO WKH YHUWH[HV RI WKH PRGHO DUH affects DV GHILQHG E\ WKH
7RPNLQV WKHRU\ ZKLFK GHVFULEHV HLJKW EDVLF HPRWLRQV 7RPNLQV  .HOO\  7RPNLQV ODEHOHG
HDFKHPRWLRQZLWKWZRGLIIHUHQWZRUGVGHSHQGLQJRQZKHWKHUWKDWHPRWLRQZDVEHLQJIHOWDWDKLJKHU
RUORZHULQWHQVLW\7RPNLQVUHIHUUHGWREDVLFHPRWLRQVDVLQQDWHDIIHFWVZKHUHDIIHFWVWDQGVIRUWKH
VWULFWO\ ELRORJLFDO SRUWLRQ RI HPRWLRQ )URP WKLV SHUVSHFWLYH HLJKW EDVLF HPRWLRQV KDYH EHHQ
LGHQWLILHG (QMR\PHQW-R\ ,QWHUHVW([FLWHPHQW 6XUSULVH $QJHU5DJH 'LVJXVW 'LVWUHVV$QJXLVK
)HDU7HUURU DQG 6KDPH+XPLOLDWLRQ 6RPH RI WKHVH HPRWLRQV DUH SURSRVHG WR EH PDQDJHG RQO\ E\
WKH LQWHUQDO SURFHVVHV RI WKH LQGLYLGXDO ZLWK VRPH RWKHUV EHLQJ LQWULQVLFDOO\ UHODWHG WR VRFLDO
LQWHUDFWLRQV /|YKHLP SURSRVHV DQ H[WHQVLYH H[SODQDWLRQ RI WKH PDSSLQJ RI WKH HPRWLRQV WKDW KH
LQKHULWHGIURP7RPNLQVRQWRWKHVSDFHGHILQHGE\KLVPRGHORIQHXURWUDQVPLWWHUV

/RZ OHYHO LQIOXHQFH RI DIIHFWV RQ EUDLQ FHOOXODU RU QHXURQDO SURYLGHV IXQGDPHQWDO EDVH IRU RXU
DIIHFWLYH FRPSXWDWLRQ PRGHO 1RUDGUHQDOLQH VHURWRQLQ GRSDPLQH GR QRW KDYH GLUHFW PDSSLQJ RQ
FRPSXWDWLRQDOSURFHVVHV

7RWUDQVODWH/|YKHLPVPRGHORIQHXURWUDQVPLWWHUVLQWRWKHFRPSXWDWLRQDOSURFHVVHVRIDPDFKLQHZH
IRXQG$UELEDQG)HOORXVVUHILQHPHQWRIWKHUHODWLRQVKLSEHWZHHQGRSDPLQHDQGVHURWRQLQDQG
HPRWLRQVLQWKHPDPPDOLDQEUDLQYHU\XVHIXO


285&2*1,7,9(352326$/1(8&2*$5

,QWKLVVHFWLRQZHSUHVHQWRXUQRYHODSSURDFKWRDUWLILFLDOFRJQLWLYHDUFKLWHFWXUHVWKH1HXURPRGXODWLQJ
&RJQLWLYH $UFKLWHFWXUH 1(8&2*$5 7KH VXJJHVWHG LGHD LV WR VSHFLI\ D PDSSLQJ RI /|YKHLPV

29

 International Journal of Artificial Intelligence

PRQRDPLQH QHXURPRGXODWRUV RQWR FRPSXWDWLRQDO V\VWHP SDUDPHWHUV 2XU ZRUN VWDUWV IURP WKH
IROORZLQJFRQVLGHUDWLRQV
 (PRWLRQVDUHQDWXUDODQGQHFHVVDU\PRGXODWRUVUHLQIRUFHGDQGPRGHOHGDWWKHVDPHWLPHE\
VRFLDOH[WHUQDOIDFWRUVRUHYHQLQWHUQDOWKRXJKWV
 0RQRDPLQHQHXURPRGXODWRUVFDQEHH[SORLWHGDVSURSRVHGE\/|YKHLPWRVLJQLI\HPRWLRQDO
UHVSRQVHVDQGWKHLUWULJJHUVDQGPRGXODWLRQ
 6HYHUDOPRGXODUPRGHOVDUHDYDLODEOH*HQHYDHPRWLRQZKHHO6FKHUHU6DFKDULQ9HW
DO3OXWFKLNVSV\FKRHYROXWLRQDU\ZKHHORIHPRWLRQV3OXWFKLN3OXWFKLN
RU /|YKHLP &XEH RI HPRWLRQ /|YKHLP  DPRQJ RWKHUV $OO RI WKHP GHILQH IROORZLQJ
GLIIHUHQWVWUDWHJLHVWKHEDVLFHPRWLRQVDQGKRZG\QDPLFWUDQVLWLRQVDPRQJWKHPDUHFUHDWHG
6WHPPLQJ IURP WKHVH DVVXPSWLRQV RXU ZRUN LV IRFXVHG RQ PRGHOLQJ WKH G\QDPLFV RI /|YKHLPV
PRQRDPLQHVDVFRPSXWDWLRQDOSURFHVVHVRIPRGHUQFRPSXWHUV

:H SURSRVH WKH DSSURDFK WR FRUUHODWH WKH VXSSRVHG LQIOXHQFH RI /|YKHLPV WKUHH D[HV RI DIIHFWLYH
SURFHVVLQJ ZLWK FRPSXWDWLRQDO SURFHVVHV FRPSXWDWLRQDO SRZHU PHPRU\ GLVWULEXWLRQ OHDUQLQJ
VWRUDJH RU GHFLVLRQ PDNLQJ :H FRQVLGHU WKLV DSSURDFK DV WKH EDVLV RI DQ DIIHFWLYH FRPSXWDWLRQ
IUDPHZRUNDQGWKDWFDQEHH[SORLWHGLQVHYHUDODSSOLFDWLRQGRPDLQV

)URPHPRWLRQVWRFRPSXWLQJV\VWHP
,WKDVEHHQKLVWRULFDOO\WKRXJKWWKDW$,V\VWHPVFRXOGEHEXLOWRQWRSRI9RQ1HXPDQQDUFKLWHFWXUHV
SURYLGHGHQRXJKFRPSXWDWLRQDOSRZHUDQGPHPRU\UHVRXUFHVZHUHLQSODFHDQGDGHTXDWHDOJRULWKPV
ZHUH DYDLODEOH ,W ZDV WKRXJKW WR EH MXVW D PDWWHU RI WLPH PRQH\ DQG WHFKQRORJ\ $IWHU WKH LQLWLDO
H[FLWHPHQW LQ WKLV UHJDUG PDLQO\ LQ WKH V RI ODVW FHQWXU\ DQG QXPHURXV IDLOXUHV LQ UHDFKLQJ WKH
GHVLUHGUHVXOWVLWKDVEHHQODWHO\K\SRWKHVL]HGWKDWWKLVPD\QRWEHWKHFDVH

:KHWKHU WKHEUDLQ LV WKH XQLTXHVXEVWUDWH RI DOO FRJQLWLYH SURFHVVHV LQ KXPDQV DQG ZKHWKHU LW LV D
7XULQJPDFKLQHRUQRWDUHVWLOOWKHVXEMHFWRIGHEDWHDQGFRQWURYHUV\WRGD\'HVSLWHWKLVSDUDOOHOVFDQ
VWLOO EH EXLOW DQG WKH WKHRULHV FDQ EH XVHIXO DV LQVSLUDWLRQ IRU WKH GHYHORSPHQW RI VXEV\VWHPV IRU
VSHFLILF SXUSRVHV 7KLV LV WKH FDVH IRU H[DPSOH RI /|YKHLP FXEH RI HPRWLRQV ZKHUH VLPSOLILHG
DQDORJLHVFDQEHEXLOWEHWZHHQKLVPRGHORIDFWLRQRIGRSDPLQHVHURWRQLQDQGQRUDGUHQDOLQHOHYHOV
LQWRWKHEORRGVWUHDPDQGDOJRULWKPLFSURFHVVHV6LPXODWLRQRIHPRWLRQVZLOOOLNHO\SOD\DJURZLQJUROH
LQVHYHUDO,7DQG$,ILHOGQRWODVWURERWLFV7KHDELOLW\RIDV\VWHPWRIHHOHPRWLRQVOLNHIRUH[DPSOH
IHDULQWHUHVWRUMR\FDQHQDEOHFRPSOH[EHKDYLRUVWKDWDUHGHVLUDEOHLIRQHKDVWREXLOGIRUH[DPSOH
DQHWKLFDOURERW

0RUDOPDFKLQHVPD\HPHUJHIURPVXFKDUHVHDUFKVWUDQGLHPDFKLQHVZLWKLQEXLOWSHUFHSWLRQVRI
ULJKW DQG ZURQJ DQG XQDEOH WR KDUP KXPDQV OLNH LQ $VLPRYV 7KUHH /DZV RI 5RERWLFV 2I FRXUVH
PLOLWDU\ DSSOLFDWLRQV FDQ EH DOVR GHYHORSHG $ URERW VROGLHU IRU H[DPSOH PD\ HQRUPRXVO\ EHQHILW
IURPIHHOLQJVOLNHIHDULQRUGHUWRVXUYLYHDGYHUVLW\RQWKHEDWWOHILHOG$QRWKHUH[DPSOHLVSHWWKHUDS\

30

 International Journal of Artificial Intelligence

ZKLFKKDVVKRZQWREHEHQHILFLDOLQKHOSLQJSDWLHQWVWRUHFRYHUIURPRUFRSHZLWKKHDOWKSUREOHPVRU
PHQWDO GLVRUGHUV (PRWLRQDO PDFKLQHV FRXOG SURYLGH HQRUPRXV VXSSRUW LQ VXFK FDVHV DQG UHSODFH
DQLPDOV ZKHQ WKH\ KDYH WR EH GHSOR\HG LQ HQYLURQPHQWV ZKHUH DQ DQLPDO FRXOG DOVR EULQJ KDUP
)XUWKHUPRUH DXWRPDWHG GHVNVHUYLFH RU FDOO FHQWHUV PD\ EH DEOH WR FRSH ZLWK FXVWRPHUV QHHGV
PDLQWDLQLQJ WKH DSSHDUDQFH RI KXPDQ WRXFK DQG SURYLGLQJ FRQWLQXRXV VHUYLFH ZLWKRXW GLVFRQWLQXLW\
GXHWRWLUHGQHVV

$V WKH KXPDQ ERG\ DV D ZKROH UHDFWV WR FKDQJHV LQ WKH EORRGVWUHDP KRUPRQDO OHYHOV PDNLQJ
LPSRVVLEOHWRQHDWO\VHSDUDWHWKHIXQFWLRQVRIEUDLQIURPWKRVHRIWKHHQGRFULQHV\VWHPLQWKHVDPH
ZD\WKHPRGHOSUHVHQWHGLQWKLVSDSHURIIHUVDJURXQGIRUVLPXODWLRQVRIHPRWLRQVLQD9RQ1HXPDQQ
DUFKLWHFWXUHZLWKWKHFOHDUXQGHUVWDQGLQJWKDWPRVWOLNHO\QRWHYHU\EUDLQIXQFWLRQFDQEHUHSURGXFHG
LQWKHVDPHZD\7KLVPDSSLQJLVFOHDUO\QRWVXIILFLHQWSURSHUDOJRULWKPVQHHGWREHGHYHORSHGWR
UHSURGXFH KLJKHU IXQFWLRQV RQ WRS RI WKH KRUPRQDO UHDFWLYH PDFKLQHU\ 7KHVH DOJRULWKPV ZLOO
PRQLWRUIRULQVWDQFH&38DFWLYLW\DQGPHPRU\DOORFDWLRQWRWUDFNZKDWZRXOGEH/|YKHLPVKRUPRQDO
OHYHOVDQGUHDFWDFFRUGLQJO\

+LJKHUIXQFWLRQVRIWKHEUDLQOLNHFRQVFLRXVQHVVDQGSURDFWLYLW\WKHDELOLW\RIQRWUHDFWLQJLQWKHVKRUW
WHUP WR LPPHGLDWH SK\VLRORJLFDO UHVSRQVHV PD\ SRVVLEO\ EH VLPXODWHG RQ WRS RI EDVLF IXQFWLRQV
+RZHYHUWKLVVWLOOUHPDLQVDQXQSURYHQK\SRWKHVLVQHHGLQJDFFXUDWHLQYHVWLJDWLRQ

1(8UR&2X*$5&RJQLWLYH$UFKLWHFWXUH
1(8&2*$5 LV EDVHG RQ /|YKHLP  PRGHO RQ QHXURWUDQVPLWWHUV ,Q )LJXUH  WKH FXEH RI
HPRWLRQV H[WHQGV WKH RULJLQDO RQH WR PDS SDUDPHWHUV UHODWHG WR FRPSXWLQJ SRZHU VWRUDJH DQG
PHPRU\7KLVFDQEHFRQVLGHUHGDVRQHRIWKHPDMRUFRQWULEXWLRQRIWKLVSDSHU7KHSURSRVHGFXEH
FDQ EH DQ LQVWUXPHQW WR EULGJH FHUWDLQ QHXURVFLHQWLILF WKHRULHV DQG FRQFHSWV RI FRPSXWHU VFLHQFH
7KLVSDSHUDLPVDWLQLWLDWLQJDGLVFXVVLRQEHWZHHQSV\FKRG\QDPLFVFKRODUVDQGFRPSXWHUVFLHQWLVWV
DQGSRVVLEO\IRVWHULQJDSURGXFWLYHFRRSHUDWLRQDPRQJVSHFLDOLVWVRIWKHVHILHOGV

8OWLPDWHO\ ZH KRSH WR EXLOG IHHGEDFN DOJRULWKPV DEOH WR GLUHFW PDFKLQHV WRZDUGV HPRWLRQGULYHQ
EHKDYLRUVE\H[SORLWLQJWKHPDSSLQJWKDWZHKDYHSURSRVHGLQ)LJXUH$VSHFLILFSRLQWRQWKHWKUHH
GLPHQVLRQDOPDSUHSUHVHQWVDWHQGHQF\WRZDUGVDQHPRWLRQPDFKLQH2QFHVXFKDSRLQWKDVEHHQ
LGHQWLILHGGHFLVLRQPDNLQJFDQOHDGWKHPDFKLQHWRZDUGVDSSURSULDWHUHDFWLRQVLQRUGHUWRLQFUHDVH
GRPDLQVSHFLILFHIIHFWLYHQHVV

31

 International Journal of Artificial Intelligence


)LJXUH&RPSXWLQJV\VWHPSDUDPHWHUVDQGHPRWLRQVWRQHXURPRGXODWRUVPDSSLQJ


&RPSXWLQJV\VWHP
,QRUGHUWRDFKLHYHVXFKDJRDOZHPDS/|YKHLPVDIIHFWVDVIROORZV 'RSDPLQHLVDVVRFLDWHGZLWK
UHZDUG UHLQIRUFHPHQW PRWLYDWLRQ )URP D FRPSXWDWLRQDO V\VWHP SHUVSHFWLYH LW FDQ SOD\ D UROH LQ
SURFHVVLQJ RU FRPSXWLQJ SRZHU ORDG DQG VWRUDJH /|YKHLP DVVRFLDWHV VHURWRQLQ ZLWK VHOI
FRQILGHQFHLQQHUVWUHQJWKDQGVDWLVIDFWLRQ7KXVLWFDQEHUHODWHGWRPHPRU\DQGVWRUDJHKDQGOLQJ
/|YKHLP FXEH RI HPRWLRQV VWDWHV WKDW >@ QRUDGUHQDOLQH KDV EHHQ FRXSOHG WR WKH ILJKW RU IOLJKW
UHVSRQVH DQG WR VWUHVV DQG DQ[LHW\ DQG DSSHDUV WR UHSUHVHQW DQ D[LV RI DFWLYDWLRQ YLJLODQFH DQG
DWWHQWLRQ,WLVWKHUHIRUHDEOHWRUHSUHVHQWERWKFRPSXWLQJDQGVWRUDJHDFWLYLWLHV

'HFLVLRQ0DNLQJ
7KHGHFLVLRQPDNLQJLVLQIOXHQFHGE\DOHUWQHVVUHGXFLQJQXPEHURIRSWLRQVLQWKHREVHUYDWLRQDQG
SRVVLEO\ PDNLQJ WKH V\VWHP PDNH ULVNLHU FKRLFHV 7KLV FRXOG EH PRGHOHG DV FRPSXWLQJ SRZHU
UHGLVWULEXWLRQ DQG ZRUNLQJ PHPRU\ 'HFLVLRQ PDNLQJ LV SULQFLSDOO\ GRQH LQ WKH GHOLEHUDWLRQ DQG
OHDUQHG UHDFWLRQ OD\HUV RI WKH 0RGHO RI VL[ E\ 0LQVN\  5HODWHG SDUDPHWHUV DUH FRQILGHQFH
VDWLVIDFWLRQDQGULVNZKLFKDUHXVHGWRKLJKOLJKWVWRUHGDFWLRQVUHPHPEHUHG

LearningLVDSDUDPHWHULGHQWLI\LQJWUDLQLQJDQGQHZLQIRUPDWLRQDFTXLVLWLRQ$FFRUGLQJWR/|YKHLPLW
ZRXOGEHLPSDFWHGE\VHURWRQLQDQGGRSDPLQHGRSDPLQHSOD\VPDMRUUROHLQDFWLYDWLRQRISUHYLRXVO\
UHPHPEHUHG SDWWHUQV DQG VHURWRQLQ LQ SDWWHUQ JHQHUDWLRQ 7KH decision making JURXS UHSUHVHQWV
SDUDPHWHUV DQG FRORULQJ WDJJLQJ IODJJLQJ RI WKH WUDLQHG LQIRUPDWLRQ H[SORLWHG GXULQJ WKH GHFLVLRQ
PDNLQJSURFHVVHV&RQILGHQFHLPSDFWRQGHFLVLRQPDNLQJLVRQHRIWKHPRVWREYLRXVDQGLPSRUWDQW
HYHQ WKH VHOHFWLRQ RI WKH LQIRUPDWLRQ LV GRQH WDNLQJ LQWR DFFRXQW KRZ V\VWHP LV FRQILGHQW LQ WKLV
LQIRUPDWLRQ &RQILGHQFH ZRXOG EH KHDYLO\ DIIHFWHG E\ VHURWRQLQ 6DWLVIDFWLRQ LV RQH RI WKH PRVW
LPSRUWDQW IODJV LQYROYHG LQ UHZDUG V\VWHPV WR VHOHFW PRVW VDWLVIDFWRU\SOHDVXUDEOH DFWLRQV RU
DSSURDFKHV 6DWLVIDFWLRQ ZRXOG DOVR EH XQGHU WKH LQIOXHQFH RI VHURWRQLQ 7KH PRWLYDWLRQ KHDYLO\

32

 International Journal of Artificial Intelligence

LQIOXHQFHVVHOHFWLRQVRIIXUWKHUDFWLYLWLHVIRUWKHV\VWHPDQGZRXOGEHOLQNHGWRGRSDPLQH7KHQXPEHU
RIRSWLRQVWRSURFHVVVWUHVVIXOVLWXDWLRQDOHUWQHVVDQGLQFOLQDWLRQWRULVN\DFWLRQVZRXOGEHUHODWHGWR
QRUDGUHQDOLQHWKH\ZRXOGIRUFHWKHV\VWHPWRPDNHTXLFNGHFLVLRQVZLWKKLJKWHQGHQF\WRULVN  7KLV
PDSSLQJ FRXOG EH XVHG DV D KLJK OHYHO PRGHO RI HPRWLRQDO SURFHVVHV D EDVLF IUDPHZRUN IRU WKH
HPRWLRQHQDEOHGV\VWHPV


35(/,0,1$5<,03/(0(17$7,21$1'5(68/76

7ZRZD\VDUHSRVVLEOHWREXLOGFRQILGHQFHWKDWDVSHFLILFELRORJLFDOO\LQVSLUHGV\VWHPZRUNVRQRQH
VLGH ZH PD\ EXLOG D UDWLRQDOH WR H[SODLQ WKH EHQHILWV WKDW ZRXOG HPHUJH E\ WKH LPSOHPHQWDWLRQ RI
VXFK D V\VWHP DQG WKH FRQVHTXHQFHV IRU WKH LQYROYHG VWDNHKROGHUV IRU H[DPSOH FRQVWUXFWLQJ D
QDUUDWLYHH[SODLQLQJZK\LPSOHPHQWLQJHPRWLRQDOGULYHVLQURERWVZRXOGEHQHILWVRFLHW\DQGKRZWKH
PDUNHWZRXOGEHQHILWRXWRILW7KLVDSSURDFKLQVRPHFRQWH[WVPD\EHDSSURSULDWHEXWZHFRQVLGHULW
ZHDNIRUWKHVSHFLILFREMHFWLYHVRIWKLVSDSHUZKHUHZHZDQWWRLQGLFDWHKRZELRORJLFDOV\VWHPVZRUN
LQQDWXUHDQGKRZZHFDQUHSOLFDWHWKHPLQWRVLOLFRQ7KHUHIRUHWRYDOLGDWHWKHV\VWHPKHUHZHVKRZ
KRZ LW ILWV FHUWDLQ PRGHOV RI QDWXUDO EHKDYLRU DQG ZH EXLOG FRQILGHQFH LQ WKH UHDGHU WKDW QDWXUDO
V\VWHPV LQ VRPH VHQVH ZRUN LQ VLPLODU ZD\V 6WDUWLQJ IURP WKLV GHPRQVWUDWLRQ ZH HPSKDVL]H WKH
EHQHILWVIURPWKHFRPSXWDWLRQDOSHUVSHFWLYH$OOWKLVKDSSHQVYLDVLPXODWLRQ



)LJXUH6LPSOLILHGGRSDPLQHSDWKZD\VFKHPDWLFDOO\GHSLFWHG$EEUHYLDWLRQV*3HJOREXVSDOOLGXV
H[WHUQDO*3LJOREXVSDOOLGXVLQWHUQDO671VXEWKDODPLFQXFOHXV61FVXEVWDQWLDQLJUDFRPSDFWD
61UVXEVWDQWLDQLJUDUHWLFXODWD

33

 International Journal of Artificial Intelligence

,QRUGHUWREXLOGVXFKDVLPXODWLRQDQGWHVWWKH1(8&2*$5DUFKLWHFWXUHZHFRQVLGHUWKHnigrostriatal
dopamine SDWKZD\7KHFKRLFHRIWKLVSDWKZD\LVPRWLYDWHGE\WKHODUJHERG\RIOLWHUDWXUHDYDLODEOH
DERXWLWHJ$OELQHWDO,Q)LJXUHWZRSDWKZD\VDUHVKRZQLQWHJUDWHGDQGEDODQFHGSDUWVRI
WKHQLJURVWULDWDOSDWKZD\

 'LUHFW SDWKZD\  &RUWH[ VWLPXODWHV  6WULDWXP LQKLELWV  61U*3L FRPSOH[ OHVV

LQKLELWLRQRIWKDODPXV7KDODPXVVWLPXODWHV&RUWH[VWLPXODWHV0XVFOHVHWF

 ,QGLUHFWSDWKZD\&RUWH[VWLPXODWHV6WULDWXPLQKLELWV*3HOHVVLQKLELWLRQRI671

671VWLPXODWHV61U*3LFRPSOH[LQKLELWV7KDODPXVLVVWLPXODWLQJOHVV&RUWH[LV

VWLPXODWLQJOHVV0XVFOHVHWF


)LJXUH,PSOHPHQWHGGRSDPLQHSDWKZD\ZLWKIHHGEDFNORRSV$EEUHYLDWLRQV*3HJOREXVSDOOLGXV
H[WHUQDO*3LJOREXVSDOOLGXVLQWHUQDO671VXEWKDODPLFQXFOHXV61FVXEVWDQWLDQLJUDFRPSDFWD
61UVXEVWDQWLDQLJUDUHWLFXODWD5HGDUURZVDUHXVHGWRLQGLFDWHWKHH[FLWDWRU\V\QDSVHEOXHIRU
LQKLELWRU\DQGJUHHQIRUQHXURPRGXODWRU\RQHV

7KHDQWDJRQLVWLFIXQFWLRQVRIWKHGLUHFWDQGLQGLUHFWSDWKZD\VDUHPRGXODWHGE\WKHVXEVWDQWLDQLJUD
SDUVFRPSDFWD61FZKLFKSURGXFHVGRSDPLQH$VEHWWHUGHVFULEHGLQ)LJXUHLQWKHSUHVHQFHRI
GRSDPLQH 'UHFHSWRUV LQ WKH EDVDO JDQJOLD VWLPXODWH WKH *$%$HUJLF QHXURQV IDYRULQJ WKH GLUHFW

34

 International Journal of Artificial Intelligence

SDWKZD\ DQG WKXV LQFUHDVLQJ PRYHPHQW 7KH *$%$HUJLF QHXURQV RI WKH LQGLUHFW SDWKZD\ DUH
VWLPXODWHG E\ H[FLWDWRU\ QHXURWUDQVPLWWHUV DFHW\OFKROLQH DQG JOXWDPDWH 7KLV VHWV RII WKH LQGLUHFW
SDWKZD\ WKDW XOWLPDWHO\ UHVXOWV LQ LQKLELWLRQ RI XSSHU PRWRU QHXURQV DQG OHVV PRYHPHQW ,Q WKH
SUHVHQFH RI GRSDPLQH 'UHFHSWRUV LQ WKH EDVDO JDQJOLD LQKLELW WKHVH *$%$HUJLF QHXURQV ZKLFK
UHGXFHVWKHLQGLUHFWSDWKZD\VLQKLELWRU\HIIHFW'RSDPLQHWKHUHIRUHLQFUHDVHVWKHH[FLWDWRU\HIIHFWRI
WKH GLUHFW SDWKZD\ FDXVLQJ PRYHPHQW DQG UHGXFHV WKH LQKLELWRU\ HIIHFW RI WKH LQGLUHFW SDWKZD\
SUHYHQWLQJIXOOLQKLELWLRQRIPRYHPHQW

7KHQLJURVWULDWDOSDWKZD\KDVEHHQPRGHOHGYLDVSLNLQJQHXUDOQHWZRUNEXLOWZLWK1(67IUDPHZRUN
1(67 ZDV FKRVHQ EHFDXVH RI LWV ZHOOHVWDEOLVKHG VSLNLQJ 11 IUDPHZRUN VXSSRUWLQJ GRSDPLQH
YROXPHWUDQVPLVVLRQRXWRIWKHER[7KLVDVSHFWFRQWULEXWHGWRWKHGHFLVLRQRIFKRRVLQJGRSDPLQHDV
WKHILUVWD[LVDQGRSWLRQWRYDOLGDWHRXUPRGHO

7KLV SDWKZD\ KDV EHHQ LPSOHPHQWHG LQ 1(67 IUDPHZRUN ZLWKLQ 3\WKRQ VFULSWLQJ :H KDYH FUHDWHG
WKH VLPSOLILHG PRGHO RI PRWRU FRUWH[ WKDODPXV VWULDWXP JOREXV SDOOLGXV H[WHUQDO JOREXV SDOOLGXV
LQWHUQD VXEWKDODPLF QXFOHXV VXEVWDQWLD QLJUD FRPSDFWD VXEVWDQWLD QLJUD UHWLFXODWD DFFRUGLQJ WR
)LJXUHVFKHPDDQGUXQVHYHUDOH[SHULPHQWVZLWKGLIIHUHQWQXPEHUVRIQHXURQV8QIRUWXQDWHO\GXH
WR ODFN RI FRPSXWDWLRQDO SRZHU ZH FRXOG QRW UXQ H[SHULPHQWV ZLWK UHDO QXPEHUV RI QHXURQV RI IRU
H[DPSOHUDWEUDLQHYHQWKHQHZERUQUDWEUDLQ

)LJXUH'RSDPLQHQHXURPRGXODWLRQUHVXOWVRQPRWRUFRUWH[OHIWDQGWKDODPXVULJKWZLWK

QHXURQV1HXURPRGXODWLRQLVSHUIRUPHGDWPV

:H KDYH UXQ WKH YDOLGDWLRQ RI WKH QLJURVWULDWDO SDWKZD\ RQ RQH D[LV GRSDPLQH RI WKH &XEH RI
(PRWLRQV GHSLFWHG LQ )LJXUH  DV LWV LQWHUDFWLRQ PHFKDQLVP LV EHWWHU GHVFULEHG LQ OLWHUDWXUH WKDQ
VHURWRQLQ DQG QRUDGUHQDOLQH ,QGHHG DFFRUGLQJ WR WKH SURSRVHG IUDPHZRUN KLJK GRSDPLQH
FRUUHVSRQGV WR DIIHFW WKH IHDU DQG WR LQFUHDVHG FRPSXWDWLRQDO SRZHU FRQVXPSWLRQ HDV\ WR UHWULHYH
LQGH[LQJLQVWRUDJHGXHWROHDUQLQJDQGRUPRWLYDWLRQWRUHXVHUHODWHGDFWLYLWLHV



KWWSZZZQHVWLQLWLDWLYHRUJ

KWWSVJLWKXEFRPPD[WDODQRYEOREPDVWHUDIIHFWLYHFRPSXWLQJUHDOLVWLFBQQVPG

35

 International Journal of Artificial Intelligence

)LJXUH'RSDPLQHQHXURPRGXODWLRQUHVXOWVRQPRWRUFRUWH[OHIWDQGWKDODPXVULJKWZLWK

QHXURQV1HXURPRGXODWLRQLVSHUIRUPHGDWPV

7KHUHVXOWVVKRZQLQ)LJXUHVDQGGHPRQVWUDWHRYHUDOOFRUUHFWQHVVRIWKHPRGHODQGLQGLFDWLRQRI
ULVH RI QHXURQDO DFWLYLW\ LQ WKDODPXV DQG PRWRU FRUWH[ ULJKW DIWHU WKH  PV ZKHQ WKH
QHXURPRGXODWLRQKDVEHHQSHUIRUPHG7KLVFRUUHVSRQGVWRULVHRIRYHUDOOFRPSXWDWLRQDODFWLYLW\IURP
SURSRVHG PDSSLQJ DQG WKXV LQFUHDVHG LQIOXHQFH RQ V\QDSVHV WKXV LQFUHDVHG VHQVLWLYLW\ RI XVHG
V\QDSVHV ,QFUHDVHG VHQVLWLYLW\ RI WKH V\QDSVHV LQIOXHQFHV WKH OHDUQLQJ DQG HDVHV LQIRUPDWLRQ
UHWULHYDODQGVWRUDJHDQGIXUWKHUPRWLYDWLRQWRXVHVNLOOVKLJKOLJKWHGZLWKGRSDPLQHQHXURPRGXODWLRQ


',6&866,21$1'&21&/86,216

7RXQGHUVWDQGWKHFXUUHQWVFLHQWLILFVWDWHRIDIIDLUVH[LVWLQJPRGHOVDQGLPSOHPHQWDWLRQVDQGWRILQG
D VXLWDEOH EDVH IRU RXU LPSOHPHQWDWLRQ ZH UDQ D FRPSDUDWLYH DQDO\VLV :H ZLOO QRW MXVW LQFOXGH
HPRWLRQRULHQWHGDUFKLWHFWXUHVLQVWHDGZHDWWHPSWWRJHWDZLGHYLHZRQWKHFXUUHQWVLWXDWLRQLQWKH
DUWLILFLDO FRJQLWLRQ GRPDLQ )RU RXU SXUSRVHV UDWKHU WKDQ SHUIRUPLQJ D WRR JHQHUDO FRPSDULVRQ VHH
WKH JRRG SDSHU E\ /LQ HW DO  IRU PRUH JHQHUDO UHDGLQJ RQ WKH WRSLF ZH KDYH VHOHFWHG 
RXWVWDQGLQJDQGUHSUHVHQWDWLYHHPRWLRQDORUHPRWLYHFRJQLWLYHDUFKLWHFWXUHVZKRVHQDWXUHJRHVIURP
WKHRUHWLFDOOHYHOWRFRPSXWDWLRQDORUURERWLFLPSOHPHQWDWLRQ

$ .,60(7 .,60(7 ZDV WKH ILUVW VXFFHVV LQWR WKH LPSOHPHQWDWLRQ RI HPRWLRQDO DUFKLWHFWXUHV
LQWR VRFLDEOH URERWV DV WKH UHVXOW RI LQQRYDWLYH LGHDV IURP %UHD]HDO DQG %URRNV 
.,60(7 FRPPXQLFDWHV LWV HPRWLYH VWDWH DQG RWKHU VRFLDO FXHV WR D KXPDQ WKURXJK IDFLDO
H[SUHVVLRQVERG\SRVWXUHJD]HGLUHFWLRQDQGTXDOLW\RIYRLFHEXWDORRNWRWKHDUFKLWHFWXUH
WR WKLV URERW S  RIIHUV WKUHH GLIIHUHQW DOWKRXJK FRPSOHPHQWDU\ DUHDV WKH FRJQLWLYH
V\VWHP WKH HPRWLYH V\VWHP DQG WKH PRWRU V\VWHP ,W LV QRUPDO EHFDXVH WKH\ IROORZHG
QHWZRUNDSSURDFKRILQWHUDFWLQJHOHPHQWVHJVXEVXPSWLRQDUFKLWHFWXUHQHXUDOQHWZRUNVRU

36

 International Journal of Artificial Intelligence

DJHQW DUFKLWHFWXUHV 7KH\ LPSOHPHQWHG .,60(7 DV DQ DJHQW DUFKLWHFWXUH ZKHUH HDFK
FRPSXWDWLRQDOHOHPHQWZDVFRQFHSWXDOL]HGDVDVSHFLDOLVW
% +&RJ$II FUHDWHG E\ 6ORPDQ  WKLV DUFKLWHFWXUH ZDV GHVLJQHG DV D WKUHHOD\HU
DUFKLWHFWXUH SHUFHSWLRQ HQYLURQPHQW DQG DFWLRQ LQ ZKLFK KDSSHQV UHDFWLYH PHWD
PDQDJHPHQWDQGGHOLEHUDWLYHSURFHVVHV'HVSLWHRIDQLQWHUHVWIRUHPRWLRQVKHFRQVLGHUHG
WKHP DQG QRQ JHQXLQHO\ QHFHVVDU\ DWWDFNLQJ DIWHU 'DPDVLRV LGHD RI HPRWLRQV DUH D
UHTXLUHPHQWIRULQWHOOLJHQFHDIDOODFLRXVLGHDWKDWKDVEHFRPHDPHPH+HUHHPRWLRQVDUHD
PHUHDQGSDUWLDOWULJJHUHU
& :$6$%, %HFNHU$VDQR DQG :DFKVPXWK  RQH RI WKH EHVW PRGHO WR VLPXODWH DIIHFWLYH
DJHQWV LW GLIIHUHQWLDWHV EHWZHHQ SULPDU\ DQG VHFRQGDU\ HPRWLRQV JURXQGHG LQ SULPDU\ LQ
ERUQHPRWLRQVFUHDWLQJELDVHGHQWLWLHVWRZDUGVRSWLPLVPDQGSHVVLPLVPDVZHOODVPDSSLQJ
HPRWLRQVDVEHOLHIV
' 520$1 URERW +LUWK HW DO  IROORZLQJ .,60(7V LGHDV 520$1 LQWHJUDWHV  EDVLF
PRGXOHV VHQVRUV GULYHV HPRWLRQV DFWLRQV DQG DFWXDWRUV ,Q WKDW DUFKLWHFWXUH WKH GULYHV
GHWHUPLQHZKDWWRGRDQGWKHHPRWLRQVGHWHUPLQHKRZWRGR7KHUHIRUHHPRWLRQVDUHMXVW
PHGLDWRUV EHWZHHQ VRPH SURFHVVHV %XW DV ZKROH WKLV HPRWLRQDO DUFKLWHFWXUH FRQVLVWV RI 
PDLQPRGXOHVWKHHPRWLRQVWKHGULYHVDQGWKHDFWLRQV'ULYHVVSHFLI\WKHQHHGVDQGJOREDO
JRDOVHPRWLRQVGHVFULEHWKHLQWHUQDOVWDWHRIWKHURERWDQGDFWLRQVUHDOL]HWKHLQWHUDFWLRQZLWK
WKHURERWHQYLURQPHQW

:HOO FRPPHQW QRZ RQ WKHVH  DSSURDFKHV DQG ZH ZLOO LQWURGXFH DW WKH HQG RXU LPSURYHPHQWV LQ
UHODWLRQ WR DOO RI WKHP D .,60(7 GLG QRW LQWHJUDWH PRWRU FRJQLWLYH DQG HPRWLRQDO DVSHFWV
FRQVLGHULQJWKHPDVLQGHSHQGHQWHOHPHQWVRIDPDFKLQHZKLFKWULHVWRHPXODWHDKXPDQEHLQJEXWLQ
UHDOQDWXUDOHQYLURQPHQWVDOOWKHVHWKUHHHOHPHQWVDUHLQWHJUDWHGE+&RJ$IILVDPHUHVHPDQWLF
VWUXFWXUH LQ ZKLFK DUELWUDU\ YDOXHV DQG UXOHV DUH FUHDWHG LQWURGXFLQJ D VHW RI YDULDEOHV FDOOHG
HPRWLRQDO EXW WKDW DUH MXVW ZHLJKWHG VHOHFWRUV F :$6$%, LV YHU\ LQWHUHVWLQJ EXW DJDLQ FRQVLGHUV
HPRWLRQVDVLQWHUQDOEHOLHIVWKDWPDQDJHVRPHEHKDYLRUDODQGSURFHVVLQJDFWLRQVGILQDOO\520$1
LVDQXSGDWHGURERWLFYHUVLRQRID+5,VNLOOHGURERWWKDWKDVDJDLQDVHSDUDWHHPRWLRQDOPRGXOH

2QWKHRWKHUKDQG1(8&2*$5DUFKLWHFWXUHLVQRWRQO\PRUHELRORJLFDOO\LQVSLUHGEXWDOVRLQWHJUDWHV
HPRWLRQDO PHFKDQLVPV LQWR WKH ZKROH FKDLQ RI LQIRUPDWLRQ SURFHVVLQJ DQG EHKDYLRUDO DFWLRQ RI DQ
HQWLW\ 1(8&2*$5 LV DEOH WR SURYLGH DQ\ $, V\VWHP ZLWK D IXOO\ UHDOLVWLF SDUDQHXURPRGXODWLQJ
DUFKLWHFWXUHWKDWGLVWULEXWHVHPRWLRQDOPHFKDQLVPVDOOWKURXJKRXWWKHV\VWHPDWGHOLEHUDWLYHOD\HUVDW
GDWDFDSWXUHVHFWLRQVDWG\QDPLFDOPXOWLPRGDOLQWHJUDWLRQSURFHVVHVDQGVRRQ:HGRQRWVXJJHVW
DQ HPRWLRQDO PRGXOH WKDW SURYLGHV VRPH UHDOLVWLF RU HPRWLRQDOOLNH IODYRU EXW D WUXH FRPSXWDWLRQDO
ZD\ WR LPSOHPHQW D ELRLQVSLUHG ERWWRPXS DUFKLWHFWXUH 7KLV DUFKLWHFWXUH IORZV IROORZLQJ
QHXURPRGXODWLQJ PHFKDQLVPV WKDW DUH FRQQHFWHG WR DOO PDQDJHPHQW SURFHVVHV DXWRPDWHG RU
GHOLEHUDWLYH7KLVLVWKHJUHDWGLIIHUHQFHDQGLPSURYHPHQWRI1(8&2*$5DUFKLWHFWXUH


37

 International Journal of Artificial Intelligence

$FFRUGLQJ WR WKHRULHV DQG ODWHVW UHVXOWV RI D WUDQVGLVFLSOLQDU\ YLHZ RQ FRJQLWLRQ WKH
HPERGLPHQWRIFRJQLWLYHSURFHVVHVFDQEHFRPSOHWHO\DIILUPHG,WVDPDWWHURIIDFWWKDWPRVWDEVWUDFW
SURFHVVSHUIRUPHGE\WKHEUDLQDUHGLUHFWO\UHODWHGWRVHQVRULPRWRUSURFHVVHV$WWKHVDPHWLPHWKH
PLQGERGLO\PDQDJHPHQWLVUHODWHGWRHPRWLRQDOVLJQDOVWKDWFDSWXUHSURFHVVDQGFODVVLI\GDWDLQSXWV
H[WHUQDORULQWHUQDO\RXQHHGWRUHPHPEHUWKHPXOWLPRGDOLW\RIHPRWLRQVDQGDOORZUHVSRQVHV7KH
IXQGDPHQWDOUROHRIHPRWLRQVLQWRFRJQLWLYHSURFHVVHVLVWKXVDPDWWHURIIDFWIRUDQ\UHVHDUFKHULQ
WKHILHOG$WWKHVDPHWLPHZHYHVHHQWKDWELRLQVSLUHGDSSURDFKHVWRDUWLILFLDOFRJQLWLYHV\VWHPVOLNH
DIIHFWLYH FRPSXWLQJ HPRWLRQDO FRJQLWLYH DUFKLWHFWXUHV RU QHXUDO QHWZRUNV GR QRW SURYLGH D UHDOLVWLF
DSSURDFKWRWKHHPRWLRQDOG\QDPLFGDWDPDQDJHPHQWSURFHVVHV$WWKHVDPHWLPHZHQHHGWRDGPLW
WKHXWLOLW\DQGJRRGUHVXOWVLQVHYHUDORIWKHVHDSSOLFDWLRQILHOGVUHO\LQJRQWKHROGIDVKLRQHGLGHDVRI
&ODVVLF$,RU*2)$,

2XUDSSURDFKLVLQQRYDWLYHIRUVHYHUDOUHDVRQV
 :H GR QRW QHHG WR UXQ D ZKROH EUDLQ HPXODWLRQ LQ RUGHU WR DFKLHYH D IXQFWLRQDO DUWLILFLDO
FRJQLWLYHV\VWHPDQGWKLVDOORZXVWRDYRLGWKHVWLOOH[LVWLQJDQGGHHSXQFHUWDLQWLHVVWLOOWREH
FODULILHG EHWZHHQ EUDLQ VWUXFWXUH DQG SHUIRUPDQFH DW WKLV KLVWRULFDO PRPHQW EUDLQ
UHVHDUFKHUVDUHDWWKHVDPHSRLQWWKDWJHQHWLFELRORJLVWVVWDUWLQJWRGHFRGHWKHJHQRPHZLWK
DORWRILQIRUPDWLRQLQIDFWWRRPXFKEXWZLWKRXWPHDQLQJ
 :H DUH QRW TXDOLWDWLYHO\ LPLWDWLQJ WKH VXUIDFH RI WKH HPRWLRQDO V\QWD[ RI KXPDQ EUDLQV DQG
ERGLHV EXW LQVWHDG RI LW ZH DUH VXJJHVWLQJ D QHZ DSSURDFK WKDW LV URRWHG LQWR WKH
QHXURPRGXODWRUVV\QWD[WKHNH\SLHFHRIEUDLQHPRWLRQDODQGFRJQLWLYHSURFHVVHV
 7KLVIXQFWLRQDODSSURDFKDOORZVWKHJHQHUDWLRQRIPRUHG\QDPLFV\VWHPVWKDWDGGVFRJQLWLYH
SRZHU EH\RQG WKH EDVLFV RI FODVVLF ZHLJKWV WKUHVKROGV RU YDOHQFHV PDNLQJ SRVVLEOH D
FRJQLWLYH DUFKLWHFWXUH WKDW JHQHUDWHV WKH VHYHUDO IDFHV RI HPRWLRQDO LQWHUDFWLRQV VXFK DV
PRRGVDWWLWXGHVHPRWLRQDOPHPRULHVRUDIIHFWLYHVWDWHV

1(8&2*$5PDNHVSRVVLEOHWKHLPSOHPHQWDWLRQRIHPRWLRQDOHOHPHQWVLQWRWKHZKROHV\VWHPDFWLQJ
DV WKH EDVLF SLHFH RI UHJXODWLRQ DQG QRW DV DQ DWWDFKHG PRGXOH ZLWK HPRWLRQDO FRQWHQWV 7KLV
DUFKLWHFWXUHDOORZVWRDSSO\LWVPRRWKO\IURPWKHVFUDWFKWKHDUWLILFLDOQHXURQRUXQLWRISURFHVVLQJWR
WKH VXSHULRU OD\HUV RI VSHFLDOL]HG FRJQLWLYH SURFHVVHV OLNH PHPRULHV SURFHVVRUV ,Q WKLV VHQVH
1(8&2*$5LVDZKROHDUWLILFLDOFRJQLWLYHV\VWHPWKDWHPXODWHVWKHEDVLFUXQQLQJRIERGLO\HPRWLRQDO
V\VWHPVIROORZLQJDPXOWLOD\HUHGRPRGXODUDUFKLWHFWXUHWKDWLVSXWWRJHWKHUDQGUXOHGE\WKHVDPH
HPRWLRQDOF\EHUQHXURPRGXODWRUV

:H DUH FUHDWLQJ D QHZ (PERGLHG $, ZLWKRXW UHDO LQVWDQWLDWHG SK\VLFDOLW\ :H WDNH DOVR LQWR
FRQVLGHUDWLRQWKHIDFWWKDWDQ\OLYLQJV\VWHPLVJRDORULHQWHGDQGWKDWLWPXVWEHLPSOHPHQWHGLQWRRXU
V\VWHP EHIRUH WR DFKLHYH DQ\ UHVXOW 7KH SRLQW LV WKDW ZH FDQ GHFLGH DUWLILFLDOO\ WKH JRDOV DQG WKHQ
DSSO\ WKH UHDO PHFKDQLVPV RI FRJQLWLYH SURFHVVLQJ 1(8&2*$5 ZLOO EH V\PELRWLFDOO\ XVHIXO IRU $,
UHVHDUFKHUV DV ZHOO DV IRU QHXURORJLVWV WKDW ZDQW WR FKHFN VRPH RI WKHLU K\SRWKHVLV DERXW PDLQ
FKHPLFDOG\QDPLFVLQGDWDSURFHVVLQJ,WLVDQHZWHFKQRORJ\IRUDQHZKRUL]RQRILQWHOOLJHQWV\VWHPV

38

 International Journal of Artificial Intelligence

$&.12:/('*0(176

3DUWLDO VXSSRUW IRU WKLV UHVHDUFK ZDV UHFHLYHG E\ WKH 6SDQLVK *RYHUQPHQWV '*,&<7 UHVHDUFK
SURMHFW)),,QQRYDWLRQLQVFLHQWLILFSUDFWLFHFRJQLWLYHDSSURDFKHVDQGWKHLUSKLORVRSKLFDO
FRQVHTXHQFHV DQG E\ WKH 5XVVLDQ 0LQLVWU\ RI HGXFDWLRQ DQG VFLHQFH DJUHHPHQW 
,'5)0(),;


5()(5(1&(6

$OELQ 5 / DQG <RXQJ $ % DQG 3HQQH\ - %  The functional anatomy of basal ganglia
disorders7UHQGVLQ1HXURVFLHQFHV
$UELE0DQG)HOORXV-Emotions: from Brain to Robot,7UHQGVLQ&RJQLWLYH6FLHQFHV

%DUVDORX/:Grounded Cognition,$QQ5HY3V\FKRORJ\
%HFNHU$VDQR & DQG :DFKVPXWK ,  Affective computing with primary and secondary
emotions in a virtual human$XWRQRPRXV$JHQWVDQG0XOWL$JHQW6\VWHPV
%HQHGLFW 5  The Chrysanthemum and the Sword: Patterns of Japanese Culture, +RXJKWRQ
0LIIOLQ
%UHD]HDO&Emotion and Sociable Humanoid Robots(OVHYLHU6FLHQFH
%UHD]HDO&DQG%URRNV5Robot emotions. a Functional Perspective. ,Q)HOORXV-0DQG
$UELE 0 $  Who Needs Emotions? The Brain Meets the Robot 2[IRUG 8QLYHUVLW\ 3UHVV
2[IRUG
%ULGJHV0'LVWHIDQR60D]]DUD00LQOHEDHY07DODQRY0DQG9DOOYHUG~-Towards
Anthropo-inspired Computational Systems: the P3 Model WK ,QWHUQDWLRQDO .(6 &RQIHUHQFH RQ
$JHQWVDQG0XOWL$JHQWV6\VWHPV7HFKQRORJLHVDQG$SSOLFDWLRQV.(6$067$
%URRNV5$Intelligence Without Representation$UWLILFLDO,QWHOOLJHQFH
%URRNV5$DQG/\QQ$6Building Brains for Bodies,$XWRQ5RERWV)HEUXDU\
&ODUN $  Natural-Born Cyborgs. Minds, Technologies, and the Future of Human Intelligence,
2[IRUG8QLYHUVLW\3UHVV
'DPDVLR $  The Feeling of What Happens: Body and Emotion in the Making of
Consciousness, (GLWHGE\+DUFRXUW%UDFH1HZ<RUN
'H/DQFH\ &  Passionate Engines: What Emotions Reveal About Mind and Artificial
Intelligence, 2[IRUG8QLYHUVLW\3UHVV
+LUWK - 6FKPLW] 1 DQG %HUQV .  Emotional Architecture for the Humanoid Robot Head
ROMAN,(((,QWHUQDWLRQDO&RQIHUHQFHRQ5RERWLFVDQG$XWRPDWLRQ
.HOO\9&A Primer of Affect Psychology, KWWSZZZWRPNLQVRUJ
/LQ - 6SUDUDJHQ 0 DQG =\GD 0  Computational Models of Emotion and Cognition
$GYDQFHVLQ&RJQLWLYH6\VWHPV
/OLQiV55I of the Vortex. From Neurons to Self. Cambridge0,73UHVV
/|YKHLP +  A New Three-Dimensional Model for Emotions and Monoamine
Neurotransmitters0HGLFDO+\SRWKHVHV
0LQVN\ 0  The Emotion Machine: Commonsense Thinking, Artificial Intelligence, and the
Future of the Human Mind,6LPRQ 6FKXVWHU
1DJDPDFKL 0  Kansei Engineering, 2 Volume Set: Kansei/Affective Engineering ,QGXVWULDO
,QQRYDWLRQ6HULHV&5&3UHVV
2UWRQ\ $ &ORUH * / DQG &ROOLQV $  The Cognitive Structure of Emotions &DPEULGJH
8QLYHUVLW\3UHVV

39

 International Journal of Artificial Intelligence

3DUHQW $ HW DO  Organization of the basal ganglia: the importance of axonal collateralization
7UHQGV1HXURVFL66
3LFDUG5:Affective Computing,0DVVDFKXVHWWV,QVWLWXWHRI7HFKQRORJ\
3OXWFKLN 5  Emotion: Theory, research, and experience: Vol. 1. Theories of emotion  1HZ
<RUN$FDGHPLF3UHVV
3OXWFKLN5The nature of emotions$PHULFDQ6FLHQWLVW
3OXWFKLN 5  Emotions and Life: Perspectives from Psychology, Biology, and Evolution
:DVKLQJWRQ'&$PHULFDQ3V\FKRORJLFDO$VVRFLDWLRQ
5DPDFKDQGUDQ96A Brief Tour of Human Consciousness 3L3UHVV3HDUVRQ
5L]]RODWWL*DQG&UDLJKHUR/The Mirror-Neuron System,$QQXDO5HYLHZRI1HXURVFLHQFH
6DFKDULQ96FKOHJHO.DQG6FKHUHU.5 Geneva Emotion Wheel rating study5HSRUW
*HQHYD6ZLW]HUODQG8QLYHUVLW\RI*HQHYD6ZLVV&HQWHUIRU$IIHFWLYH6FLHQFHV
6FKHUHU . 5  What are emotions? And how can they be measured? 6RFLDO 6FLHQFH
,QIRUPDWLRQ 
6ORPDQ$Beyond Shallow Models of Emotion. &RJQLWLYH3URFHVVLQJ
6ORPDQ$DQG&KULVOH\5Virtual Machines and Consciousness.-RXUQDORI&RQVFLRXVQHVV
6WXGLHV
7DODQRY 0 9DOOYHUGX - 'LVWHIDQR 6 0D]]DUD 0 DQG 'HOKLEDEX 5  Neuromodulating
WK
Cognitive Architecture: Towards Biomimetic Emotional AI   ,((( ,QWHUQDWLRQDO &RQIHUHQFH RQ
$GYDQFHG,QIRUPDWLRQ1HWZRUNLQJDQG$SSOLFDWLRQV$,1$SS
7RPNLQV6Affect Imagery Consciousness Volume I the Positive Affects1HZ<RUN6SULQJHU
3XEOLVKLQJ&RPSDQ\

40