BAA Bt Al SALA GIBB) =A! 2003, Vol 13, No. 1, pp. 45-50 SEA 2243 DANNS Bet Seaso ae eA Analyzing the Acoustic Elements and Emotion Recognition from Speech Signal Based on DRNN AWS. wate. ae” Kwee-Bo Sim” - Chang-Hyun Park” - Young-Hoon Joo” « Betcha Mx 7) Bape ~ BAGS Aas Bee 2e AH IOe Kol Me Meo] AST OE PIS o|Fa sla, ee ee Mo] See as qs qos | UAVS Ae] AeAolehs ello] sPYs|aL ah, Liew! Hye) Fae Mol PG MES ab she S29) AVS MAO] VRS BHAA ALaole|s| Aves} ALLA APE ele a, Maw ofa, PHou Wes, VS So AAS PHI GOR MOAT Sq MMA Te US T+ ASS weld. Fah 44 GA OE BAS Solel, GIA RAs God Poet de eo) AGT as, e ees AS 81 BAL oleh AMOS] MAF FALE SOLE sha]S ASS KL MA] MTS FEA) 2% DRNN To} ab ABSTRACT Recently, robots technique has been developed remarkably. Emotion recognition is necessary to make an intimate robot. This paper shows the simulator and simulation result which recognize or classify’ emotions by learning pitch pattem. Also, because the pitch is not suiicient for recognizing emotion, we added acoustic elements. For that reason, Wwe analyze the relation between emotion and acoustic elements. The simulator is composed of the DRNN(Dynamic Recurrent Neural Network), Feature extraction. DRNN is a learning algorithm for pitch patter, Key Words : 52)(Pitch), 88% s}/+(Formant Frequency), 2, DRNN LAM ste o 2ato] Ala] dis alae, ae sete EF MS pole}. rela, F : ng HAINES) 8) AER ACHLS Bate Cy lo] Foe dds] aes oy He SSS Qe =e UE ABS So} BF US selec. gg age ae Hof] Uap vay Baleel wae Aa ales F WMA SORA] 14, -F 7h] We Bele) anh. Bo] qe Jee sad s de lend= vase, B SEIS FU S S8eavag Mag arash JQ 2 Aes] SAYS Bolts Auowe wy th 719 SLL WS FA SVos oBssich wale eeht vaiele telah ahh, dad galelele gal FIALE 3) EMS HSH AOL HI OA | See le aida) ween WP} sa, Me ap gy M77 Wel FPS le, aT lee BS aeAS el] Hades Sel azo} HApsiAoF ach, ola) 9) WAS FA SILL SA LS. Chene M3)9) S Baad Ao] q39] lfeleh elie] Q3g eyapy qe AS Hash oe AB Sk o)eheleh ae Set Su ge yee ddaq see sede a BAUS He Ae BA UD, BIAS Ha aH lth, BF] 2 Ase] eels Ae aatsith. Joy AS Se, ole Abeto] esol] Sete Gest yes Nicholson 88 ele 4 ae aA ‘3]3)9} $39) Powers SCE EM, HEY OT Paarl: 20024 118 9B S| 8-91) Sub-neural network(MLP)=& #9] Feber sieela} : 20034 2a 121 wlals}a2 2 Sub-neural network2] S8) BEL. Decision EMRE LAF] 2000S AAICHA ISH BAR Logic WAI AS SS AS Somyge. B ELS ap ae FT Sa SF SB7ISNL: Hl WSFA Ae Meg Ws I49 Fe goed Fee solu, yh Ale! ‘Autonomous Family Machine(AFM) 227/27wk 7-4 os 7} Bat SYA RAS) AS VAS Fal hop (NO9-A08-4301-05), 21 ASAE O|FOMSo, Bef rh 77] PAE, BRR, Sebe) abel] wap APLFS| APULA MOL BAS RICH. 2 2d Ue LAS Sao] 4 Passe] tes, alee) AML, ME sft ae va 48aia] 3! A)SAAS881 A) 2008, Vol 13, No. 1 OG, 2, 1d S44 Se] WH Bohs Aeleh, ela, Sled SUES olBeol Li, hes IS Aa) WAS ach spades Gadel] So] wa] DRNN (Dynamic Recurrent Neural Network) & AHF e122} Seats Wetebat Bal AAVMeL oye ap waa SA AAsoleh, eee oleae Fue Vevey PE AFAS FAs UY AAS AV $ ele Aaa 9] Bagel, oleg@ Se 2s zo} vi DRNNOIG. 0} Ae as AM Bea AaGOe bet sash ole} UE) SYS AAmolaleh 28 € 21 See gaat elo] Quavols Aso] ae ohh BAAS BTS Reefs ale SUE gs Agee Peel @ Sele Aw AF LAY Rae adele 2 $9] Awad Aer: deh Ae] da Felebs| olzol sae ole] FaiFES] BeOw ofFopyseue sy] BAS Oe ASE TF ald Woo) EE SU. S4bel9} 712-2(Pundamental Frequency) & 125H2~ 500K FH0I"] io] AME G0) AS SACL S Gel. OA] Soh SAS VS IRZF7E Be ago Baas We Azt e altel Batol: Agu obey) aio] 277k | ABS CG AS ele eh. ease SOOHLZOLA BA HS E11 2 Mol He Ba LA Table 2. Acoustical analysis for MI M2 MAPS] MLAS Mae iRiHa | eh 300 on oo aru | uo | 120 SR | 00 200 [2780 abut) | 380 NU p_| 3400 sKaw | 570 nu__| 400 Max [08 16 06 14 Tnidb) | Tab | S8db | Tad | Sb Puch [tite | sor | loo | sole 3. M2a oie SO EA Bore ols, Geele| Ye FE LHL SHA Table 3. Acoustical analysis for M2 2MOHe eH NAIA THR WE eT oy AEST iam | 82 | 7m | «= 31 FE dete ube ae] Sa arin | 120 | 12%» | 0 | 60 ‘Table 1. The relations of frecuency and feeling Spay} ans0] ar00 son —T 0 Fae wa Se ara [m0 | 4, | 200 | 300 5 lsxae eave aziz, sara) [seam | NU ae | 300 Natta Stotle dearbal ebat Sel} Maz _|_16 16 m oa Bike sionrie [SHAG Sawa GAA Sa] [ey | aT & 74 Dae SSH. cH Cee aml] [pun | 17 | 2 os [a8 annotte~sboorie fo) Es Be Aeby Ue, Baa ste sete] Aeebs aha 8 Bore fla eS Brad 24. $d Sheds] teas Ase ai] Ee PS Otol Gey LE aaa Lae Geb uo 211 ee as ea BEES SYALE Wiese, Ag ag be, BA 9) G71 Al. G74] LES] WAS slate} ey oFo} ujgt ela sbsleh ab 2 of] F : Kormant , Mag Magnitude , Int + Intensity, NU + Non Uniform , ML Man 1 & elolatc 32 fe] Ja ole} S eae Ato. 9] a Po] Molt Pitch Melee THO HS skc, WALS elo] WS 345 Forman} IEA AS oF NC. WS Bel MSE] APE ol aiajey, volte] GE els) 2+ Uniforme Formant®] 2365 Mo}}, QILe £2) S vlelestey & WAGE 345 FormantlA] Non Uniform’ 3% magick ol Bole Ael7} Beal ek telat, 105 SOKA: Eh Selo] Ree} ate See, IF 46 ‘Table 4. Variance and spectrum analysis Be aa ee “gays [ees [ee] s A [axss| 6 [isos [oo [aes [a Bf [a rae 95s [2 cc [ame | 1 Peon [2s [sie Gees We AHEM SIE A) EAL AGL YA Woh shal [oH 3742) Seolez etshe BFA BBS WA weAaht aaboleh wt % GS Web| Weeks AF Gy hy Ta Aabe EQS Wolts Yoese] Vals) wine spate wolal LE BS} del, WHPACEA) Walle BF Abe] woh Aa, SE BU FH Eto] Mobale Wee Ee UC Ha, SE FF VIA] VAS des @ woz, We THF ALS OY UTE F oo, Ameatal BAYA ASAE WE 299 AES SE AO2M, Hole Us} $17] 1, WE Sal BEE St EE oFYe} wees St Aa] gab Was, ee Bel AG SA Sele Alc} Beh) aeeob AL EE ad A of42)9) 10-2067} Makes AS Bs ath Be H olelth RAS wom Urbal(ea) ashe Be} AGONY} Yok shoe Ade] AA chet vag al 13 Ut = I, 21.2 UBat wajol els) aaa, may Pitch Contour? 228340 5} (Shout Type) 71) Magnitude, Intensity7} 7% 2c} Uy a Edel GAS) Accented Point? £4 ath, cH oo) Hol Se Bele Ae Uh SOS 7h ug Sle se) 1 &EBroad Laugh) 7h) dsol7b ARCH ( Formants| 447+ Uniform) U) aS) Ae ela atc op) Salat Vee} welch a; Aaldon gaeop meson) 7H \TDOHZ - 2500Hzo ofl217 79 we] AEST (2el9] the} $8) Up SLAl7e oA Sec 22 DANNS o1@tt Z4tel a 22.1 SSS aad ABaeleis) de gout sald aes AMBalth =, 47b7| AC BASLE SH Dol He HE #2 DRNN (Dynamic Recurrent Neural Network) 25 ol Sabo} obi alaleieh sla} FSab71 alah Fal Autocorrelation Approach using Center-Clipping Function & ABC AUS Samat 2) @ wae Clipping, function 2 Air) tol Bate Autocorrelation G44 YeRHs1, Center Autocorrelation Fal Mele $12} & ABH: geet y(n) = ate) 2 A (2)9) 29H 1. &l Center clipping function: HERAT FE PANE VAD ALCL] Lee 1s a = Yala, CL we} el Gad sal CL eh ole BANS $A] To AGH: BRE 27h Bal eb HE SOS SAA PolYeS aAatS oleh. eh 2, CLE SQ ws] 79S SAAS MMs) OE 71 = aU] 218] 2 center clipping function & *}8-315i-% she] SHE YehyS ROEM (ave Aes] es YehMR (©) 7} clipped signal Beh (bs (ais FF ele] 5} clipped signalel 84 autocorrelation @ 29S Geb ue 8 clipped signals} 20 F717b cS eet Oba) Veh He 2s Hel BF alk. ABAD} DRNNE 01ST BASSI ZA oI) as ce x 2H} 1, Center clipping 4° Fig, 1. Center clipping function SANT lh =H! 2 Center clipping 4°21 3 Fig. 2, The effect af applying the center clipping function 22.2 A|Balolet 9) 3, & Belles] FS ee hols Bat 2} e8o) Uae, SA) sale Fan Fee Se 4 hol Sel hes ANG, es Batol AIS] Weight 415 47s Sat W412 “ss => 24 3. AVBelolete] FR Fig, 3. Simulator structure a7Bx] BI X|SALA E1818] SB] 2003, Vo. 13, No. 1 223 halo} ead Pats Bane Fests (1100)-ES EAS Evaluation function & 38 #3} 23HGt| AE Abe a2 Penalty Rule o1B2H* "Py 7S 0] gate} vata. 29) 4, (1+100) Fig. 4. (14100)-BS Penalty Rule® 23 got Bs} se} apo]s) vol) ale BALE DvD Os Wlolch, Sak alah aah Ge Bit ola, FH) MES 1~1 Abele] AF hE ALASIeL OFS] OMA A vis} ao) 1S 7HasbOHS MEET OM OSA sie 24 dit ol = wae Asa 2 BE BE AME S BEL Lola ge me] MEN 1 olsvaaion, a) gt F 2 ves zie ao} aolalel 1 al SE Mel] oe Aelm= case 2 gto] wale daeata. 35. Penalty 19] 48 Table 5. Penalty rule Case 1: itgoallisujal-=-1 && resultifsujjal>0) ill Case 2: For(sujja-Oesuija<=tsuliat) if(cresuti(3}-resulti{suifa)}<03 && sajjal=3) ddif-cif-3+(03-result1 (3H result ICsuiial 2 2.2.4 DRNN DRNNE “191 54] 54 eM a2 4 ala, ale qudse Agee Back Propagation Hage ee, gad ay Ja an lth Se why Bohie Woe we EAS GEA food forward networks 2F8}51 EE4}91 Back Propagation GHGS AGN AOTC, Jordan's sequence generating network Blman’s seauence nrediction networkel 2+ APE A-lY, ole WYSE aAabeso] wr Mel # Wo] Plse Mao] eh B eee wale) Ba Aoe eas owe] Sd FE AS 100-ESE ARE otol aia) aes stakeatal 48 ‘Fully connected, ‘Input, Hidden:2, Oued ‘Input: Pitch, ‘Out! Normal, Angry, laugh, Surprise KORTE 8 8s S686 [rou J [acon] ouou 14} 5. DRNN2| Fa Fig. 5 Dynamic Recurrent Neural Network structure Dynamic Recurrent Neural Network) aia, Sol elo] Azo wet At DRNNe} ole}@h EH) Holelal 4gebch as] SIL FS A eh = Ab GI) +000) @ HA (Zre wx (O42 (0) aw CS AE (OD) MT i at EO CHAE AONE, x D% AR eo] SIF SIL) nonlinear derivative activation function 10151 2 4, « 1) tom) Trew S24) “ 28, ABatold aah 23.1 Waa Seol te aa aualolels| qhetes: slals} eres Glo) Abe BES MAE AME 7: Ae) Bate we 6, 7b Oh Sethe) Gaels sIvle 1 sla Ge) gel THE Chole} Mol A7t TSS} Vojok sH=eH, eye AS BAS ds She AlEeo|AeS ole] Aste] YRS ol ach 16 sal} SURE Noe DA AE ae ‘Table 6. When both pitch and formant are used ass the input. 4 wat a CaBse/ Ee) Normal ui Anu a8 Lauh wa Surprise 037, sale hae Be ‘Table 7. Only pitch is used. a cassava) Noa vi dows va tea 4 Son 8 292 wr7H+el Sol ole wa HSS AVHIFEN SE deh ah aol ae ume US olGso] SL AAS BS Ws BE Bh 43 GS) AE BH Penaltys: FOL HS wee] She ah 3, 9) Bah st MOLE DL MEE Bee Ae) of a2] Qe BF EY Penaltys Fo} BE soll wel PVRAVASS Of: Penalty rule® 3-H}9Ac4. 5 24-2] 2 Be HSS] KRG} Mola)is uhh ade. FE 8 Raw difierenc ate ae ‘le Brat by lence na esaateyaity Soma a toa 8 Lah v4 Sure ws 9 Penalty rule PB 8% Table 9, Evaluated by penalty nile ws cuenavensty Noma vi ana « Tah uu —, w 2a e 2 RES SOF AAEM Gael Bal oa wert 2, IAS ACS OE sho} Bales Fe BA POAC He VIA Bola Ashe ObFe Belch 2a ole AML E BA shehvle| S| Aol Base, bd HE soles Ase ee] OS sbatoge See ase ae alee Fe seh = at AA] Aviat BVA 2. 2) EAE olBate} HEI] ME OHS oleh ae 2 S2EALT DRNNG Ole SASSI ZN eI) UJ JS. Han, Speech Signal Processing, Seoul, 0 “Sung media, ».90, 2000. [2] CH, Park, K. 8. Heo, D. W. Lee, Y. H. Joo, and K. B. Sim, “Emotion Recognition based on Fre~ quency Analysis of Speech Signal,” Proc of the FIRA Robot 2002 Conference, 2002 [3] KB. Sim, Methodology of Artificial Life, Seoul, Dream-Medis, 2000, [a] M.A. Arbib, The Handbook of Brain Theory ane Neural Nenvorks, The MIT. Press, Cambridge, Massachusetts, pp, 796-799, 196. [5] HB. Jun, D. W. Lee, D. J. Kim, and KB. Sim, “Fuzzy Inference-based Reinforcement Learning, ‘of Dynamic Recurrent Neural Networks,” Proc. of SICE, pp 1083-1088, 1997. AL XE 7H ‘1712 (Kwee-Bo Sim) IRI: SYED AAA Beha 19864: RL AArE AY ApS A 199044: The University of Tokyo 2713 ap} Ess} DOsd~ Wa: Baal LSA Aap 3) 83/2 Boor 2002: whabadz7}ets) alo} y FES Masa wo heels} 200064 ~ al: MoPHESPALAR ES! ol Af 1901 ~ Gal: FOhHSp aap r|-BOp! sae ean, dali eed, est] DARE WW, ARRAY, 2a, a ala & 182-2817 0553 kbsim@cau.ackr ‘wtatel (Chang-Hyun Park) uot : Saeistat WeRV TLE SDI aal :-B sisi aaa lsay ABBA Bleek asad, Wasa Phone! +82 319 E-mail : Sne@alie.cauackr 49Bx] BI X|SALA E1818] SB] 2003, Vo. 13, No. 1 9.£(Young Hoon Joo} Iga: GAT AS SY, ised alata) ehaegl 71S eR: Beebe) Ase) Sa TN, 1986~ 199641 BA: baz) abe aL PA (AD Wa), 199864 12)~ 196m 221: 155 University ‘of Huston 417] 3) 24ft} #5} Postdoc. 20a ~ Hal: apy] SL ah Slew sts] W olAy mores eal aigh 7188) bolas algal al 1995'd 9~ Ba: Baka Sd AaPyMsspyt wy MIF: oizlabol, A}Salo}, Heal Phone + 053-459-4706 Fax: 053-409-4706 mail: yhjoo@kunsan ae ke neg, Ase EM