Web Site: www.ijettcs.org Email: editor@ijettcs.org, editorijettcs@gmail.

com Volume 2, Issue 2, March April 2013 ISSN 2278-6856

Study of comparison of human bio-signals for emotion detection using HCI

Ram K. Nawasalkar1, Harshal R. Lawange2, Surajkumar D. Gupta3, Pradeep K. Butey4
1,2,3

Deptt. of Computer Science ,Arts, Comm. and Science College, Kiran Nagar, Amravati. S.G.B. Amravati University, Amravati, India
4

Deptt. of Computer Science, Kamla Nehru Mahavidyalaya, Nagpur. R.T.M. Nagpur University, Nagpur, India

Abstract: The bio-signals for emotion detection is becoming

an interesting domain for human computer interaction. This paper focused on recognizing the effect of semi-classical and rock music on human body. In this proposed system there are two bio-signals such as RR (Respiration rate) and NIBP (Non - Invasive Blood Pressure)are analyzed by using HCI. In order to perform this assessment, we have gathered a dataset of bio-signals before and after hearing of semi classical and rock music and then effect shown by using graphs.

Keywords: Physiological signals, RR, NIBP, psychophysiological signals, Human Computer Interaction (HCI)

1. INTRODUCTION
Recently researchers are define human emotions in several categories and proposed many models for recognition. and also provides different techniques. The use of emotion in computers is a field which is becoming increasingly en vogue at the current time. In many ways emotions are one of the last and least explored frontiers of intuitive human computer interaction (HCI). This can perhaps be explained by the fact that computers are traditionally viewed as logical and rational tools, something which is incompatible with the often irrational and seeming illogical nature of emotions [4,9]. Some of the recent trends related to HCI tend to focus on how a computer can automatically detect the emotional state of a user and then adapt its behavior accordingly[10]. Emotion encompasses our daily life and play a key role in non-verbal communication. Although emotions are essential to human life there is not a universal definition for them. In general emotions cannot be measured. According to Scherer, they may be conceived of as consisting of various components: cognitive appraisal, physiological activation, motor expressions, behavior intentions, and subjective feeling [1]. An emotion is a complex psycho-physiological short-time experience resulting from the interaction of biochemical (internal) and environmental (external) factors [2]. It is interesting to note that the word emotion (from Latin: exmovere or moving out) shares the same root as the word motivation. Emotions have indeed a very strong impact in our decisions. Recent studies have shown that emotions can have memory enhancing effects in our Volume 2, Issue 2 March April 2013

mind: emotional events actually tend to be recalled more often and with more clarity and details than neutral events1. Basically, qualities of song are based on versatility like Khyal, Thumri, Dadra, Kajri which is distinctive of musical gharana or singing style. In our research we have noted the changes in bio-signals after hearing of semi-classical and rock music. In section II given the related work and in section III we have described the definitions and different bio signals used in present work. In section IV, we had shown a methodology for describing, how the data can be handled and results, graphs can be presented and in the last section V the conclusion made and future work was discussed.

2. Related work
2.1 Emotion recognition using bio-sensors Various bio-signals used in Emotion recognition such as Electromyography (EMG) electrocardiogram (ECG), electrodermal response (EDR), Skin temperature, and Blood volume pulse (BVP) and respiration changes. In this article, we discus about characteristic waveform detection, feature extraction and pattern classification stages using . Hilbert- Huang Transform (HHT) method and classification techniques such as Support Vector Machine. The highest recognition accuracy is obtained by using the fission based HHT features method with 28 features, 76% of test cases are correctly classified[3]. A procedure can be described, to train computers to recognize emotions using multiple signals from many different bio-sensors. In particular, they describe the procedure, they adopted to elicit emotions and to train our system to recognize them. Also they are present a set of preliminary results which indicate that our neural net classifier is able to obtain accuracy rates of 96.6% and 89.9% for recognition of emotion arousal and valence respectively[4]. 2.2 Emotion recognition using physiological responses We focus on the understanding of physiological responses to both stressor and physical activity and perform stress recognition, particularly in situations having multiple stimuli: physical activity and stressors. We construct stress models that correspond to individual situations, and we validate our stress modeling in the presence of Page 449

Web Site: www.ijettcs.org Email: editor@ijettcs.org, editorijettcs@gmail.com Volume 2, Issue 2, March April 2013 ISSN 2278-6856
physical activity. Analysis of our experiments provides an understanding on how physiological responses change with different stressors and how physical activity confounds stress recognition with physiological responses. In both objective and subjective settings, the accuracy of stress recognition drops by more than 14% when physical activity is performed. However, by modularizing stress models with respect to physical activity, we can recognize stress with accuracies of 82% (objective stress) and 87% (subjective stress), achieving more than a 5-10% improvement from approaches that do not take physical activity into account[5]. We implemented a novel 2-stage architecture of the emotion recognition system in order to improve the performance When dealing with multi-subject context. This type of system is more realistic practical implementation. Instead of directly classifying data from all the mixed subjects, one step was added ahead to transform a traditional subject independent case into several subject-dependent cases by classifying new coming sample into each existing subject model using Gaussian Mixture Model (GMM). For simultaneous classification on four affective states, the correct classification ration (CCR) shows significant improvement from 80.7% to over 90% which supports the feasibility of the system[6]. Describe the algorithms developed to analyze physiological signals associated with emotions, in order to recognize the affective states of users via noninvasive technologies. and propose a framework for modeling user's emotions from the sensory inputs and interpretations of our multi-modal system [7]. safety of fighter pilots during high altitude flights is monitored closely.

Fig 1: Respiration rate indicated in digital form

Fig 2: Respiration rate indicated in wave form during deep breathing

Fig 3: Respiration rate indicated in wave form during normal breathing Measurement of Respiration rate 3.2.1 Principles of Measuring Monitor measures RESP with the method of impedance. When a patient exhales and inhales, changes will take place in the size and shape of the thoracic cavity, causing consequent changes in the impedance between the two electrodes installed at the patients chest. Based on the cycle of impedance changes, the respiration rate can be calculated. 3.2.2 Preparatory steps of the measurements of RESP 1) Plug the 5-lead ECG cable into the ECG socket of the monitor. 2) Place the various pads of the electrodes onto the body of patient and connect them to the relevant lead cables. At this moment, the screen will show RESP waves and the RESP rate will be calculated. 3) Set the parameters relevant to RESP monitoring. 3.3 Electrocardiogram (ECG) ECG is a test which measures the electrical activity of heart and it is used to calculate rate and regularities of heart beats. The ECG signal is the manifestation of contractile activity of the heart. This can be recorded either directly on the surface of the chest or alternatively on the limbs. Recording ECG from the limbs is less inconvenient but more vulnerable to artifacts. For the initial system we decided to use the ECG from the chest to prevent the system being influenced by artifacts. It can be used to measure heart rate (HR) and inter-beat intervals (IBI) to determine the heart rate variability (HRV). A low HRV can indicate a state of relaxation, whereas an increased HRV can indicate a potential state of mental stress or frustration[3,8]. 3.4 NIBP Page 450

3. Bio- Signals
3.1 Definition of Bio-signal Within the scope of biomedical signals and sensors, a biosignal can be defined as a description of a physiological phenomenon, irrespective of the nature of this description. Since there is a nearly unlimited number of a physiological mechanism of interest, the number of possible bio-signals is very large. In the broadest sense, the variety of bio-signals extends from a visual inspection of the patient up to signals recorded from the human body using sensors[2]. 3.2 Respiration Rate Respiratory rate also known as ventilation rate, respiration rate, breathing rate, pulmonary ventilation rate, breathing frequency, and respiratory frequency or Rf = the number of breaths a person takes during one minute. It is usually measured at rest, while sitting. When we breathe more than the medical norm, we lose CO2 and reduce body oxygenation due to vasoconstriction and the suppressed Bohr effect caused by hypocapnia (CO2 deficiency). Hence, over breathing leads to reduced cell oxygenation, while slower and easier breathing improves cell-oxygen content. Respiratory rate is a critical indicator of performance in various professions. The health and

Volume 2, Issue 2 March April 2013

Web Site: www.ijettcs.org Email: editor@ijettcs.org, editorijettcs@gmail.com Volume 2, Issue 2, March April 2013 ISSN 2278-6856
Blood pressure (BP), sometimes referred to as arterial blood pressure, is the pressure exerted by circulating blood upon the walls of blood vessels, and is one of the principal vital signs. When used without further specification, "blood pressure" usually refers to the arterial pressure of the systemic circulation. During each heartbeat, blood pressure varies between a maximum (systolic) and a minimum (diastolic) pressure. The blood pressure in the circulation is principally due to the pumping action of the heart. Differences in mean blood pressure are responsible for blood flow from one location to another in the circulation. The rate of mean blood flow depends on the resistance to flow presented by the blood vessels. Mean blood pressure decreases as the circulating blood moves away from the heart through arteries and capillaries due to viscous losses of energy. Mean blood pressure drops over the whole circulation, although most of the fall occurs along the small arteries and arterioles. Gravity affects blood pressure via hydrostatic forces (e.g., during standing) and valves in veins, breathing, and pumping from contraction of skeletal muscles also influence blood pressure in veins[8]. Measurement of NIBP. 3.4.1 Principles of Measuring Monitor automatically conducts measurement of NIBP with the method of shockwave. The method of shockwave indirectly estimates the systolic and diastolic pressure within the blood vessels by measuring the change of the pressure within blood pressure cuff along with the volume of the arteries and calculates the average pressure. The measurements time of BP on a clam patient is less then 40s, and when each measurement ends, the cuff automatically deflates to zero. The monitor applies to any standards of the cuffs for child and adult. The monitor measures the blood pressure during the time of deflation. Monitor automatically conducts the seconds and third inflation measurement in case during the first inflation it is unable to measure the value of BP, and gives out the information for measurement failures. The longest cuff pressure maintaining duration is 120 seconds (90 seconds in neonate mode), and when the time is exceeded, the air will be deflated automatically. The monitor has been designed with hardware protection circuit regarding overpressure, errors of microprocessors, and the occurrence of power failure. should be loosened by turning counterclockwise first before unplugging). 2)Tie the cuff on the arm of patient. 3) Set the parameters and modes relevant to NIBP.

4. Methodology
4.1 Participants In this research work total 9 participants (age varied between 20 to 35) were involved but two of the participants were not in position to follow the instructions in the experimental phase, therefore their data was excluded from further analysis. 4.2 Emotions Elicited: happy, surprise, sadness, excitement Before choosing the song we used in this experiment and conducted a panel study to measure, how the musical elements can be elicited effectively and emotions through the participants. Sr. No. Type of Song Emotion A Dadra Happy B Zula Surprise C Thumri Pilu Sadness D Rock Song Excitment In above table, Dadra, Zula, Thumri pilu are type of song, usually sung in semi classical music. 4.3 Results In the first experiment, which we had conducted in one session, with a pause of 3-4 minutes in between the related songs. After stipulated duration, we took the readings before hearing the music as well as after hearing of music. Knowledge base in relation with above different type of 4 songs (of table) were having duration of 7 - 9 min, and each song was in the format of mp3, also with its own emotions, were reflected in the table as follows. The followings normal values of RR and NIBP for bio-signals are as followsRR (Respiration Rate) 10-15 NIBP 80-120 In the graphsOn X-axis A B C D represent the four different songs, among these A,B,C songs are semi classical and D is Rock song and On Y-axis digits represent the parameter values of different Bio-signals. Table 1 Respiration rate

Fig 4: NIBP and its mean indicated on in digital form 3.4.2 Preparatory steps of measurement of NIBP 1) Plug the air hose of cuff into the NIBP socket of the monitor and tighten it clockwise to ensure secure contact of the plug and the socket (Please note that the plug

Figure 5: Respiration Rate

Volume 2, Issue 2 March April 2013

Page 451

Web Site: www.ijettcs.org Email: editor@ijettcs.org, editorijettcs@gmail.com Volume 2, Issue 2, March April 2013 ISSN 2278-6856
Following are the mean of NIBP table for 7 subjects along with graph. Table 2 Mean of NIBP [3] Revathi priya Muthusamy, Emotion Recognition from Physiological signals using Bio-sensors, http://diuf.unifr.ch/main/diva/teaching/seminars/emo tion-recognition, Submitted for Research Seminar on Emotion Recognition on 15.02.2012. [4] Andreas Haag, Silke Goronzy, Peter Schaich, Jason Williams, Emotion Recognition Using Bio-Sensors: First Steps Towards an Automatic System, 2008 [5] Jin-Hyuk Hong Julian Ramos Anind K. Dey, Understanding Physiological Responses to Stressors during Physical Activity, ACM 978-1-4503-12240/12/09, UbiComp 12, Sep 5 Sep 8, 2012, [6] Gu Yuan, Tan Su Lim, Wong Kai Juan, A GMM based 2-stage architecture for multi-subject emotion recognition using physiological responses, ACM 978-1-60558-825-4/10/04..., Augmented Human Conference, April 2-3, 2010, Megve, France. [7] Fatma Nasoz Christine L. Lisetti Kaye Alvarez Neal Finkelstein, Emotion Recognition from Physiological Signals for User Modeling of Affect, 2008. [8] Manual for handling BPL Excello Multiparamonitor (Eco) [9] Picard R.W., Affective Computing, MIT Press, 2000. [10] Preeti Khanna,Multimodal Emotion Recognition For Enhancing Human Computer Interaction, http://shodhganga.inflibnet.ac.in/handle/10603/7529 ?mode=full&submit_simple=Show+full+item+record , P.hd. thesis submitted in MAY 2012.

Figure 6: Mean of NIBP

5. Conclusion and Future work

The songs having many qualities like rock, semi classical etc. were applied by knowledge base to the selected subjects. After hearing the above all songs with many qualities by age group between 20 to 35, number of variations, in the form of graphs found considerably. The vast difference between respiration rate (RR) was recorded largely after hearing the above various music qualities. We conclude that after hearing of song C and B, the maximum changes has found in RR and NIBP Mean respectively. For extracting the features, pertains with bio-signals, we can use to choose different algorithms. As compare with above NIBP and RR we can do work and research in EEG also. As we considered Rock and Semi Classical music, we also can choose other type of music quality and apply all above conditions on different age group and especially during the various time span (Prahar) of the day.

6. Acknowledgments
This research work was carried out in bio-medical research lab of Brijlal Biyani Science College, Amravati. Authors wish to thanks the faculty and students who took part in this project, and special thanks to Dr. D.S. Dhote, head of the department of Electronics, who permit this research work in his lab.

Reference
[1] K. R. Scherer, Approaches to emotion, chapter on the nature and function of emotion: A component process approach, pp. 293-317. Lawrence Erlbaum, Hillsdale New Jersey, 1984. [2] E. Kaniusas, Biomedical Signals and Sensors I, Biological and Medical Physics, Biomedical Engineering, DOI 10.1007/978-3-642-24843-6 1, Springer-Verlag Berlin Heidelberg 2012. Volume 2, Issue 2 March April 2013 Page 452