A SEMINAR REPORT

ON

“RECOGNITION OF PUBLIC SPEAKING

ANXIETY USING ACTIVE SHAPE MODEL”

Submitted to
THE SAVITRIBAI PHULE PUNE UNIVERSITY,
PUNE

THIRD YEAR OF ENGINEERING

COMPUTER ENGINEERING

BY
SANIYA SALIM SAYED T150434365

UNDER THE GUIDANCE OF

PROF. SONA R.PAWARA
DEPARTMENT OF COMPUTER ENGINEERING
SINHGAD ACADEMY OF ENGINEERING
KONDHWA, PUNE - 411048
2018-2019
 SINHGAD ACADEMY OF ENGINEERING
Department of Computer Engineering
KONDHWA, Pune 411048

CERTIFICATE
This is certify that the seminar entitled
“RECOGNITION OF PUBLIC SPEAKING ANXIETY USING
ACTIVE SHAPE MODEL“
submitted by

Saniya Sayed T150434365

is a record of bonafide work carried out under the supervision of Prof. Sona R.Pawara,
in the partial fulfilment of the requirement for the award of Degree of Bachelor of
Engineering (Computer Engineering) at Sinhgad Academy of Engineering, Pune un-
der the Savitribai Phule Pune University. This work is done during year 2018-2019.

Date: / /

(Prof. SONA R.PAWARA) (Prof. MAHENDRA NIVANGUNE)

Seminar Guide Seminar Coordinator

(Prof. B.B.GITE) (Dr. K.P.PATIL)

HOD, Computer Department Principal
 Acknowledgements

I am profoundly grateful to Prof. Sona R.Pawara for her expert guidance and
continuous encouragement throughout to see that this seminar rights its target since
its commencement to its completion.

I would like to express deepest appreciation towards Dr. K.P.Patil, Principal,

Sinhgad Academy of Engineering, Prof. B.B.Gite, Head of Department of Com-
puter Engineering and Prof. Mahendra Nivangune, Seminar Coordinator whose
invaluable guidance supported me in completing this seminar.

At last I would like to express my sincere heartfelt gratitude to all the staff mem-
bers of Computer Engineering Department who helped me directly or indirectly dur-
ing this course of work.

Ms. Saniya Sayed

 ABSTRACT

Public Speaking Anxiety (PSA) is one of the most universal subtypes of social
anxiety, and the facial expression recognition of PSA is an immediate area of re-
search focus. The experiment obtained the facial expression data of 18 postgradu-
ates in their thesis defense for the masters degree. Then, valid data were selected by
using self-evaluation of the subject and the average evaluation of 5 audience. Next,
the paper used Active Shape Model (ASM) algorithm for feature extraction of 68
feature points in 5 facial zones. Moreover, the paper applied support vector machine
algorithm to recognize the social anxiety state according to the displacements of the
feature points, and backward selection (BS) algorithm was used to find out the most
important features in recognition. The result illustrated the validity of the method
and obtained the best correct rate of 99 percent in the recognition of social anxiety
state. Meanwhile, the maximum displacement zone in social anxiety state is jaw, and
followed by mouth, eyebrow, eyes and nose. Obvious individual differences were
found in the formation of anxious facial expression.
Keywords: Public speaking anxiety, facial expression, Active Shape Model,
Support Vector Machine, Backward Selection
Contents

1 Introduction 2
1.1 The Idea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Point Distribution Model . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Active Shape Model . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.5 Backward Selection . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.6 Support Vector Machine . . . . . . . . . . . . . . . . . . . . . . . 5

2 Literature Review 7

3 Existing Systems 8
3.1 Recognition using Personal Report of Confidence as a Public Speaker
(PRCS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.1.1 PRCS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.1.2 Working . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.1.3 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2 Recognition by analysing Back Propagation Neural Network Algo-
rithm signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2.1 Working . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4 Methodology 11
4.1 Flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.1.1 Image Pre-Processing . . . . . . . . . . . . . . . . . . . . . 12
4.1.2 Extraction of Facial Expression . . . . . . . . . . . . . . . 13
4.1.3 The displacement of feature points during anxiety . . . . . . 13
4.1.4 Feature Selection and Classification . . . . . . . . . . . . . 14

5 Result 16
5.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
6 Conclusion 18

References 18
List of Figures

1.1 PDM of a hand. . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2 Operation of the shape model . . . . . . . . . . . . . . . . . . . . 4
1.3 Backward Selection . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 Support Vector Machine . . . . . . . . . . . . . . . . . . . . . . 6

3.1 Scattered Plot of the PRCS anxiety score against it’s ground truth 9
3.2 Evaluation of anxiety . . . . . . . . . . . . . . . . . . . . . . . . 10

4.1 Flowchart of the system using Machine Learning . . . . . . . . . 11

4.2 Schematic View of the face cut range . . . . . . . . . . . . . . . . 12
4.3 Mark order of characteristic points . . . . . . . . . . . . . . . . 13
4.4 Histogram feature point displacement . . . . . . . . . . . . . . . 14
4.5 The selection results of all feature points . . . . . . . . . . . . . . 15
4.6 The selection results of all the relative displacements . . . . . . . 15
 Recognition of Public Speaking Anxiety using ASM

Chapter 1

Introduction

1.1 The Idea

Anxiety is the state of a person where a person feels worried, nervous and un-
easy about something with an uncertain outcome.It is one of the immediate area of
research.Out of all the many types of anxiety, Public Speaking Anxiety is the most
common and can be determined by the facial expression of the person.Recognition
of Public Speaking Anxiety with traditional methods seems inconvenient, time con-
suming and do not guarantee the accuracy.Therefore, a more convenient and accurate
method for the recognition of Public Speaking Anxiety is brought into the picture ,
which gives an accuracy of 99 percent.The paper focuses on determining the anxiety
using facial expression of the person.Considering the facial expressions, the person’s
anxiety could be determined easily without going through a lengthy process.This is
not heavily used in mainstream but is becoming more natural in recent technologies.

1.2 Machine Learning

Machine learning is an application of artificial intelligence (AI) that provides sys-

tems the ability to automatically learn and improve from experience without being
explicitly programmed. Machine learning focuses on the development of computer
programs that can access data and use it learn for themselves.
The process of learning begins with observations or data, such as examples,
direct experience, or instruction, in order to look for patterns in data and make better
decisions in the future based on the examples that we provide. The primary aim is
to allow the computers learn automatically without human intervention or assistance
and adjust actions accordingly.

Department of Computer Engineering, SAOE, Pune 1

 Recognition of Public Speaking Anxiety using ASM

1.3 Point Distribution Model

The Point Distribution Model (PDM) is a shape description technique that is used
in locating new instances of shapes in images. It is very useful for describing features
that have well understood general shape, but which cannot be easily described by
a rigid model. The PDM has seen enormous application in a short period of time.
PDM basically represents the mean geometry of a shape, along with statistical modes
of geometric variation inferred from a training set of shapes.In the figure 1.1:,the dots
mark the possible location of landmark and the line denotes the mean shape.

Figure 1.1: PDM of a hand.

courtesy-https://prateekvjoshi.com/2014/05/17/recognizing-shapes-using-point-distribution-
models/

1.4 Active Shape Model

Active shape models (ASMs) are statistical models of the shape of objects which
iteratively deform to fit to an example of the object in a new image, developed by
Tim Cootes and Chris Taylor in 1995.The shapes are constrained by the PDM (point
distribution model) Statistical Shape Model to vary only in ways seen in a training
set of labelled examples. The shape of an object is represented by a set of points

Department of Computer Engineering,SAOE, Pune 2

 Recognition of Public Speaking Anxiety using ASM

(controlled by the shape model). The ASM algorithm aims to match the model to a
new image.

Figure 1.2: Operation of the shape model

courtesy-https://people.eecs.berkeley.edu/ efros/courses/AP06/Papers/cootes-pami-
01.pdf

1.5 Backward Selection

The Backward Selection(BS) procedure looks to remove variables from the model.
Once removed, those variables cannot re-enter the model even if they would add sig-
nificantly in the later step.

Department of Computer Engineering,SAOE, Pune 3

 Recognition of Public Speaking Anxiety using ASM

Figure 1.3: Backward Selection

courtesy-https://www.researchgate.net/figure/Backward-selection-forward-selection-
and-stepwise-selection f ig92 8356272

1.6 Support Vector Machine

In machine learning, support-vector machines (SVM) are supervised learning

models with associated learning algorithms that analyze data used for classification
and regression analysis. Given a set of training examples, each marked as belonging
to one or the other of two categories, an SVM training algorithm builds a model that
assigns new examples to one category or the other, making it a non-probabilistic bi-
nary linear classifier (although methods such as Platt scaling exist to use SVM in a
probabilistic classification setting). A SVM model is a representation of the exam-
ples as points in space, mapped so that the examples of the separate categories are
divided by a clear gap that is as wide as possible. New examples are then mapped
into that same space and predicted to belong to a category based on which side of
the gap they fall.

Department of Computer Engineering,SAOE, Pune 4

 Recognition of Public Speaking Anxiety using ASM

Figure 1.4: Support Vector Machine

courtesy-http://francescopochetti.com/support-vector-machines/

Department of Computer Engineering,SAOE, Pune 5

 Recognition of Public Speaking Anxiety using ASM

Chapter 2

Literature Review

a. Torsten Wrtwein, Louis-Philippe Morency and Stefan Scherer in their paper ti-
tled ”Automatic assessment and analysis of public speaking anxiety: A virtual
audience case study” analysed the automatic, acoustic and visual behavior of the
person to determine and recognize the public speaking anxiety.

b. Xu Zhang, Wanhui Wen, Guangyuan Liu and Hui Hu in their paper titled ”Recog-
nition of public speaking anxiety on the recurrence quantification analysis of
GSR signals” used the Galvanic Skin Response(GSR) through recurrence plot
and recurrence quantification analysis.

c. M. Slater, D.-P. Pertaub and A. Steed in their paper titled ”Public speaking in
virtual reality: facing an audience of avatars” used DIVE (Distributive Interac-
tive Virtual Environment) as the basis for constructing a working prototype of a
virtual public speaking simulation and thereby recognizing the public speaking
anxiety.

Department of Computer Engineering, SAOE, Pune 6

 Recognition of Public Speaking Anxiety using ASM

Chapter 3

Existing Systems

3.1 Recognition using Personal Report of Confidence as a Public

Speaker (PRCS)

3.1.1 PRCS

IN PCRS, the investigation of acoustic and visual behavioral indicators, such as

pause time and eye contact, which correlate with a self-assessed public speaking
anxiety score, i.e. the Personal Report of Confidence as a Public Speaker (PRCS) is
evaluated .

3.1.2 Working

In this method, basically three research questions were focused on the behavior of
participants with public speaking anxiety, the automatic assessment of public speak-
ing anxiety scores, and finally the interaction of anxious participants with a virtual
audience and their propensity to engage in learning. The first research question (Q1)
aimed at identifying acoustic and visual noninvasive behavioral indicators related
to public speaking anxiety. It was found that several indicators for public speaking
anxiety: a higher pause time, a lower variation in speech parameters, less eye contact
with the audience, and more fearful facial expressions. Based on these behavioral
differences the attempt to automatically estimate the self-assessed PRCS score in
research question Q2. The multimodal estimation yielded a high correlation with
the ground truth (r = 0.825) and a small mean absolute error (mae = 0.12). The last
research question Q3 focused on the interaction of anxious participants with a vir-
tual audience and their behavioral changes. Highly anxious participants were more
self-focused and less emotionally attached to the virtual audience. However, behav-
iors relevant to public speaking anxiety and performance, especially eye contact and
pause time, improved more for anxious participants after training.

Department of Computer Engineering, SAOE, Pune 7

 Recognition of Public Speaking Anxiety using ASM

Figure 3.1: Scattered Plot of the PRCS anxiety score against it’s ground truth

3.1.3 Limitations

1.The major disadvantage of this procedure is that the automatic assessment is fully
based on PRCS score.
2.The efficiency of the system is just 65 percent.

3.2 Recognition by analysing Back Propagation Neural Network

Algorithm signals

3.2.1 Working

In this, the application of Back Propagation Neural Network Algorithm(BP) is done

to achieve the goal of binary classifications between calmness and high anxiety state,
as well as between, high anxiety state and low anxiety state.

Department of Computer Engineering,SAOE, Pune 8

 Recognition of Public Speaking Anxiety using ASM

Figure 3.2: Evaluation of anxiety

3.2.2 Limitations

Recognition between calmness and anxiety is upto 86 percent correct but recognition
between high anxiety and low anxiety drops down to 74 percent.

Department of Computer Engineering,SAOE, Pune 9

 Recognition of Public Speaking Anxiety using ASM

Chapter 4

Methodology

4.1 Flowchart

Figure 4.1: Flowchart of the system using Machine Learning

Department of Computer Engineering, SAOE, Pune 10

 Recognition of Public Speaking Anxiety using ASM

The proposed system includes a combination of various algorithm like Active

Shape Model, Point Distribution Model, Backward Selection and Support Vector
Machine Algorithm.The facial extraction is done with the help of ASM and PDM.The
feature selection is done using SVM and BS algorithm.

4.1.1 Image Pre-Processing

The image or the face into action needs to be normalized.The normalization of

the image facilitated the extraction of facial expression features and made the light
compensation. The examples of the original and the normalized images are shown in
the table 1.1 . Specific steps of image normalization are as follows: Load the picture
and process it with histogram equalization. Mark three points (the mid of the pupil
of both eyes a1, a2 and the nasal tip a3) as the normalized reference points. Assume
the distance between a1 and a2 is d. The center is o. Make sure the consistency of
the direction of faces according to pitch rotation image for a1 and a2.
Ensure the minimum matrix feature region containing all features based on the facial
features, with o as the benchmark, according to the left and right distance d and up
and down distances 0.5d and 1.5d.
Transform the image of the interception area into uniform size by scale transforma-
tion. The distance between a3 and o is 80 pixels.

Figure 4.2: Schematic View of the face cut range

Department of Computer Engineering,SAOE, Pune 11

 Recognition of Public Speaking Anxiety using ASM

4.1.2 Extraction of Facial Expression

Active Shape Model (ASM) is applied to extract the feature points of the face. It is
an algorithm based on point distribution model (Point Distribution Model. In PDM,
the shape of similar objects, such as human face, hands, heart and other geometric
shapes can be formed by a number of key points (landmarks) of the coordinates of
a shape vector. In figure 4.3, 68 feature points of the face are extracted and labeled
with ASM algorithm.

Figure 4.3: Mark order of characteristic points

4.1.3 The displacement of feature points during anxiety

The distance between the two inside corners was marked as a standard scale, be-
cause the points between inner corners would not change. And then align all picture
features of the subject with apex to realize the feature point location normalization.
When the subjects are in anxiety, the five facial regions all have certain displacement,
but the displacements are different.

Department of Computer Engineering,SAOE, Pune 12

 Recognition of Public Speaking Anxiety using ASM

Figure 4.4: Histogram feature point displacement

As shown in Figure 4.4, anxiety caused larger displacements (blue bars) in re-
gions 1, 2, 3 and 5 than nonanxiety (red bars). The region having the most displace-
ment is the chin, followed by the mouth region. In the horizontal axis, the numbers
1, 2, 3, 4 and 5 mean the regions of chin, eyebrow, eyes, nose and mouth area, re-
spectively. The nonanxious facial images were obtained in a reading aloud task.The
features in the chin and mouth regions played great roles in the recognition of public
speaking anxiety.

4.1.4 Feature Selection and Classification

In order to find out the key facial feature points to distinguish anxiety status, fea-
ture selection through the backward selection (BS) algorithm. The 68 facial points
and the relative displacement of these facial points are respectively regarded as the
original feature sets. The BS algorithm was applied on each of the two original
feature sets, and the results are shown in Figure 4.5 and Figure 4.6. The features
were evaluated with the false rate of the binary classification of anxiety and non-
anxiety by using the support machine vector (SVM) classifier in the leave-one-out
cross validation process.

Department of Computer Engineering,SAOE, Pune 13

 Recognition of Public Speaking Anxiety using ASM

Figure 4.5: The selection results of all feature points

Figure 4.6: The selection results of all the relative displacements

Department of Computer Engineering,SAOE, Pune 14

 Recognition of Public Speaking Anxiety using ASM

Chapter 5

Result

5.1 Results

Tables 5.1 to 5.6 listed the selected facial points in each facial region when the
least false rate was obtained. ”1” in Tables 5.1 to 5.6 means a selected feature, and
”0” means a non-selected feature. As shown in Tables 5.1 to 5.6 when the error rate
is the lowest, many points of the chin and mouth regions are selected, and only 1
point in the nose area is selected, further proving the important role of the chin and
mouth area in the identification of anxiety.
Facial contour point 1-5 6 7 8 9 10 11-15
The 94th iteration result of the feature point 0 1 1 1 1 1 0
0 0 1 0 1 0 0
The 103rd iteration result of the relative displacement 0 1 1 1 1 1 0
0 0 1 1 1 0 0

Table 5.1: The result of BS in facial contour point.

Eyebrows contour point 16 17-21 22 23 24 25 26-27

The 94th iteration result of the feature point 1 0 1 0 0 1 0
0 0 0 0 1 0 0
The 103rd iteration result of the relative displacement 1 0 0 0 0 0 0
0 0 0 1 1 0 0

Table 5.2: The result of BS in eyebrows contour point.

Eyes contour point 28 29-32 33 34 35 36 37 38

The 94th iteration result of the feature point 1 0 0 1 0 0 1 0
0 0 0 0 0 1 0 0
The 103rd iteration result of the relative displacement 1 0 1 0 0 0 0 0
0 0 0 0 0 1 0 0

Table 5.3: The result of BS in eyes contour point.

Department of Computer Engineering, SAOE, Pune 15

 Recognition of Public Speaking Anxiety using ASM

Nose contour point 39-43 45 46 47 48 68

The 94th iteration result of the feature point 0 0 0 0 0 1
0 0 0 0 0 0
The 103rd iteration result of the relative displacement 1 0 1 0 0 0
0 0 1 0 0 0

Table 5.4: The result of BS in Nose contour point.

Mouth contour point 49 50 51 52 53 54 55 56 57

The 94th iteration result of the feature point 1 1 0 1 0 0 1 1 0
1 1 0 0 1 1 1 1 1
The 103rd iteration result of the relative displacement 1 0 1 0 1 0 1 1 1
1 1 0 1 1 1 1 1 0

Table 5.5: The result of BS in Mouth contour point.

Chin contour point 58 59-61 62 63 64 65 66 67

The 94th iteration result of the feature point 1 1 1 0 1 0 0 0
0 1 0 1 0 1 1 0
The 103rd iteration result of the relative displacement 1 1 0 1 1 0 0 0
0 1 0 0 1 1 1 1

Table 5.6: The result of BS in Chin contour point.

Department of Computer Engineering,SAOE, Pune 16

 Recognition of Public Speaking Anxiety using ASM

Chapter 6

Conclusion

The paper obtained 68 facial points to depict the main features of human face by
ASM and used SVM classifier to identify facial expressions of public speaking anx-
iety. The chin and mouth areas have relatively large displacement of feature points,
which have played an important role in identifying anxiety. However, the move-
ment of specific feature points during the formation of anxious facial expression has
shown great individual differences, which indicate the complexity of the formation
of anxious facial expressions.

Department of Computer Engineering, SAOE, Pune 17

 Recognition of Public Speaking Anxiety using ASM

References

[1] Automatic assessment and analysis of public speaking anxiety: A virtual audi-
ence case study; Torsten Wrtwein ; Louis-Philippe Morency ; Stefan Scherer

[2] https://adaa.org/understanding-anxiety/
social-anxiety-disorder

[3] Facial Expression Recognition of Public Speaking Anxiety;Fuli Huang ; Wan-

hui Wen ; Guangyuan Liu

[4] Public speaking in virtual reality: facing an audience of avatars;M. Slater ;

D.-P. Pertaub ; A. Steed

[5] http://www.cs.ucf.edu/˜bagci/teaching/mic16/lec12.pdf

[6] http://www.biostat.jhsph.edu/˜iruczins/teaching/jf/ch10.pdf

[7] Virtual speech anxiety training Effects of simulation fidelity on user experi-
ence;Sandra Poeschl ; Nicola Doering

Department of Computer Engineering, SAOE, Pune 18