Speaker Verification: Spoofing and Counter

Measures

Mid-Term Report submitted for Major Project Evaluation

DEPARTMENT OF ELECTRONICS AND COMMUNICATION

ENGINEERING

P Sai Sreeja B150747EC

B Lekhana B150462EC
P Anand Kumar B150493EC
P Samhitha B150540EC
Durga Bhavani M B150861EC

Project Guide : Dr.Waquar Ahmad

 Speaker verification : Spoofing and Counter measures
Abstract— Various measurable physiological and be-
havioural traits which are distinctive have been investi-
gated for biometric recognition. Speech is the primary
method of communication in which physiological and
behavioral characteristics have individual differences i.e,
distinctive features of the vocal tract shape and intonation
can be captured and utilized for Automatic Speaker
Verification(ASV). Based on speech sample, the ASV’s
function is to accept or reject a claimed identity. Even
though biometric authentication has advanced signifi-
cantly, they remain vulnerable to spoofing attacks. One
can synthetically produce an individual speech and use
it for authentication purposes. Our project focuses on
fundamental recognition performances, as opposed to
security to spoofing by modifying some existing features
like Mel-Frequency Cepstral Coefficients(MFCC), Linear
Predictive Cepstral Coefficients(LPCC) and Perceptual
Linear Prediction (PLP) and conduct a study on imper-
sonation, replay speech synthesis and voice conversion
spoofing attacks.
I. INTRODUCTION
Speech can be used as a biometric feature to
distinguish between human beings as each speaker
has different style of speech delivery. Size and
shape of vocal tract, size of larynx falls under
the category of physiological structures of speaker.
This causes difference between the speakers in
speech production. Speaker modelling is essential
for many tasks, which include speaker recogni-
tion, speaker diarization, speaker change detec-
tion, speaker clustering, etc. Speaker recognition
refers to recognizing a person based on voice
samples of that particular person. On the other
hand, speaker diarization deals with finding who
spoke when, which is useful to find the speech of a
particular speaker from a conversation of multiple
speakers[2].
Our project focuses on the speaker knowledge
utilized for authenticating a person from speech
signal, which is referred to as speaker recognition.
Based on the task objective it can be categorized
as speaker verification(SV) and speaker identi-
fication(SI). In Speaker verification, a claim is
associated with each test example. On the contrary,
in Speaker identification the speaker does not make
any explicit claim and it rather attempts to find
the best match of the test speech against the
set of available trained models of the speakers.
SV systems are more practically acclaimed for
deployment purpose and SI systems are used in
forensic applications. Based on text SV, systems
can be classified as text dependent and text inde-
pendent systems. In text dependent system, text is
fixed, which makes the speaker to utter the same
text during training and testing. Where as in the
text independent system there is no restriction on
speech content of the users during training and
testing.
II. PROBLEM STATEMENT
A study of existing feature extraction algorithms
in Automatic speaker verification and and imple-
mentation of a new feature extraction algorithm
which includes detailed analysis of extraction algo-
rithms and draw conclusions based on their equal
error rates.
III. MOTIVATION
With the development of various techniques
suitable for modelling speaker characteristics, the
area of Speaker verification has gone through
major improvement in recent decades. These ad-
vancements in the field of SV has opened doors
towards practical deployable systems for person
authentication. In recent years people developed
a smart home security application for controlling
different household activities using short utter-
ances with the knowledge of speaker and speech
recognition. On the other hand a speech biometric
attendance system is developed for marking atten-
dance over online telephone based network. These
works showcase SV as an emerging area for having
deployable systems with real world applications.
 The use of short utterances for SV not only
reduces the required testing time, but also provides
comfort to the speakers as they are less burdened
for speaking. In this regard, the text-dependent
SV is found as a favourable one for having a
deployable system due to less amount of train and
test time involved. However, there is a very high
chance of spoofing by the unauthorized users as
the fixed phrases global across all the users of
the system. In the case of text-dependent SV, less
speaker variability is captured due to consideration
of a small phrase for speaker modelling. On the
contrary, the text-independent SV captures the
speaker characteristics in a more generic manner
and is less susceptible to the attackers. Further
as there is no constraint on the users on what
to be spoken, it is more convenient for the users
of systems in practice. All these salient features
altogether project text-independent Speaker verifi-
cation as a better choice for a robust system for
implementation.
IV. LITERATURE SURVEY
Speaker verification is the process to accept or
reject an identity claim by comparing two speech
samples in which one is used as reference of the
identity and the other that is collected during the
test from the person who makes the claim.
Feature extraction maps every interval in speech
to a multidimensional feature space. A speech in-
terval typically of 1030 ms of the speech waveform
and is referred to as a frame of speech. Feature
vectors are then compared to speaker models by
using matching. The obtained match score for
each feature vector or sequence of feature vectors
measures the similarity of the input feature vectors
to models of the claimed speaker or feature vector
patterns for the claimed speaker then a decision is
made to either accept or reject the claim according
to the match score or sequence of match scores
Automatic speaker verificaton contains
four steps:
1) voice recording
2) feature extraction
3) matching
4) Decision
A. FEATURE EXTRACTION METHODS
A speech signal has voice tone, prosodic
and language content. Each speaker can be
seperated by short-term spectral, prosodic
and highlevel unique features. Short-term
spectral features are extracted from short
frames of 20-30 milliseconds duration.
They describe the short-term spectral en-
velope which is an acoustic correlate of
voice timbre. MFCCs, LPCCs and PLP are
popular spectral features. There are also
some features like IMFCC, RFCC.
Prosodic features are extracted from
longer segments such as syllables and
word-like units to characterise speaking
style These features, such as pitch, energy
and duration are less sensitive to chan-
nel effects. However, due to their sparsity,
the extraction of prosodic features requires
relatively large amounts of training data
and pitch extraction algorithms are gener-
ally unreliable in noisy environments. High
level features are extracted from a lexicon
or other discrete tokens to represent speaker
behaviour.
Feature extraction methods:
1) MFCC(popular)
2) LPCC
3) RFCC
4) IMFCC
1) MFCC FEATURE EXTRACTION
METHOD: MFCC is a representation
of the real cepstral of a windowed short
time signal derived from the Fast Fourier
Transform (FFT) of that signal. The
difference from the real cepstral is that a
nonlinear frequency scale is used, which
approximates the behaviour of the auditory
system. It is an audio feature extraction
technique that extracts parameters from
the speech similar to ones that are used by
humans for hearing speech(Mel-scale)
Features extraction in ASR(Automatic
Speaker Recognition) is the computation
of a sequence of feature vectors which
provides a compact representation of the
given speech signal.
STEPS IN MFCC FEATURE EX-
TRACTION METHOD
1) The speech signal is divided into time
frames containing arbitrary number of
samples.FIR filter is used to enhance
higher frequencies of the input speech
signal where as Pre emphasis filter is
used to make the signal less suscepti-
ble to finite precision.
y(n) = x(n) − x(n − 1)
where x(n) is the input speech signal
and 0.9 ≤ α ≤ 1. The speech signal
is divided into number of frames of
duration 20-30 msec in which speech
signal assumed to be stationary with
50 percent overlap between two suc-
cessive frames in order to avoid any
loss of information.
2) Each time frame is then windowed
with Hamming window to eliminate
discontinuities at the edges. The fil-
ter coefficients w(n) of a Hamming
window of length n are computed
according to the formula:
W (n) = 0.54 − 0.46 ∗ cos(2πn/M )
3) In the next step after windowing, Fast
Fourier Transformation (FFT) is cal-
culated to convert each time frame to
Fig. 1. Melscale filter bank
frequency domain representation.The
magnitude of windowed signal is used
to obtain the power spectrum.
4) The Mel-Scaled filter bank is ap-
plied to the Fourier transformed frame
which is triangular. This scale is
approximately linear up to 1 kHz,
and logarithmic at greater frequencies.
The relation between frequency of
speech and Mel scale is
1 + f (Hz)
F requency(M elscale) = [2595log
700
In Mel-scale filter bank the higher fre-
quency filters have greater bandwidth
when comapred to lower frequency
filters, but their temporal resolutions
are the same.(Fig.1)
5) The last step is to calculate Discrete
Cosine Transformation (DCT) of the
outputs from the filter bank. DCT
coefficients range according to sig-
nificance, whereas the 0th coefficient
is not considered since it is unreli-
able.For each speech frame MFCC
is computed. The coefficients set ob-
tained in this frame is called an acous-
tic vector that represents the pho-
netically important characteristics of
speech which is very useful for further
analysis and processing in Speaker
verification . For example We can
take audio of 2 Second which gives

Fig. 2. MFCC and LPCC Block Diagram
approximate 128 frames each contain
128 samples having (window size =
16ms). We can use first 20 to 40
frames that give good estimation of
speech[1].
2) LPCC FEATURE EXTRACTION
METHOD: LPC is the most powerful
speech analysing method and
speech sample for the current time
can be approximated as a linear
sequence of past speech samples.
The methodology behind the usage
of LPC is to reduce the squared
difference between original and
estimated speech signal at a finite
time. With the help of LPC, we
can derive the LPCC coefficients
which is further translate into cepstral
coefficients. LPCC is obtained by the
method called autocorrelation.
The cepstral coefficients obtained
from both linear prediction (LP) anal-
ysis or a filter bank approach are
almost treated as standard front end
features , when speech is recorded in
a clean environment speech systems
developed based on these features will
obtain a high level of accuracy .These
spectral features represent phonetic
information, as they are derived di-
rectly from spectra. The features ex-
tracted from spectra emphasize the
contribution of all frequency compo-
nents of a speech signal. LPCCs are
generally used to capture emotion-
specific information obtained through
vocal tract features.Cepstrum may be
obtained using linear prediction anal-
ysis of a speech signal.
The basic idea behind linear predic-
tive analysis is that the nth speech
sample can be estimated by a linear
combination of its previous p samples
as shown in the following equation.
s(n) ≈ a1 s(n1)+a2 s(n2)+a3 s(n3)+· · ·
· · · + ap s(np)
where a1, a2, a3, ... are assumed to
be constants over a speech analysis
frame.
These are known as predictor coeffi-
cients or linear predictive coefficients.
These coefficients are used to predict
the speech samples. The difference of
actual and predicted speech samples
is known as an error. It is given by
p
X
e(n) = s(n)−ŝ(n) = s(n)− ak s(n−k)
k=1
where e(n) = error in prediction, s(n)
= original speech signal, s(n) = pre-
dicted speech signal, ak ,s are the pre-
dictor coefficients. For caluculating
unique set of predictor coefficients,
the sum of squared differences be-
tween the actual and predicted speech
samples is minimized as shown in the
equation below
 p
2 X
E{e [n]} = E{s[n] + ak s[n − k]}
k=1
where n = number of samples in an
analysis frame. For obtaining LP co-
efficients, En has to be differentiated
with respect to each ak and the result
is equated to zero En / ak = 0, for k
= 1, 2, 3,..., p
After finding the ak s, we can find
cepstral coefficients using the follow-
ing recursion.
C0 = loge p Fig. 3. Filterbanks of a)RFCC b)LPCC c)MFCC d)IMFCC

m−1
X k
C m = am + Ck am−k against MFCC. It is defined as one
k=1 m
of the characteristic property of the
for 1<m <p audio system.
m−1
X k 4) RFCC FEATURE EXTRACTION
Cm = Ck am−k METHOD:
k=m−p m

for m >p V. PHASE INFORMATION

3) IMFCC FEATURE EXTRACTION
METHOD: This extraction method
follows the opposite path of evolution
of human audio system. The basic
idea is to retrieve the information lost
or missed in the original MFCC. The
structure of the filter bank is inverted.
Doing as such higher frequency range
is average by more accurately spaced
filters and small number of widely
spaced filters used in lower frequency
range. Such a feature set is named
as Inverted Mel frequency Cepstral
Coefficients (IMFCC). The procedure
for IMFCC is somewhat same as
that of MFCC but used reverse filter
bank structure. The nature of IMFCC
is reciprocal in nature as compared
EXTRACTION
Speech signal can be processed by
short-time Fourier analysis. Given a
speech signal x(n), the short-time
Fourier transform (STFT) is given as
follows:
X(ω) = |X(ω)| ∗ ej∗ψ(ω)
Re |X(ω)| is the short-time magni-
tude spectrum, and ψ(ω) is the short-
time phase spectrum. The square of
magnitude spectrum, |X(ω)|2 , is usu-
ally called the power spectrum.
As natural phase information is miss-
ing in the reconstructed speech or
converted speech, phase spectrum can
be used to derive features providing
the evidence of converted speech. In
this study, two different features will
be derived from phase spectrum as
follows.
A. COSINE NORMALISATION OF
PHASE SPECTRUM
As the original phase spectrum is not
continuous in frequency domain, we
first unwrap the phase spectrum as a
continuous function of frequency. Af-
ter unwrapping, the range of spectrum
could vary which makes it difficult
to model phase information. Cosine
function is then applied on the un-
wrapped phase spectrum to normalize
the range into [1.0, 1.0]. Then discrete
cosine transform (DCT) is applied on
the Cosine normalized phase spec-
trum to reduce dimensionality. Finally
we keep 12 cepstral coefficients, ex-
cluding the 0th coefficient. This fea-
ture is called cos-phase.
B. FREQUENCY DERIVATIVE OF
PHASE SPECTRUM
The next feature is the frequency
derivative of phase spectrum, obtained
by the group delay function (GDF)
which is a measure of the nonlinearity
of the phase spectrum and defined as
the negative derivative of the phase
spectrum with respect to ω
τ (ω) = [XR (ω) ∗ YR (ω) + YI (ω) ∗ XI (ω)]/|X(ω)| 2
where Y(ω ) is the STFT of nx(n);
XR (ω), XI (ω) and YR (ω), YI (ω) are
the real part and imaginary part of
X(ω ) and Y(ω ), respectively. To cap-
ture the fine structure of group delay
phase spectrum, in practice, a modi-
fied group delay function is adopted.
Smoothed power spectrum is used in-
stead of original power spectrum and
two variables are introduced to em-
phasis the fine structure of the phase
spectrum. The modified group delay
function is described as follows.
τγ (ω) = [XR (ω) ∗ YR (ω) + YI (ω) ∗ XI (ω)]/|S
τα , γ(ω) = [τγ (ω)/|τγ (ω)|]∗[|τγ (ω)|α ]
where |S(ω)|2 is the smoothed power
spectrum, τα ,γ (ω) is the modified
group delay phase spectrum, α and
γ are two variables to make the
phase spectrum be presented in favor-
able form. After modified group delay
phase spectrum is obtained, discrete
cosine transform (DCT) is applied.
We keep 12 cepstral coefficient, ex-
cluding the 0th coefficient. The 12
dimension feature are used for model
training and detection and this feature
is called MGDF-phase[5].
VI. METHODOLOGY
Performance Evaluation Classifier de-
scription After extracting the features
from the speech we need to use a
classifier to see the match percent-
age between
the natural and synthetic
speech ,so we used a classifier . ba-
sically there are two types of classi-
fiers 1.GMM 2.LBP-SVM In different
studies we came across we saw that
GMM technique gives us reasonably
good accuracy in database we have
used i.e ASV 2017.So we choose this specific GMM trained samples from
classifier for benchmarking various a particular speaker. UBM is used
features as a prior model in MAP parameter
Gaussian Mixture Model (GMM)- estimation.. The GMM shows best
Universal Background Model performance as compared to the other
(UBM). known method. A Gaussian mixture
model is a weighted sum of M com-
A Gaussian Mixture Model (GMM) ponent Gaussian densities as given by
is one of the parametric models. the equation
In GMM we find out the proba-
x M x
bility of all the Gaussian compo- p( ) =
X
wi g( P )
nents densities. Speaker verification λ i=1 µ, i
is one of the application area where Where; x is a D-dimensional
GMM are used. GMM parameters continuous-valued data vector (i.e.
are estimated from training data us- measurement or features)
ing Expectation-Maximization (EM) x 1 1 0 X−1
g( P )= D/2
exp − (x − µi ) (x − µi )
µ, 2
P
2π | |1/2
algorithm or Maximum a Posteri- i i i

ori (MAP) estimation from a well-
trained prior model . The EM al-
gorithm is iterative in nature. GMM
are generally used for text indepen-
dent speaker identification.There is no
need for the vocabulary database or
big phoneme. Capturing the general
characteristics of a population and
accordingly adapting it to individual
speaker is the basic idea of UBM.
UBM is defined as the model which
is used to compare the persons inde-
pendent feature characteristics against
person specific feature model during
decision of acceptance or rejection.
UBM is also said as GMM only
with large set of speakers. Firstly
likelihood score or ratio for an un-
known speech sample is found after
that the match score of speaker spe-
cific mode and universal background
model is formed by using speaker
With mean vector and covariance
matrix the mixture weights satisfy the
constraint that The complete Gaussian
mixture model is parameterized by the
mean vectors, covariance matrices and
mixture weights from all component
densities. These parameters are col-
lectively represented by the notation,
X
λ = {ω, µ, }
i
For a sequence of T training vectors
X = {x1 , · · · , xT }
The GMM likelihood, assuming inde-
pendence between the vectors, can be
written as
r
Y
p(X | λ) = p(xt | λ)
t=1
For utterances with T frames, the log-
likelihood of a speaker models is;
Ls (X) =
For speaker identification the value of
) (XL s is computed for all speaker
models s enrolled in the system and
the owner of the model that generates
the highest value is the returned as
the identified speaker. During train-
ing phase, Feature vectors are being
trained using Expectation and Maxi-
mization (EM) algorithm. An iterative
update of each of the parameters in ,
with a consecutive increase in the log
likelihood at each step.
EQUAL ERROR RATE:
In an automatic speaker verification
(ASV) system, the equal error rate
(EER) is a measure to evaluate the
system performance. Usually it needs
a large number of testing samples
to calculate the EER. In order to
estimate the EER without the ex-
periments using testing samples, a
method of model-based EER estima-
tion which computes likelihood scores
directly from client speaker models
is proposed. However, the distribution
of the computed likelihood scores is
significantly biased against the distri-
bution of likelihood scores obtained
from testing samples. Now we design
and manipulate the speaker models of
the client speakers and the imposters
so that the distribution of the com-
puted likelihood scores is closer to
the distribution of likelihood scores
obtained from testing samples. Then a
more reliable EER can be calculated
by the speaker model.
The natural/converted speech decision
is made using log-scale likelihood ra-
tio as follow
λ(C) = logp(C|λconverted)logp(C|λnatural
where C is the feature vector sequence
of a speech signal, converted is the
GMM model for converted speech,
and natural is the GMM model for
natural speech. Under the three differ-
ent situations, we have the same nat-
ural speech model natural, but three
different converted speech model con-
verted . The number of Gaussian com-
ponents of GMM is set to 512. Equal
error rate (EER) is reported as the
evaluation criterion.
Feature Equal error Rate(%)
MFCCs 16.80
cos-phase 6.60
MGDF-phase 9.13
Source: 2006 NIST Speaker
Recognition Evaluation Test Set
VII. FUTURE WORK
Our future works include identify-
ing best possible feature extraction
method and improving its perfor-
mance. Based on this data Speaker
verification system is to be trained
and tested for Gaussian mixture
model(GMM). Other blocks in SV
system will be studied and imple-
mented. Vulnerabilty of SV system
towards different spoofing attacks
will be studied. Countermeasures for
spoofing attacks will be proposed
 and implemented and system will be
trained accordingly and performance
check will be conducted.
VIII. WORKDONE
We evaluated the perfomance of dif-
ferent feature extraction methods like
[4] Douglas A. Reynolds A Speaker identification and verification
using Gaussian mixture speaker models, Speech Communi-
cation 17 (1995) 91-108 ESCA Workshop on Automatic
Speaker Recognition, Identification and Verification, Mar-
tigny, 5-7 April 1994.
[5] Longbiao Wang, Yuta Kawakami Relative Phase Information
for Detecting Human Speech and Spoofed Speech INTER-
SPEECH 2015.
[6] C. J. Kaufman, Rocky Mountain Research Lab., Boulder, CO,
private communication, May 1995.

MFCC,LFCC,RFCC of speech samples un-

der ASV 2017 database and caluculated the
equal error rate using a GMM classifier
using EM(expectation maximization) algo-
rithm.
A. RESULTS
ASV Database 2017

Feature Extraction Mixtures Scores EER(using development trial)(%)

MFCC trail1 32 25.56
MFCC trial2 128 13.53
MFCC trial3 256 12.57
LFCC trial1 32 16.38
LFCC trial2 128 12.29
LFCC trial3 256 7.56
IMFCC trial1 32 15.46
IMFCC trial2 128 9.76
IMFCC trial3 256 6.8
RFCC 512 8.40

Source: 2006 NIST Speaker Recognition

Evaluation Test Set
R EFERENCES
[1] Seiichi Nakagawa, Longbiao Wang, Shinji Ohtsuka Speaker
Identification and Verification by Combining MFCC and
Phase Information in IEEE TRANSACTIONS ON AUDIO,
SPEECH, AND LANGUAGE PROCESSING, VOL. 20, NO.
4, MAY 20121085.
[2] Gkay Diken, Zekeriya Tfeki, Ltf Saribulut Ulus evik (2016)
A Review on Feature Extraction for Speaker Recognition
under Degraded Conditions IETE Technical Review, DOI:
10.1080/02564602.2016.1185976.
[3] Fred eric Bimbot, Jean-Franc ois Bonastre EURASIP
Journal on Applied Signal Processin g 2004:4, 430451 c 2004
Hindawi Publishing Corporation.