Audio and Speech Processing for Data Mining
from an utterance. It is achieved through the following
Bayesian decision rule:
Wˆ = arg max P(W | Y ) = arg max P(W ) P(Y | W )
W W
where P(W) is the a priori probability of observing
some specified word sequence W and is given by a
language model, for example tri-grams, and P(Y|W) is
the probability of observing speech data Y given word
sequence W and is determined by an acoustic model,
often being HMMs.
HMM models are trained on a collection of acous-
tic data to characterize the distributions of selected
speech units. The distributions estimated on training
data, however, may not represent those in test data.
Variations such as background noise will introduce
mismatches between training and test conditions, lead-
ing to severe performance degradation (Gong, 1995).
Robustness strategies are therefore demanded to reduce
the mismatches. This is a significant challenge placed
by various recording conditions, speaker variations
and dialect divergences. The challenge is even more
significant in the context of speech data mining, where
speech is often recorded under less control and has more
unpredictable variations. Here we put an emphasis on
robustness against noise.
Noise robustness can be improved through feature-
based or model-based compensation or the combination
of the two. Feature compensation is achieved through
three means: feature enhancement, distribution nor-
malization and noise robust feature extraction. Feature
enhancement attempts to clean noise-corrupted features,
as in spectral subtraction. Distribution normalization
reduces the distribution mismatches between training
and test speech; cepstral mean subtraction and vari-
ance normalization are good examples. Noise robust
feature extraction includes improved mel-frequency
cepstral coefficients and completely new features. Two
classes of model domain methods are model adaptation
and multi-condition training (Xu, Tan, Dalsgaard, &
Lindberg, 2006).
Speech enhancement unavoidably brings in uncer-
tainties and these uncertainties can be exploited in the
HMM decoding process to improve its performance.
Uncertain decoding is such an approach in which the
uncertainty of features introduced by the background
noise is incorporated in the decoding process by using a
modified Bayesian decision rule (Liao & Gales, 2005).
00
This is an elegant compromise between feature-based
and model-based compensation and is considered an
interesting addition to the category of joint feature and
model domain compensation which contains well-
known techniques such as missing data and weighted
Viterbi decoding.
Another recent research focus is on robustness
against transmission errors and packet losses for
speech recognition over communication networks
(Tan, Dalsgaard, & Lindberg, 2007). This becomes
important when there are more and more speech traffic
through networks.
Speech Data Mining
Speech data mining relies on audio diarization, speech
recognition and event detection for generating data de-
scription and then applies machine learning techniques
to find patterns, trends, and associations.
The simplest way is to use text mining tools on speech
transcription. Different from written text, however,
textual transcription of speech is inevitably erroneous
and lacks formatting such as punctuation marks. Speech,
in particular spontaneous speech, furthermore contains
hesitations, repairs, repetitions, and partial words. On
the other hand, speech is an information-rich media
including such information as language, text, mean-
ing, speaker identity and emotion. This characteristic
lends a high potential for data mining to speech, and
techniques for extracting various types of informa-
tion embedded in speech have undergone substantial
development in recent years.
Data mining can be applied to various aspects of
speech. As an example, large-scale spoken dialog sys-
tems receive millions of calls every year and generate
terabytes of log and audio data. Dialog mining has been
successfully applied to these data to generate alerts
(Douglas, Agarwal, Alonso, Bell, Gilbert, Swayne, &
Volinsky, 2005). This is done by labeling calls based on
subsequent outcomes, extracting features from dialog
and speech, and then finding patterns. Other interesting
work includes semantic data mining of speech utterances
and data mining for recognition error detection.
Whether speech summarization is also considered
under this umbrella is a matter of debate, but it is nev-
ertheless worthwhile to refer to it here. Speech sum-
marization is the generation of short text summaries
of speech (Koumpis & Renals, 2005). An intuitive
approach is to apply text-based methods to speech