Applications of Kernal Methods
KEY TERMS
Bioinformatics: This is the application of informat-
ics to analysis of experimental data and simulation of
biological systems.
Biomedicine: This refers to the application of engi-
neering to the medicine. It involves the design of medical
equipment, prosthesis, and systems and algorithms for
diagnose and therapy.
Communications: Communication is the act of send-
ing a message to one or more receivers. In this context, we
use the word communication to refer to the technologies
and theory of telecommunications engineering.
Composite Kernels: A composite kernel (see chapter
1 for a definition of kernel) is a linear combination of
several Mercer kernels Ki(xi,xj) of the form
L Hidden Markov Models have been used for speech
K (xi , x j ) = ∑ ai K i (xi , x j )
i =1
For the composite kernel to be a Mercer kernel, it needs
to be semi definite positive. A sufficient condition for
a linear combination of Mercer kernels to be a valid
Mercer kernel is simply ai ≥ 0.
Image Processing: This term refers to any manipula-
tion of digital images in order to compress/decompress,
transfer it through a communications channel or alter its
properties, such as color, textures, definition, etc. More
interestingly, image processing includes all techniques
used in order to obtain information embedded in a digi-
tal image or a sequence of digital images, for example,
detection and classification of objects or events into a
sequence of images.
Kernel Methods: Ashortened name for Kernel-based
learning methods (see previous chapter for an introduc-
tion to kernel methods). Kernel methods include all
machine learning algorithms that are intrinsically linear
but, through a nonlinear transformation of the input
data into a highly dimensional Hilbert space, present
nonlinear properties from the point of view of the input
data. The Hilbert space must be provided with an inner
product that can be expressed as a function of the input
data itself, thus avoiding the explicit use of vectors in
these spaces. Such an inner product satisfies the so called
Mercer conditions and is called a Mercer Kernel.
Signal Processing: It is the analysis, interpretation
and manipulation of signals. Usually, one calls signal
to a set of data that has been collected sequentially
and thus it has temporal properties. Signal processing A
includes, among others, storage, reconstruction, detec-
tion of information in presence of noise, interferences or
distortion, compression, encoding, decoding and many
other processing techniques that may involve machine
learning.
Speech Processing: Signal processing of speech
signals. An important group of speech processing tech-
niques are those devoted to speech communications.
Here, the processing includes analog to digital and digital
to analog conversions, and compression/decompression
algorithms that usually use speech modeling through
autoregressive models of small voice frames, exploit-
ing their local stationarity properties. Another important
block of techniques is the speech recognition, which
involves machine learning techniques. Traditionally,
recognition, but recently kernel methods have been used
with promising results. Also, there is an intense research
in text-to-speech conversion.
Support Vector Machines (SVM): A SVM is a lin-
ear learning machine constructed through an algorithm
that uses an optimization criterion which is based on the
compromise between the training error and the complex-
ity of the resulting learning machine. The optimization
criterion for classification is
1 N
w + C∑
2
L=
2 i =1
i
subject to
yi (w T xi + b ) > 1 − i
i ≥0
where w are the parameters of the learning machine, xi
are the training data and yi are their corresponding labels,
and xi are the so called slack variables. The criterion
minimizes the training data classification error plus the
complexity of the machine through the minimization
of the norm of the machine parameters. This is equiva-
lent to maximize the generalization properties of the
machine. The resulting parameters are expressed as a
linear combination of a subset of the training data (the
support vectors). This algorithm is easily extensible to
a nonlinear algorithm using the kernel trick, reason for
what it is usually considered a kernel method. A similar
algorithm is used for regression.