1.

Introduction to
Pattern Recognition and
Machine Learning.
Prof. A.L. Yuille.
Dept. Statistics. UCLA.
Stat 231. Fall 2004.
 Structure.
• Examples of Patterns.
• Discriminate/Decisions about Patterns.
• Schools of Pattern Recognition.
• Learning Theory.
 What are Patterns?
• Laws of Physics & Chemistry generate patterns.
 Patterns in Astronomy.
• Humans tend to see patterns everywhere.
 Patterns in Biology.
• Applications: Biometrics, Computational
Anatomy, Brain Mapping.
 Patterns of Brain Activity.
• Relations between brain activity, emotion,
cognition, and behaviour.
 Variations of Patterns.
• Patterns vary with expression, lighting,
occlusions.
 Speech Patterns.
• Acoustic signals.
 Goal of Pattern Recognition.
• Recognize Patterns. Make decisions about
patterns.
• Visual Example – is this person happy or
sad?
• Speech Example – did the speaker say
“Yes” or “No”?
• Physics Example – is this an atom or a
molecule?
 Applications of Pattern Recognition.
Handwritten digit/letter recognition
Biometrics: voice, iris, fingerprint, face, and gait recognition
Speech recognition
Smell recognition (e-nose, sensor networks)
Defect detection in chip manufacturing
Interpreting DNA sequences
Fruit/vegetable recognition
Medical diagnosis
Terrorist Detection
Credit Fraud Detection
Credit Applications.
……
 Two Extreme Approaches
• Generative Methods:
Determine models of how patterns are formed.
Use these models to perform discrimination.
Pattern Theory. Grenander.

• Discriminative Methods:
Don’t model pattern formation.
Instead extract features from patterns and make
decision using these features.
Example: Salmon versus Sea Bass.
• Generative methods
attempt to model the
full appearance of
Salmon and Sea
Bass.
• Discriminative
methods extract
features sufficient to
make the decision
(e.g. length and
brightness).
 Fish Features. Length.
• Salmon are usually shorter than Sea Bass.
 Fish Features. Lightness.
• Sea Bass are usually brighter than Salmon.
 Decision Boundaries.
• Classify fish as
Salmon or Sea Bass
based on a decision
boundary in feature
space.
Generative Models for Speech.
• Stochastic Grammars for Speech & Natural
Language. (Manning & Schutze).
 Bayes Decision Theory
• Bayes Decision Theory gives a framework for
Generative and Discriminative approaches.
• Current Wisdom:
(i) Discriminative methods are simpler,
computationally faster, and easier to apply.
(ii) Generative methods are needed for most
complex problems.
• Hybrid methods are increasingly popular.
• Stat 231 concentrates on Discriminative Methods
and simple Generative Models.
• Other courses by Prof.s Zhu & Yuille deal with
complex Generative Models.
 Learning Theory.
• Both Generative and Discriminative
methods require training data to learn the
models/features/decision rules.
• Machine Learning concentrates on
learning discrimination rules.
• Key Issue: do we have enough training
data to learn?
 Course Elements.
• Bayes Decision Theory as theoretical
basis.
• Simple discriminative and generative
methods.
• Machine Learning.
• Advanced Discriminative Methods.