The Catholic University of America

CSC 551 Pattern Recognition

Lecture 1

Instructor
Lin-Ching Chang

1
Something you want to know….
 Grading
 Homework, Quiz & Class participation
 Exam
 1 midterm and 1 final examinations
 Term project
 Project presentation (10%)
 Final report & code (20%)

 Weight (%)
 Homework & Quiz 20%
 Project 30%
 Midterm 20%
 Final Exam 30%

2
Textbooks (option)
 Easy to read but limited information.

3
Textbooks (option)
Pattern Recognition and Machine
Learning (Information Science
and Statistics) by Christopher M.
Bishop
Introduction to Statistical Pattern
Recognition (Computer Science &
Scientific Computing) 2nd Edition
by Keinosuke Fukunaga
4
Textbooks (option)
Pattern Classification (2nd ed)
by R. O. Duda, P. E. Hart and
D. G. Stork, John Wiley &
Sons, 2000
Pattern Recognition, Fourth
Edition by Sergios Theodoridis
and Konstantinos Koutroumbas,
Academic Press, 2008
5
Machine Learning
 Learning/training: build a classification or regression rule
from a set of samples

 Prediction: assign a class or value to new samples

6
What is pattern recognition?
 The task: we assign labels to unknown objects.
 Assign unknown objects – patterns – into the correct
class. This is known as classification.
 The question: P belongs to which class?

7
What is pattern recognition?
Definitions from the literature
 “The assignment of a physical object or event to one of several pre-
specified categories” –Duda and Hart
 “A problem of estimating density functions in a high-dimensional
space and dividing the space into the regions of categories or
classes” – Fukunaga
 “Given some examples of complex signals and the correct
decisions for them, make decisions automatically for a stream of
future examples” –Ripley
 “The science that concerns the description or classification
(recognition) of measurements” –Schalkoff
 “The process of giving names ω to observations x”, –Schürmann
 Pattern Recognition is concerned with answering the question
“What is this?” –Morse
8
What is pattern?

 A pattern is an object, process or event that can be

given a name.
 A pattern class (or category) is a set of patterns sharing
common attributes and usually originating from the
same source.
 During recognition (or classification) given objects are
assigned to prescribed classes.
 A classifier is a machine which performs classification.

9
Examples of pattern recognition tasks

 How do you classify human into male and

female?
 How do you classify human into tall and short?

10
Machine Perception

 Build a machine that can recognize patterns

automatically
 Applications
 Speech recognition
 Fingerprint identification
 OCR (Optical Character Recognition)
 DNA sequence identification
 Image recognition
 Medical diagnose
 …
11
Application: Speech Recognition

 Some of the best example of speech recognition

products are Google Voice, Siri, Cortana etc.

 You simply speak out the message and it will be

converted to text.

 However, at times, you would realize, speech

recognition doesn’t perform accurately.

12
Application: Fraud and Risk
Detection
 Over the years, banking companies learned to
divide and conquer data via customer profiling,
past expenditures and other essential variables
to analyze the probabilities of risk and default.

 Moreover, it also helped them to push their

banking products based on customer’s
purchasing power.

13
Application: Image Recognition
 You upload your image with friends on Facebook and
you start getting suggestions to tag your friends. This
automatic tag suggestion feature uses face recognition
algorithm.

 Similarly, while using whatsapp web, you scan a barcode

in your web browser using your mobile phone.

 Google provides you the option to search for images by

uploading them. It uses image recognition and provides
related search results.

14
Application: Character recognition

 Automated mail sorting, processing bank checks

 Scanner captures an image of the text
 Image is converted into constituent characters

15
16
Application: Finger prints recognition

17
Application:
Image Segmentation/
Brain Segmentation

18
Application: Brain Tissue Segmentation

19
Sony IPT-DS1 Party-Shot
What’s the definition
of a smile?

Take a shot when

you see a smile
20
Jobs Rated Report 2016:
Ranking 200 Jobs
 http://www.careercast.com/jobs-rated/best-jobs-
2016
 The high demand for data scientists and
statisticians comes from a growing emphasis on
collecting and evaluating massive quantities of
data.

21
Data Scientist – again, best job in
America for 2017

22
Software Developer - best job in
America for 2018

23
24
 Components of a pattern recognition system

 A basic pattern classification system contains

 A sensor (for data collection)
 A preprocessing mechanism
 A feature extraction mechanism (manual or automated)
 A classification algorithm
 A set of examples (training set) already classified or described

25
The pattern recognition design cycle

 Data collection
 Probably the most time-intensive component of a PR
project
 How many examples are enough?

 Attention: Don’t spend your time in collecting the

data yourself for your term project !!

26
Data sets can be used for pattern
recognition project
 UC Irvine: more than 200 sets available for downloading
 http://archive.ics.uci.edu/ml/datasets.html

27
The pattern recognition design cycle
 Data collection
 Probably the most time-intensive component of a PR project
 How many examples are enough?
 Feature choice
 Critical to the success of the PR problem
 “Garbage in, garbage out”
 Requires basic prior knowledge
 Model choice
 Statistical, neural and structural approaches
 Parameter settings
 Training
 Given a feature set and a “blank” model, adapt the model to explain
the data
 Supervised, unsupervised and reinforcement learning
 Evaluation
 How well does the trained model do?
 Overfitting vs. generalization
28
 Consider the following scenario
 A fish processing plan wants to automate the process of sorting
incoming fish according to species (salmon or sea bass)
 The automation system consists of
 a conveyor belt for incoming products
 two conveyor belts for sorted products
 a pick-and-place robotic arm
 a vision system with an overhead CCD camera
 a computer to analyze images and control the robot arm

29
From [Duda, Hart and Stork, 2001]
An Example from the Book

 Goal: sorting incoming fish on a conveyor

according to species using optical sensing

Sea bass
Species
Salmon

The objects to be classified; a. Sea bass, and b. salmon.

Data Collection
Problem Analysis

 Set up a camera and take some sample images to

extract features
 Length
 Lightness
 Width
 Number and shape of fins
 Position of the mouth, etc…

 This is the set of all suggested features to explore for

use in our classifier!
Preprocessing
 Use a segmentation operation to isolate fishes
from one another and from the background

 Information from a single fish is sent to a feature

extractor whose purpose is to reduce the data by
measuring certain features

 The features are passed to a classifier

Pattern Classification, Chapter 1

33
Classification

 Select the length of the fish as a possible feature

for discrimination

 Is the length a good feature?

Classification

 Select the lightness of the fish as a possible

feature for discrimination

 Is the lightness a good feature?

 Threshold decision boundary and cost
relationship

 Move our decision boundary toward smaller

values of lightness in order to minimize the
cost (reduce the number of sea bass that are
classified salmon!)

Task of decision theory

Pattern Classification, Chapter 1

36
 Adopt the lightness and add the width of the fish
 Combining two features for classification

Fish xT = [x1, x2]

Lightness Width
Next class

 Background review
 linear algebra
 probability and statistics

39
Term project

 Find the data for your project

 Find a topic
 Write a one-page proposal to describe your data
and the task

40
The Catholic University of America
Fall 2018

CSC 551 Pattern Recognition

Setup Python Environment

Instructor
Lin-Ching Chang

41
Step 1
 Download the Python 3.X version of the Anaconda distribution for
your operating system.
 https://www.anaconda.com/download/

 You will avoid a lot of install-related headaches by choosing this

pre-bundled distribution. It comes with most of the important data
analysis packages pre-installed.

 Anaconda package lists

 https://docs.anaconda.com/anaconda/packages/pkg-docs

42
Step 1
 If you already have Python installed. You can install other 3rd party
library by using tools such as pip or conda

 You can still install Anaconda if you have Python installed already.
 Anaconda is installed in its own directory, so both installations
shouldn't interfere at all.
 The only problem you could encounter is if you add Anaconda to
your PATH environment variable, which will make it the default
Python in your system.
 There's an option in the installer which let's you decide if you
want to accept that change or not.

43
Python Libraries you may need
 Scikit-Learn
 Machine Learning in Python.

 Numpy
 Numpy will help you to manage multi-dimensional arrays very
efficiently.
 Scipy
 Numerical routines such as integration, interpolation,
optimization, etc.
 Many routines in Scikit-Learn library highly relies on it.

 Pandas
 Pandas is the “SQL of Python

 To analyze data, we like to use two-dimensional tables – like in

SQL and in Excel. Originally, Python didn’t have this feature.
 Matplotlib
 data visualization 44
Step 2

 Install PyCharm (an IDE written in Java)

 Download the community edition of Pycharm for
your operating system
 https://www.jetbrains.com/pycharm/download/

 Create a new Python project to make sure your

installation is successful
 I will not go with you step by step on this, you can
find many tutorials on youtube

45
Step 3
 Installing Packages (if you still need some more)

 Method 1: I would recommend installing packages via

the command line (it is far more reliable). You can do
either a conda install or a pip install

 Method 2: You can also install modules from pycharm

itself. I don’t have experience in doing it
 pycharms website provide instructions on how to do it
 https://www.jetbrains.com/help/pycharm/project-interpreter-
2.html

46
 You are good to go

47