Lecture 26

Artificial Intelligence
CS-3431w (V2)
Instructor: Dr. Syed Mohammad Irteza
Assistant Professor, Department of Computer Science
14 January, 2021
Previous Lecture
• Chapter 18 – Learning From Observation
• Decision Trees
• Which feature to choose first?
• Entropy
• Information Gain

2
Finally
• We choose Patrons (instead of Type and other attributes) as the first
attribute for classification, since it has the highest IG

3
Summary
• Learning needed for unknown environments, lazy designers
• Learning agent = performance element + learning element
• Learning method depends on type of performance element, available
feedback, type of component to be improved, and its representation
• For supervised learning, the aim is to find a simple hypothesis that is
approximately consistent with training examples
• Decision tree learning
• Learning performance = prediction accuracy measured on test set

4
Today’s Lecture
• Artificial Neural Networks

5
Artificial Neural Networks
• Chapter 20, Section 5 (Russell and Norvig)
• Chapter 4, (Machine Learning, Tom Mitchell)

6
Connectionist Models
• Consider humans:
• Neuron switching time: ~0.001 seconds
• Number of neurons: ~1010
• Connections per neuron: ~104−5
• Scene recognition time: ~0.1 second
• Much parallel computation
• Properties of artificial neural networks (ANNs):
• Many neuron-like threshold switching units
• Many weighted interconnections among units
• Highly parallel, distributed process
• Emphasis on tuning weights automatically
7
When to consider Neural Networks?
• Input is high-dimensional discrete or real-valued (e.g., raw sensor
input)
• Output is discrete or real valued
• Output is a vector of values
• Possibly noisy data
• Form of target function is unknown
• Human readability of result is unimportant

8
Examples
• Speech phoneme recognition [Waibel]
• Image classification [Kanade, Baluja, Rowley]
• Financial prediction

9
Perceptron

10
 House Price Prediction
Price

Size of House 11
Neural Network
• Simplest example of a neural network

Size Price

12
Adding Complexity
• Middle layer → Hidden layer (hidden nodes)

𝑥1

𝑥2

Price
𝑥3

𝑥4

13
Benefits of Neural Networks
• In general, given enough training examples with (x, y) pairs, neural
networks are able to compute accurate functions that help us predict
y for future values of x

14
Neural Networks
• Algorithms that try to mimic how the brain functions
• Has worked extremely well to recognize:
• Handwritten characters (LeCun et al., 1989)
• Spoken words (Lang et al., 1990)
• Faces (Cottrel, 1990)
• Extensively studied in the 1990s with some success
• Now, much renewed interest in deep learning, thanks to algorithmic
and computational progress
• The first algorithm used was the Perceptron (Rosenblatt, 1959)

15
Perceptron
Given some number of examples, and 𝑛 features
𝑛

𝑓 𝑥𝑖 , 𝑥2 , … . , 𝑥𝑛 = 𝑠𝑖𝑔𝑛(෍ 𝑤𝑖 𝑥𝑖 )
𝑖=0

16
Perceptron Expressiveness
• Consider the perceptron with the step function
• Idea: Iterative method that starts with a random hyperplane and
adjust it using your training data
• It can represent Boolean functions such as AND, OR, NOT, but not the
function for XOR
• It produces a linear separator in the input space

17
Screenshots

18
19
20
21
22
23
Acknowledgement
• I have taken help for these slides from the work of:
• Book Slides (AIMA, Berkeley)

24