Artificial Neural Networks - 6

Dr. Aditya Abhyankar

Today

Pattern Association. continued

Fundamentals revisited
Autoassociative Nets
Hopfield Nets
BAMs
MATLAB coding

Pattern Association
Learning process of forming associations
between related patterns
Heteroassociative NNs
Autoassociative NNs
Hopfield Net

Linear Vector Spaces

A linear vector space X set of vectors
defined over scalar field F,
1.Vector addition x X , y X , x y X
2. x y y x
3. ( x y) z x ( y z )
4.There is a unique vector called zero
vector, x x, x X

Linear Vector Spaces

5. For each vector, x ( x) 0, x X

6. a F , x X , ax X
7. a F , b F , x X , a(bx) (ab) x
8. (a b) x ax bx
9. a( x y ) ax ay

Linear Vector Spaces

Linear Independence
Spanning a space
Inner Product
Norm
Orthogonality

Linear Vector Spaces - Ex

LI: x1=[1 1]T, x2=[1 -1]T
LI: [1 1 1], [1 0 1], [1 2 1] Ts
LI: [1 1 1 1], [1 0 1 1], [1 2 1 1] Ts

Linear Vector Spaces

Eigenvalues
Eigenvectors
Significance:

A: X X

Az z
Ex: A cos sin
sin cos

1 1
A

Autoassociative Net

Concept
Ex: [1 1 1 1]
Test with one mistakes / missing
Test with two mistakes / missing
Store [1 1 -1 -1] and [-1 1 1 -1]
Limit!!!

Autoassociative Net

Store [1 1 -1 -1] and [-1 1 1 -1]

Limit!!!
Store [1 -1 -1 1] and [1 1 1 1]
With no self-connection Ex: [1 1 1 -1]
Ex: [1 1 -1 -1], [-1 1 1 -1], [-1 1 -1 1], [1 1 1
1]
Storage capacity theorem

Autoassociative Net

Autoassociative Net

Recurrent Nets
[1 1 1 -1]
Three missing components!!
2 mistakes: [-1 -1 1 -1]
Fixed point cycle!!

Discrete Hopfield

Discrete Hopfield

Discrete Hopfield

Ex: [1 1 1 0]
Let W be bipolar
Let update bin order be y1,y4,y3,y2
Test: [0 0 1 0]

BAM

BAM