Professional Documents
Culture Documents
Neural Networks and Particle Swarms
Neural Networks and Particle Swarms
By Jeremy Nunn
1
NN Math Why use NNs?
• Looking at some neuron, U, and its threshold TU, we • Have been used in speech synthesis,
calculate the activation value AU as
diagnostic problems, computer vision
AU = w1Ux1 + w2Ux2 + … + wnUxn
• Can be used when a rule-based expert
∑
n
= wiU xi
i =1 system would be too complicated to
where wij is the weight value of the edge from input i design
to neuron j. • Can be used to recognize complicated
• The neuron fires (output is 1.0) if AU > TU, usually patterns, even with noisy input
• Other activation functions exist, such as just allowing
the output to be AU, or mapping RÎR (i.e. sigmoid) • Can use parallel algorithms
2
PSO Algorithm PSO Algorithm
For each particle
• Velocity as Initialize particle
END
v v v v
v = v + c1 (r1 ⊗ ( pbest − present )) + c2 (r2 ⊗ ( gbest − present )) (1)
Do
For each particle
Calculate fitness value
where c1 and c2 are learning factors, and r1 If the fitness value is better than the best fitness value (pBest) in history set
current value as the new pBest
and r2 are vectors of random values End
between (0,1) Choose the particle with the best fitness value of all the particles as the gBest
For each particle
Calculate particle velocity according equation (1)
Update particle position according equation (2)
End
• Position as While maximum iterations or minimum error criteria is not attained
v
present = present + v (2)
[3]
3
Training (con’t) My Project, Briefly
• Every edge (or weight) in the NN is an • Using a NN to classify songs into genres
element in our particle; if our NN has k • Training the NN on ~500 song samples, in
weights, our particles will have k
.wav format
dimensions
• The fitness value is how well the NN • Initial setup will use 5 outputs (genres)
performs on the test data • Once the NN gets all the test data right, try
• We run the PSO for a given number of non-test data
iterations or until a minimum error is • Compare the NN results with human
reached results to verify correctness
References
5. Boids, http://www.red3d.com/cwr/boids,
Reynolds, Craig