Deep Learning

Deep Learning is a new area of Machine Learning research, which has been introduced with the
objective of moving Machine Learning closer to one of its original goals: Artificial Intelligence.

Motivation — Biological Neuron-

Dendrite: Receives signals from other neurons

Soma: Processes the information

Axon: Transmits the output of this neuron

Synapse: Point of connection to other neurons

McCulloch-Pitts Neuron(MP Neuron)-

The first computational model of a neuron was proposed by Warren MuCulloch (neuroscientist)
and Walter Pitts (logician) in 1943.

It may be divided into 2 parts. The first part, g takes an input (ahem dendrite ahem), performs
an aggregation and based on the aggregated value the second part, f makes a decision.

Inputs for the above problem could be

x1 — IsRainingTomorrow (Whether it's going to rain tomorrow or not)

x2 — IsScifiMovie (I like science fiction movies)

x3 — IsSickTomorrow (Whether I am going to be sick tomorrow or not depends on any
symptoms, eg: fever)

x4 — IsDirectedByNolan (Movie directed by Christopher Nolan or not.) etc….

In this scenario, if x3 — IsSickTomorrow is equal to 1, then the output will always be 0. If I am not
feeling well on the day of the movie then no matter whoever is the actor or director of the
movie, I wouldn’t be going for a movie.

The MP Neuron model is implemented according to the Six jars of Machine Learning-

1) Data-

The MP neuron takes binary input and gives binary output. If the input data is not binary it can
be compacted to binary before it can be feeded to the model.

Let's take an example of buying a phone based on some features of the features in the binary
format. { y — 0: Not buying a phone and y — 1: buying a phone}

2) Classification-

The classification is also binary which is 0 or 1. The model can give a yes or no answer based on
the input and the threshold.
3) Model-

It consists of a function with a single parameter. The input is aggregated(g). There is a threshold
value which is decided. If the value of the function is equal to or greater than the threshold
value, it gives a positive output and vice versa.

The MP Neuron model basically draws a line where positive values lie on the line or above the
line whereas the negative values below the line.
After finding the best threshold value b from the learning algorithm, we can evaluate the model
on the test data by comparing the predicted outcome and the actual outcome.

4) Learning -

Learning in MP Neuron consists of finding the threshold value with lowest error for prediction.
This is done with brute force for a single parameter.

The purpose of the learning algorithm is to find out the best value for the parameter 'b' so that
the loss of the model will be minimum. In the ideal scenario, the loss of the model for the best
value of b would be zero.

we can use the brute force approach to find the best value of b.

Initialize the 'b' with a random integer [0,n]

For each observation

Find the predicted outcome, by using the formula in the fig above.
Calculate the summation of inputs and check whether its greater than or equal to 'b'. If its
greater than or equal to 'b', then the predicted outcome will be 1 or else it will be 0.

5) Loss function-

The squared loss function is applied. It finds the difference between the predicted value and the
actual prediction as a square.

6) Accuracy-

Accuracy is given by the standard matrix of the division of the number of right predictions by the
total number of predictions.

The MP Neuron basically helps to find a line that separates the positive value from the negative
ones.

The disadvantages of MP Neuron are-

1. Boolean Inputs.

2. Boolean Outputs.

3. Threshold b can take only a few possible values.

4. All the inputs to the model have equal weights.

Summary-
1. Intialize weights

2. Find value of b(bias) by using learning function.

3. Calculate the loss, if loss is greater. find different value of b.

4. Check for the accuracy of the model you created.

References-
https://hackernoon.com/mcculloch-pitts-neuron-deep-learning-building-blocks-7928f4e0504d

https://medium.com/datadriveninvestor/mp-neuron-and-perceptron-98abbd542e47

https://padhai.onefourthlabs.in/