Questions

1. What is the role of logit and sigmoid functions in logistic

regression?

Consider a car and its price is Rs 4.5lakhs with an

additional feature called a Sunroof.

and its value is 1 if it is present .

Find the probability of sell for this car?

Given that the coefficients b0=0.39 and b1=0.173 and b2=1.55

------------------------------------------------------------------------------------
----

We know that log(p/1-p) = b0+b1*price+b2+Sunroof

 =0.764

Hence, the car has a probability of 76.4%

chances of a sale.
Question 2
The data set of pass/fail of in an exam for 5
students is given in the following table.
Hours studied Result (1 pass 0 fail)
29 0
15 0
33 1
28 1
39 1
1. What is the role of sigmoid function and
logit function in logistic regression?
a. Calculate the probability of pass for
the student who studied 33 hours
b.At least how many hours the student
should study that makes sure will pass the
course with the probability of more than
95%
Assume log(odds)= -64+2*hours

Z=2
P=0.88
A student who studied for 33 hours has
88% chance of passing the course.

b.Given that P=0.95

 =0.952

Student needs to study for approximately 33.5 hours to achieve

more than 95%

Other supporting material

Logistic Regression in Machine Learning
o Logistic regression is Supervised Learning technique. It is used for predicting
the categorical dependent variable using a given set of independent variables.
o Logistic regression predicts the output of a categorical dependent variable.
Therefore the outcome must be a categorical or discrete value. It can be either
Yes or No, 0 or 1, true or False, etc. but instead of giving the exact value as 0
and 1, it gives the probabilistic values which lie between 0 and 1.
o Logistic Regression is much similar to the Linear Regression except that how
they are used. Linear Regression is used for solving Regression problems,
whereas Logistic regression is used for solving the classification problems.
o In Logistic regression, instead of fitting a regression line, we fit an "S" shaped
logistic function

Logistic Function (Sigmoid Function):

 The sigmoid function is a mathematical function used to map the predicted

values to probabilities.
 It maps any real value into another value within a range of 0 and 1.
 The S-form curve is called the Sigmoid function or the logistic function.
 o We use the concept of the threshold value, which defines the probability of
either 0 or 1. Such as values above the threshold value tends to 1, and a value
below the threshold values tends to 0.

It’s an S-shaped curve that can take any real-valued number and map it into a value
between 0 and 1.
1 / (1 + e^-value)

Where e is the base of the natural logarithms (Euler’s number or the EXP() function in
your spreadsheet) and value is the actual numerical value that you want to transform.
Below is a plot of the numbers between -5 and 5 transformed into the range 0 and 1
using the logistic function.
The odds equals the probability that Y=1 divided by
the probability that Y=0. For example, if the
probability that Y =1 is 0.8 then the probability that
Y=0 is 1-0.8 or 0.2
Odds = P(Y=1)/P(Y=0) = 0.8/0.2 = 4
Log odds = Ln(Odds) = Ln(P(Y=1)/P(Y=0)) =
Ln(P(Y=1)/[1-P(Y=1)])

Logistic Regression uses logit() to classify the outcomes.

If the probability of an event occurring (P) and the probability

that it will not occur is (1-P)

Odds Ratio = P/(1-P)

Taking the log of Odds ratio gives us:

Log of Odds = log (p/(1-P))

This is the logit function

Probability ranges from 0 to 1

Odds range from 0 to ∞

Log Odds range from −∞ to ∞.

The inverse of the logit function is the sigmoid function. That is, if
you have a probability p, sigmoid(logit(p)) = p. The sigmoid
function maps arbitrary real values back to the range [0, 1]

Sigmoid function

Sigmoid is a mathematical function that takes any real number

and maps it to a probability between 1 and 0.

Representation Used for Logistic Regression

where:

p = probability that y=1 given the values of the input

features, x.
x1,x2,..,xk = set of input features, x.
B0,B1,..,Bk = parameter values to be estimated via
the maximum likelihood method.
B0,B1,..Bk are estimated as the ‘log-odds’ of a unit
change in the input feature it is associated with.
Bt = vector of coefficients
X = vector of input features
Estimating the values of B0,B1,..,Bk involves the
concepts of probability, odds and log odds

Probability ranges from 0 to 1

Odds range from 0 to ∞
Log odds range from -∞ to +∞
Consider the dataset

The crosstab of the variable hon with female shows that there are 109
females and 91 males; 32 of those 109 females secured honours.

Probability:
The probability of an event is the number of instances of that event divided
by the total number of instances present.

Thus, the probability of females securing honours:

=0.29

Odds:

The odds of an event is the probability of that event occurring (probability

that y=1), divided by the probability that it does not occur.

Thus, the odds of females securing honours:

0.42

This is interpreted as:

32/77 => For every 32 females that secure honours, there are 77 females
that do not secure honours.
32/77 => There are 32 females that secure honours, for every 109 (ie
32+77) females.

Log odds:

The Logit or log-odds of an event is the log of the odds. This refers to the
natural log (base ‘e’). Thus,

Thus, the log-odds of females securing honours:

Q: Find the odds ratio of graduating with honours for females and males.

Calculations for probability:

Where :

B0,B1,..Bk are estimated as the ‘log-odds’ of a unit change in the input

feature it is associated with.
As B0 is the coefficient not associated with any input feature, B0= log-odds
of the reference variable, x=0 (ie x=male). ie Here, B0= log[odds(male
graduating with honours)]
As B1 is the coefficient of the input feature ‘female’,
B1= log-odds obtained with a unit change in x= female.
B1= log-odds obtained when x=female and x=male.

Calculations:

From the calculation in the section ‘odds ratio(OR)’,

B1= log (1.82)

B1= 0.593
Thus, the LogR equation becomes

y= -1.47 + 0.593* female

where the value of female is substituted as 0 or 1 for male and female

respectively.

Now, let us try to find out the probability of a female securing honours when
there is only 1 input feature present-‘female’.

Substitute female=1 in: y= -1.47 + 0.593* female

Thus, y=log[odds(female)]= -1.47 + 0.593*1 = -0.877

As log-odds = -0.877.
Thus, odds= e^ (Bt.X)= e^ (-0.877)= 0.416
And, probability is calculated as:
Suppose we want to calculate the effect of being female on the probability
of graduating with honours.

The task is to predict which students graduated with honours or not (y = 1

or 0), for 200 students with fields female, read, write, math, hon,
femalexmath . The fields describe the gender (female=1 if female), reading
scores, writing scores, math scores, honours status (hon=1 if graduated
with honours) and femalexmath showing the math score if female=1.

A key difference from linear regression is that the output value being modeled is a binary
values (0 or 1) rather than a numeric value.
Below is an example logistic regression equation:

y = e^(b0 + b1*x) / (1 + e^(b0 + b1*x))

Where y is the predicted output, b0 is the bias or intercept term and b1 is the coefficient
for the single input value (x).

Making Predictions with Logistic Regression

Making predictions with a logistic regression model is as simple as plugging in numbers
into the logistic regression equation and calculating a result.

Let’s make this concrete with a specific example.

Let’s say we have a model that can predict whether a person is male or female based on
their height (completely fictitious). Given a height of 150cm is the person male or female.

We have learned the coefficients of b0 = -100 and b1 = 0.6. Using the equation above
we can calculate the probability of male given a height of 150cm or more formally
P(male|height=150). We will use EXP() for e, because that is what you can use if you
type this example into your spreadsheet:
 y = e^(b0 + b1*X) / (1 + e^(b0 + b1*X))

y = exp(-100 + 0.6*150) / (1 + EXP(-100 + 0.6*X))

y = 0.0000453978687

Or a probability of near zero that the person is a male.

0 if p(male) < 0.5

1 if p(male) >= 0.5

---------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------