Lec-4 : Linear

Regression -
3

evaluation metric
-

Adjusted R2 Score -

-
Statsmodel Implementation
-

Assumptions of Linear Regression

-
Assumption of Linearity
Recap
-

L = lyi - y e

L
W wo
*
=
arguin
i ,
Wo

i
x T(y y)
e Lz

lyi - Yildi
2
2
-

&
-
-
& Fr
E T
-

aw , GW2

↳
GW .
I
-

Elyi - iel n
update rule :
With
it-y L

with Not-ycwo
 N

y .

.
y
2 -

In I XW + Wo

Is

i
 Importance of feature scaling

= If we want weights to give feature importance

similar scale-
then the feature should
be
of
might dominate the
do not scale one feature
* If we

other features

* Scaled features >

- GD will be faster
l
X d

I
-

-
fi
d f2
>
I
fu L
 fi =
km-driven f2 =

age
fa
fod d

- -to-
o
. -

·
f - S

When features fl
not scaled
features are

are
normalized
 R-2
Adjusted
Score - variant
of R-2 scre

irrelevant feature detaset

,
of If we
add an
to our

to R-2 Score ?
what will happen

Ans) R2 Score either increase or remain same

Mythe
↳ Met a
Wid , ten ,

reduce
want s in
Meg might
to Zero
 used and
Eg -

predicting car
selling prices you

"ice sales the

incorporporated a feature cream at

time
of manufacture"

solu weird
correlation
age as
price Y↑

ice-cream sales
-
age
-
R2 can also go up
Goal : To heate c metric that penalizes the

feature
irrelevant to
.
the model
incorporation of an

[
1 (1 R2) (n-1)
-

Adjusted
-

Rascose =

(n -
d -
1)
-

R2 - R
data points R2 R2 R2 scose
n #
of
=
= =

d = #
of features
 to Adj R2 'd'increases ?
Of what happens .
as

di n - d- 1d +4 1 - t d
S ,
n -d -1 & u-d-1

=> as d 4 adj .
R2 score should decrease
, .

which does not

Of What happens when we add a feature
R2 score ?
impoore
decrease
Ans Adj R2 score will
 which
of what happens if we add a
feature
the model performance ?
improves -

R2 ↑ and di
Ans

R↑ (I-R21 1 - (1 - R4) * * CdjRL score

,

R24 adj R2 scose 4

& adj R2 scored

#
Implement the Adj .
R2 score in
Python
R2 d
inputs = score
,
n
,
 Quiz R2 I
Ste
=
-

-M In -
AdiR2 = 1
(n - d -

1)

duz) Range of values

of adjusted R2 score

adj Rascos =
I (man)
zok)if R2 =
1 =

O
adiR2 Score
-

d+ 1
=

if n = =

R2 E C-8 1]
Adj
,
.
Sure
Stats model Implementation =

Add two
= Wili t Wade
....

fif2f ... fd fdtt Rid+ 1

-

① 21 4 ,2 . .

Red 1
⑲ = W, 2 TW2 M12 . . Walled +Wo (1)
② I

1
④ ↓ in this case we can use wo like a

1
④ which has
for affth feature
weight
i value = 1 for all the data points .

I does this
Stab model
* linear regression
implementation
OLS model >
-

Ordinary least Squares + minimize MSE

MSt * (Yi -ii)

goal minimize -
:
i =1
v

Squared error

↓
minimite

used statisticians
stats model >
-

by
lot statistical data that be
>
-
a
of
used to evaluate the model .
 age
- =

Min Man Scaler()

[
Man I Scaler =

E
= 10

Mi -

~
X-train
X-train : Scaler fit-transform (X-train)
.

C ↳
& Ex-test learns Scales the

↳ Max min
, data
all these data
scaled X-test = Scaler transform (X-test)
points will be
d
10 data
using testing
max =

scale the
min
=
I
values
min, mon
which were learnt from using date
data ? learnt from training
training
 normalization ?
the date before doing
of why are we
splitting
be unseen
and don't
Ans) we want Xetest to

test to
flow from
want any information
did
train which Can happen if we

normalization before splitting

.
- --

Scaler fit-transform (ain)

6)
.

m
eman- . . .

object >
- attributes
↳ min--

Scaler .
T(X-fest)
 Scale . transform (4-tet)

lab single unseen

data point , can me

say you get a

Information to normalize
to scale.
use this individual
date -
itself
always comes from the training
 Statsmodel summary

+ P < /t 10 .
025 0 .
975]
Coef stderror

7 2 4 5
:

O
-

-8 293
-
.

5: 9
:

0 7
km-driven
- .

95 % CF
of
the weight of
the feature

E
-
n : 2 -5 9 .
-6 . 2 - -- I
, ,

Y
d
Std . error -
95 % CI T
 statsmodel
Hypothesis test run
by
:

Ho : Feature is not relevant Weight =

O

Ha : feature is relevant = weight +O

E-ni2,5 9 -6 2 I
Su .
. - --

:
,

P < It
P-value D

20 5
P-value
.
Assumptions of Linear Regression

Linear Regression

L d

Statistics optimization
-
>
-

Assumptions ar required assumptions more

no
Linearity needed

I
-

multicollinearity
&

No
-

valid
-

Normality of residuals
if all assumptions an

-
No Heteroskadasticity we have a
very
Linear

Autocorrelation
-
No
regression .
Assumption
of Linearity
price
Prize T
-

a
i
-

- +

a
,

-
X

>

Run-driven
-
-
There is a linear relationship Non-hear relationship
and feature
b) ~ the target

relationship blu target and

· There should be a linear
the features
 satisfied the model will
>
-

If the assumption is then

us better performance
give

quit) fic Ac ,
fs fu,

m
-

Uncar related non-linearly

X
v

which features to use for in

Reg
O) How do check for linearity ?
you

the feature and target

check 1 :
Scatterplot of
creation blu feature and target
checke : Check
 -in
Doubts
-

variance