Multiclass

Classification
Multiclass Classification using LR
• Examples of multiclass classification:
• Classification of different fruits: apples, oranges, peaches, bananas
y= 1 2 3 4
• Category of players: gold, silver, platinum
y=1 2 3
• Weather: dry, rain, snow
y= 0 1 2
Multiclass Classification using LR
One vs All

(x)

(x)

(x)
One vs All

(x)

h ( 𝑥 ) =𝑃 ( 𝑦=𝑖| 𝑥 ; 𝜃 ) (𝑖=1 ,2 , 3)
𝑖
𝜃

(x)
• Train an LR classifier for each class i to predict the
probability that y = I
• On a new data point x, to make a prediction, choose
the class i that maximizes the hypothesis h

h1=0.8 (x)
H2 = 0.3
H3=0.55
Regularization
for Lin Reg
and Log Reg
 Regularization for Linear
Regression
• Simple models are prone to underfitting (High bias)
• Not even good for training examples
• Complex models are prone to overfitting (High
variance)
• Too many features
• Works well with training examples
• Not good for test example (not generalizable)
• Example of housing price predictor
Regularization for Linear Regression
• Consider we have only one feature, that is the size of house to predict the price.
• Suppose we penalize and make very small.
• +
• Example:
• Features: x1, x2, …, x100
• Parameters: 0, 1, 2, …, 100

• In regularization, we select small values of , , … to avoid overfitting.

Regularization for Linear Regression
• Gradient Descent
 𝑖
𝑥 𝑗

𝑋 =¿
𝑚
1 𝜆 2
𝜃2=𝜃 2 −𝛼 [ ∑ ( h 𝜃 ( 𝑥 ) − 𝑦 ) 𝑥2 + 𝜃 𝑗 ]
1 1 1
2 𝑖=1 𝑚
Regularization for Logistic Regression
• Gradient descent for Log Reg w/o regularization:

• Gradient descent for Log Reg with regularization:

𝑚
𝜃0 =𝜃 0 − 𝛼 ∑ ( h 𝜃 𝑥 − 𝑦 ) 𝑥0 ( 𝑗=0)
( 𝑖
) 𝑖 𝑖

𝑖=1