Exercise #1

Anirban Bhowmick ID:7251843

Machine Learning Course
University of Hamburg

May 2, 2020

Question 1
Generate 100 artificial data points (xi , yi )where each xi is randomly generated
from the interval [0, 1] and yi = sin(2πi) + . Here, θ is a random noise value in
the interval [0.3, 0.3]

Question 2
Implement in your favorite programming language the Stochastic Gradient De-
scent algorithm to solve the regression problem using the 100 data points you
generated.
Loop
{
for i = 1 to m {
θj := θj + α[yi −hθ (xi )](xi )j (for every j).
}
}

Question 3
Make your initial learning rate constant = 0.001, and train a polynomial model
using your artificially created data. A polynomial model has the form h (x) =
θ0 + θ1 x + θ2 x2 + ... + θD xD , where D is the degree of the polynomial.

Question 4
All initial i parameters are randomly generated in the interval [0.5, 0.5]

Question 5
Try different values for D

1
 Question 6
Try different values to speed up the learning process

Polynomial function and Final model

1 # generate the ouput of the polynomial h0 ( x ) and return
2 def g e n e r a t e P o l y n o m i a l ( deg , train _ ex ) :
3 x = np . array ([ X * * i for i in range ( deg +1) ])
4 res = np . dot ( th , x )
5 return res
6
7
8 # generate the model
9 epoch = 0
10 learning _ rate = 0.1
11 degree = 3
12 th = np . random . uniform ( low = -0.3 , high =0.3 , size =( degree +1) )
13 E _ RMS = []
14 while ( epoch <5000) :
15 for i in range ( len ( X ) ) :
16 h = g e n e r a t e P o l y n o m i a l ( degree , i )
17 polyFactorArr = np . array ([ X [ i ] * * j for j in range ( degree +1) ])
18 diff = ( Y [ i ] - h [ i ]) * polyFactorArr
19 th = th + learning _ rate * diff
20 # Error = Error + (( h [ i ] - Y [ i ]) * * 2)
21 Error = np . sum ( np . square (h - Y ) )
22
23 epoch +=1
24 Error = 0.5 * Error
25 E _ RMS . append ( math . sqrt ((2 * Error ) / len ( X ) ) )

0.1 Final α value

α = 0.1, D = 3, iterations(Epoch) = 5000

2
0.2 graph containing the cloud of the training data points, the
sine function, and the learned polynomial function

3
0.3 A second graph showing the error curve