Computational Methods in Electrical and Electronic Engineering

Report 1
DO LE DUY・ 1026322038
ECS-ID： a0192284

Introduction
We use Python with Jupyter Notebook on Visual Studio Code to solve the tasks in this report. We will only use matplotlib for
plotting usage and no other 3rd-party libraries. Instead, we will define some helper functions for general computing as follow:

In [ ]:
import matplotlib.pyplot as plt

# l2 norm
def norm(v):
import math
val = 0
for i in range(len(v)):
val += v[i] ** 2
return math.sqrt(val)

Task 1.1
The 4 by 4 matrix A is defined as follow:

2 3
1 x0 x x
⎛ 0 0 ⎞
2 3
⎜1 x1 x x ⎟
⎜ 1 1 ⎟
A = ⎜ ⎟
⎜ 2 3 ⎟
⎜1 x2 x
2
x
2 ⎟

⎝ 2 3 ⎠
1 x3 x x
3 3

Let x 0
= 1, x1 = 4/3, x2 = 5/3, x3 = 2 , then the value of A is calculated to be:

In [ ]:
x = [1, 4.0/3.0, 5.0/3.0, 2]
A = [[x[j]**i for i in range(4)] for j in range(4)]
A

[[1, 1, 1, 1],
Out[ ]:
[1.0, 1.3333333333333333, 1.7777777777777777, 2.37037037037037],
[1.0, 1.6666666666666667, 2.777777777777778, 4.629629629629631],
[1, 2, 4, 8]]

Task 1.1.1
The product of a matrix A with a column vector x is :
ij j

y = (Ax)i = ∑ Aij vj
i

j=1

which is equivalent to

i i j
y = A x
j

Thus, we will implement a subroutine of matrix-vector multiplication as follow:

In [ ]:
# matrix and column vector multiplication
def mul_mat_vec(A, x):
rs = len(A)
cs = len(A[0])
 y = [0]*rs
for i in range(rs):
for j in range(cs):
y[i] += A[i][j] * x[j]
return y

We test the subroutine by multiplying A with an ones-vector to confirm the correctness of the implementation.

In [ ]:
mul_mat_vec(A, [1] * len(A))

[4, 6.481481481481481, 10.074074074074076, 15]

Out[ ]:

Task 1.1.2
Let V be a matrix with column vector v such that f
i
0 (x) = (x − x1 ) (x − x2 ) (x − x3 ) = v
0

0
0
+ v x + v x
1
0

2
2 0
+ v x
3
3
, etc., then
we would have:

2 3 0
1 x0 x x v
⎛ 0 0 ⎞⎛ 0 ⎞ f0 (x0 ) f0 (x0 )
⎛ ⎞ ⎛ ⎞
2 3 0
⎜1 x1 x x ⎟ ⎜v ⎟
⎜ 1 1 ⎟ ⎜ 1 ⎟ ⎜ f0 (x1 ) ⎟ ⎜ 0 ⎟
⎜ ⎟
AV [0] = ⎜ ⎟ ⎜ ⎟ = = ⎜ ⎟
⎜ 0 ⎟ ⎜ ⎟
⎜ 2 3 ⎟ ⎜ f0 (x2 ) ⎟ ⎜ ⎟
⎜1 x2 x
2
x
2 ⎟ ⎜ v2 ⎟ 0

⎝ ⎠ ⎝ ⎠
⎝ 2 3 ⎠⎝ 0 ⎠ f0 (x3 ) 0
1 x3 x x v
3 3 3

We apply the same for other vector v , we could confirm that:

i

f0 (x0 ) 0 0 0
⎛ ⎞

⎜ 0 f1 (x1 ) 0 0 ⎟
⎜ ⎟
AV = (AV [0] AV [1] AV [2] AV [3]) =
⎜ ⎟
⎜ 0 0 f2 (x2 ) 0 ⎟

⎝ ⎠
0 0 0 f3 (x3 )

Thus, the inverse of A is:

−1
A = (V [0]/f0 (x0 ) V [1]/f1 (x1 ) V [2]/f2 (x2 ) V [3]/f0 (33 ))

We find v and calculate the product between A and v with the rountine below:
i i

In [ ]:
def calc_V(x_ori):
V = []
# find element of vi according to definition in task description
for i in range(len(x_ori)):
# remove one x value from the list
x = x_ori[:i] + x_ori[i+1:]
# find v vector according to definition
V.append([-x[0]*x[1]*x[2], x[0]*x[1] + x[0]*x[2] + x[1]*x[2], -(x[0] + x[1] + x[2]), 1])
return V
V = calc_V(x)
for i in range(len(V)):
print("v{} = {}".format(i, V[i]))
print()
for i in range(len(V)):
print("Av{} = {}".format(i, mul_mat_vec(A, V[i])))

v0 = [-4.444444444444445, 8.222222222222223, -5.0, 1]

v1 = [-3.3333333333333335, 7.0, -4.666666666666667, 1]
v2 = [-2.6666666666666665, 6.0, -4.333333333333333, 1]
v3 = [-2.2222222222222223, 5.222222222222222, -4.0, 1]

Av0 = [-0.22222222222222143, -4.440892098500626e-16, 8.881784197001252e-16, 1.7763568394002505e-15]

Av1 = [-4.440892098500626e-16, 0.07407407407407218, -8.881784197001252e-16, -1.7763568394002505e-15]
Av2 = [4.440892098500626e-16, 1.3322676295501878e-15, -0.07407407407407351, 1.7763568394002505e-15]
Av3 = [0.0, -4.440892098500626e-16, 0.0, 0.22222222222222143]

Task 1.2
 In this task, we are required to find the roots of an equation with numerical methods, namely Bisection Method and Newton's
Method. The equation and its derivative are shown below:

5 4 3 2
f (x) = x + x − x − x − 2x − 2

′ 4 3 2
f (x) = 5x + 4x − 3x − 2x − 2

In [ ]:
def f(x):
return x**5 + x**4 - x**3 - x**2 - 2*x - 2

def df(x):
return 5*x**4 + 4*x**3 - 3*x**2 - 2*x - 2

Task 1.2.1
Bisection Method

The bisection method finds the roots of a function by repeatedly bisecting the interval defined by two values x and x and then L R

selecting the subinterval so that there is a zero crossing i.e the root is within the new interval.

The method start by inputing the interval x - x such that f (x

L R L
) , f (xR ) would have opposite sign. Then, each iteration
performs the following steps:

Set

xM = (xR + xL ) /2

if f (x L
) , f (xM ) have same sign, set:

xL = xM

else, set:

xR = xM

The loop is finished when f (x) is sufficiently small.

We could also set the stopping condition to be when the interval becomes sufficently small. This could also be achieved by
setting a maximum interation as the number of iteration could be determined from the interval condition:

(0) (0)
∣ ∣
x − x
(k) (k)
∣ L R ∣
∣ ∣
lim ( x − x ) = lim < ϵ
∣ L R ∣ k
k→∞ k→∞
2

In [ ]:
def bisection(f, xl, xr, e, N, plotting = True):
error_list = []
for count in range(N):
x = (xl + xr)/2

if f(xl) * f(x) < 0:

xr = x
else:
xl = x

# append error value to list if plotting is set to True

# would result in Error if x_real is not defined globally
if plotting:
global x_real
error_list.append(abs(x - x_real))

if abs(f(x)) < e:
break
if plotting:
print("x = {}, stop after {} iters".format(x, count))
plt.plot(range(count+1), error_list)
plt.yscale("log")
plt.show()
 return x

# Input Parameters
xl = 0
xr = 2
e = 1e-10
N = 100

x_real = bisection(f, xl, xr, e, N, plotting = False)

x_real = bisection(f, xl, xr, e, N, plotting = True)

x = 1.4142135623696959, stop after 36 iters

Observing the graph, we could see that the error converge linearly i.e decreases exponentially. In particular:

−k
log (Error) = log (Error)/ log (10) ≈ −k/3.5 ⟹ Error ≈ 2 .
10 2 2

This agree with the fact that the interval decreases by half after each interation:

(0) (0)
∣ ∣
x − x
∣ L R ∣
(k) (k)
∣ ∣
x − x =
∣ L R ∣
k
2

The Newton's Method

The Newton's Method could find successively better approximations to a root of a real-valued function with the recurrence
formula as follow:

(k)
f (x )
(k+1) (k) (k)
x = g (x ) = x −
′ (k)
f (x )

This is because, the tangent of the given function at a given point x (k)
given by the equation

(k) ′ (k) (k)

hk (x) = f (x ) + f (x ) (x − x )

would intersect with the x-axis at point x (k+1)

which is closer to the root than x (k)
. The point x (k+1)
is found by solving for
hk (x
(k+1)
) = 0 , which results in:

(k)
f (x )
(k+1) (k)
x = x −
′ (k)
f (x )

Then, we continue to find better approximation of the root by repeating the step for next points until f (x) is sufficiently small.

In [ ]:
def newton(f, df, x0, e, N, plotting = True):
x = x0

error_list = []
for count in range(N):

x = x - f(x)/df(x) # result in error if derivative becomes equal zero

 # append error value to list if plotting is set to True
# would result in Error if x_real is not defined globally
if plotting:
global x_real
error_list.append(abs(x - x_real))

if abs(f(x)) < e:
break
if plotting:
plt.plot(range(count+1), error_list)
print("Initialize x to be {}, find solution x = {}, stop after {} iters".format(x0, x, count))
plt.yscale("log")
plt.show()

return x

# Input Parameters
x0 = 1
e = 1e-10
N = 100

x_real = newton(f, df, x0, e, N, plotting = False)

x_real = newton(f, df, x0, e, N, plotting = True)

Initialize x to be 1, find solution x = 1.4142135623730954, stop after 8 iters

Differentiate g(α), whereas α is a root, we get:

′ 2 ′′
{f (α)} − f (α)f (α)
′
g (α) = 1 − = 0
2
′
{f (α)}

By using Taylor's expansion we could expression the x (k+1)

as follow:

2
Δx
(k+1) (k) ′ ′′
x = g (x ) = g(α + Δx) = g(α) + Δxg (α) + g (α) + ⋯
2!

As g ′
, ,
(α) = 0 g(α) = α Δx = e
(k)
and other high-order terms are insignificant:

2
(k)
(e )
(k+1) (k+1) ′′
x − g(α) = e = g (α) + ⋯
2!

Therefore, if the higher-order terms become sufficiently small,

2
(k+1) (k)
∣x − α∣ ∣ − α∣
∣ ∣ ≃ C ∣x ∣

As a result, it can be seen that Newton's Method has quadratic convergence. Then, we could express the relation between error
and iteration in log scale as

(k) ′ k
log(e ) = −C  2

Inspecting the convergence graph, we observe that the line generally agrees with the above formula, although we have too few
iterations to reach a concrete conclusion.
 Task 1.2.2
To find other solutions of the equation, we need to input different initial values to the algorithm. We create a list of initial x and
find solution for each value. We plot the Initial Value versus the Convergence Destination and also the function f (x) for
comparison.

In [ ]:
x_init_list = [x/10.0 for x in range(-20, 20)] # Create a list of initial x from -2 to 2
sol_list = [] # Initialize list of solutions

for x0 in x_init_list:
x = newton(f, df, x0, e, N, plotting = False)
x = round(x, 3)
sol_list.append(x)

# Remove duplicates in sol_list and print the unique solutions

print("The roots of the equation are: ", list(dict.fromkeys(sol_list)))

# Plotting
plt.scatter(x_init_list, sol_list)
plt.title("Initial Value vs Convergence Destination")
plt.grid(True, which='both')
plt.show()
plt.plot(x_init_list, [f(x) for x in x_init_list])
plt.title("f(x)")
plt.grid(True, which='both')
plt.show()

The roots of the equation are: [-1.414, -1.0, 1.414]

From the plot, we could observe 4 intervals correlated to 3 convergence destinations.

:
−1.414 (−∞, −1.3] + [0.8, 0.9]

:
−1.0 [−1.2, 0.7]

:
1.414 (1, ∞)
 We initally speculate that the point where convergence destination changes would be related to the extreme points where the
derivative changes sign. We could observe that this is only partially true because there exists the interval (0.8, 0.9) leading to
convergence at −1.414.

Task 1.3
In this task, we need to apply Newton's method to solve a system of nonlinear equations. In particular, we will find solutions to
the following system:

3 2
f1 (x1 , x2 ) = x − 3x1 x − 1
1 2

2 3
f2 (x1 , x2 ) = 3x x2 − x
1 2

, as f 1 (x1 , x2 ) = f2 (x1 , x2 ) = 0 . We could rewrite the problem as

F (x) = (f1 f2 ) = (0 0)

Similar to Newton's method with one variable, from an approximate solution x , we find a better approximation x k k+1
by solving
the following equation:

k k k+1 k
F (x ) + J (x ) (x − x ) = 0,

where J (x k
) is Jacobian of F (x )
k
. We write the equation in the linear system form:

−1
k k
δ = J (x ) F (x ) ,

where δ = −x
k+1
+ x
k
.

Thus, each iteration of Newton's method consists of two steps:

1. Solve the linear system J (x k

) δ = −F (x )
k
with respect to δ.
2. Set x k+1
= x
k
+ δ .

We repeat the process until the norm |F (x)| is sufficiently small.

Task 1.3.1
To apply the Newton's Method, we need to determine the inverse Jacobian matrix of F . First we determine the Jacobian to be:

2 2
3x − 3x −6x1 x2
1 2
J (x) = ( )
2 2
6x1 x2 3x − 3x
1 2

As determinant of J is det J = (3x

2
1
2
− 3x )
1
2
+ (6x1 x2 )
2
, the inverse Jacobian matrix is:

2 2
1 3x − 3x 6x1 x2
1 2
J (x) = ( )
2 2 2 2 2 2
(3x − 3x ) + (6x1 x2 ) −6x1 x2 3x − 3x
1 1 1 2

In [ ]:
def F(x):
return [x[0]**3 - 3*x[0]*(x[1]**2) - 1, 3*(x[0]**2)*x[1] - x[1]**3]

def Jacobian(x):
return [[(3*x[0]**2 - 3*x[1]**2), -6*x[0]*x[1]],
[6*x[0]*x[1], (3*x[0]**2 - 3*x[1]**2)]]

def inverse_Jacobian_22(x):
try:
det_Jacobian = ( (3*x[0]**2 - 3*x[1]**2)**2 + (6*x[0]*x[1])**2 )
return [[(3*x[0]**2 - 3*x[1]**2) / det_Jacobian, 6*x[0]*x[1] / det_Jacobian ],
[-6*x[0]*x[1] / det_Jacobian, (3*x[0]**2 - 3*x[1]**2) / det_Jacobian ]]
except ZeroDivisionError:
return "Error, Jacobian is singular"

def newton_system(F, inv_J, x0, e, N):

 F_val = F(x0)
x = x0[:]

for count in range(N):

delta = mul_mat_vec(inv_J(x), F_val)
x = [x[i] - delta[i] for i in range(len(x))]
F_val = F(x)

if norm(F_val) < e:
break

return x, count

import math
x0 = [math.sqrt(2), math.sqrt(2)]
e = 1e-10
N = 1000

newton_system(F, inverse_Jacobian_22, x0, e, N)

([1.0, 4.094645006333355e-23], 8)
Out[ ]:

Task 1.3.2
Similar to Task 1.1.2, we need to examine how the convergence destination changes when the initial approximate solution of
Newton's method is changed. The following code block will finds all solutions of the system and creates grid_converge - a 2D
array of approximate solutions associated with each initial value.

In [ ]:
e = 1e-10
N = 1000
grid_converge = [] # Initialize the 2d array of solution associated with each initial value
sol_list = [] # Initialize the list of unique solutions

# Create a 2D grid of range -2 : 2 with step 0.05. Fill it with indices of solutions
for i in range(-40, 40):
column_converge = []
for j in range(-40, 40):
x0 = [j/20.0, -i/20.0] # j is x axis index, i is y axis index.
# because plt.imshow iterate y axis from postive to negative, add minus sign before i
if type(inverse_Jacobian_22(x0)) == str:
column_converge.append(-1)
else:
x = newton_system(F, inverse_Jacobian_22, x0, e, N)[0]
x = [round(xi, 4) for xi in x]
for sol in sol_list:
if sol == x:
break
else: sol_list.append(x)

column_converge.append(sol_list.index(x))
grid_converge.append(column_converge)

print("Solutions of the system: [x1, x2] =", sol_list)

Solutions of the system: [x1, x2] = [[-0.5, 0.866], [1.0, -0.0], [-0.5, -0.866]]
We plot the 2D array of approximate solutions as a color map as follow:

In [ ]:
import matplotlib.patches as mpatches

im = plt.imshow(grid_converge, extent=[-2,2,-2,2])
colors = [ im.cmap(im.norm(value)) for value in range(len(sol_list))]
# create a patch for every color associated with a solution
patches = [ mpatches.Patch(color=colors[i], label="Solution: {}".format(sol_list[i])) for i in range(len(sol_list
# put those patches into the legend
plt.legend(handles=patches, bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0. )
plt.ylabel("x2")
plt.xlabel("x1")
plt.grid()
We obseve the plane is generally divided into three regions, each one correlates with a solution. We also recognize that we have
generated a "Newton fractal", which is a boundary set in the complex plane characterized by Newton's method applied to a fixed
polynomial. This is because the system of equation would be considered a holomorphic function if evaluated in the complex
plane.