Assignment 3

February 2020

1 Task 1
Consider the n × n matrix A with entries Aij = 5/(i + 2j − 1).

1.1 Compute the Infinity-Norm

n n 
X X 5 
kAk∞ = max |aij | = max (1)
 
i + 2j − 1
 
1≤i≤n 1≤i≤n
j=1 j=1

For some n, let us study the rows and their one-norms. Let us compare the row
ai,∗ to the row ai+1,∗ :
n 
X 5 
kai,∗ k1 = (2)
 
i + 2j − 1
 
j=1
n 
X 5 
kai+1,∗ k1 = (3)

i + 2j
 
j=1

As we can see in equations 2 and 3, the denominators of the terms of the sum
are consistently greater in ai+1,∗ , while the numerator is constant in each term
and sum. If the numerator is constants and the denominator is greater, then
the quotient will be lesser. Therefore, the row with the greatest one-norm will
always be the first row, which gives us equation 4.
n  n  n  
X 5  X
 5  5 X 1

kAk∞ = =  = (4)
 
1 + 2j − 1 2j 2 j=1 j
  
j=1 j=1

Now, let us study how the infinity-norm of the n×n-A relates to the n+1×n+1-
A. See equation 5.
n+1 n
5 X  1  5 X  1  5  1 
kAn+1×n+1 k∞ =  =  + =
2 j=1 j 2 j=1 j 2 n+1

(5)
5  1 
kAn×n k∞ + 
2 n+1


1
Following are the calculations for n ∈ [6, 15]. Note that numerical values have
been truncated to six decimals.

n=6
6
5 X  1  5  1 1 1 1 1 1  5 147
kAk∞ =  = + + + + + = ∗ = 6.125 (6)
2 j=1 j 2 1 2 3 4 5 6 2 60

n=7
7
5 X  1  5  1 
kAk∞ =   = kA6×6 k∞ +   ≈ 6.125 + 0.357142 ≈ 6.482142 (7)
2 j=1 j 2 7

n=8
8
5 X  1  5  1 
kAk∞ =   = kA7×7 k∞ +   ≈ 6.482142 + 0.3125 ≈ 6.794642 (8)
2 j=1 j 2 8

n=9
9
5 X  1  5  1 
kAk∞ =   = kA8×8 k∞ +   ≈ 6.794642 + 0.277777 ≈ 7.0724206 (9)
2 j=1 j 2 9

n = 10
10
5 X  1  5  1 
kAk∞ =   = kA9×9 k∞ +   ≈ 7.0724206 + 0.25 ≈ 7.322420 (10)
2 j=1 j 2 10

n = 11
11
5 X  1  5  1 
kAk∞ =   = kA10×10 k∞ +   ≈ 7.322420 + 0.227272 ≈ 7.549693
2 j=1 j 2 11
(11)

2
n = 12
12
5 X  1  5  1 
kAk∞ =   = kA11×11 k∞ +   ≈ 7.549693 + 0.208333 ≈ 7.758026
2 j=1 j 2 12
(12)

n = 13
13
5 X  1  5  1 
kAk∞ =   = kA12×12 k∞ +   ≈ 7.758026 + 0.192307 ≈ 7.950334
2 j=1 j 2 13
(13)

n = 14
14
5 X  1  5  1 
kAk∞ =   = kA13×13 k∞ +   ≈ 7.950334 + 0.178571 ≈ 8.128905
2 j=1 j 2 14
(14)

n = 15
15
5 X  1  5  1 
kAk∞ =   = kA14×14 k∞ +   ≈ 8.128905 + 0.166666 ≈ 8.295572
2 j=1 j 2 15
(15)

1.2 Swamping
The approximations of x can be found in listing 9 in the appendix.

1.3 Condition Numbers

κ(A) = kAk · kA−1 k (16)

In figure 1, we can see that for smaller n, the condition number of A grows
exponentially.

3
 100
1015
κ(A)

η
10−5

1010

10−10
10 15 10 15
n n

Figure 1: To the left are the condition numbers of A for n ∈ [6, 15] in task 1.
To the right are the relative errors of the solutions to Ax = b for n ∈ [6, 15] in
task 1.

1.4 Relative Error

The definition of the relative error η is given in equation 17, with the exact
solution ν and the approximation νapprox. The one-norm was used for absolute
values of vectors. See figure 1.
ν − ν 
approx 
η= (17)

ν


When η = 10−1 , the error may be of the same order of magnitude as the
approximation, meaning that all significant digits could be wrong. This happens
when n = 12. With that order of magnitude, all modern computers should
produce the same result.
As a note, our results diverged as early as n = 11. I do not consider 0.999104
have all digits incorrect, since 0.999 · · · ≡ 1.

4
 1

v1
0

−1
−1 0 1
v2

Figure 2: All one-norm unit vectors v.

2 Task 2
A vector v is a unit vector if kvk = 1.

2.1 One-norm
One-norm is defined in equation 18 below.
n
X
kxk1 = |xi | (18)
i=1

Let us start with the one-norm unit vectors. Since we are working in R2 , our
unit vectors have two elements, so

kvk1 = |v1 | + |v2 | = 1 ⇔ |v1 | = 1 − |v2 | ⇔ v1 = ±1 ∓ |v2 |

(
v1 = 1 − |v2 | or,
⇒ (19)
v1 = −1 + |v2 |
Clearly, vi ≤ 1. We want to plot all vectors v for which the conditions in
equation 19 hold true. Let us plot two separate lines, one for each case. See
figure 2.

2.2 Two-norm
v
u n
uX
kxk2 = t x2i (20)
i=1

In two dimensions, we get from equation 20 that

q q
kvk2 = x21 + x22 = 1 ⇔ x1 = ± 1 − x22

5
v1 1 1

v1
0 0

−1 −1
−1 0 1 −1 0 1
v2 v2

Figure 3: To the left, all two-norm unit vectors v and to the right, all infinity
norm unit vectors v.

p
This is the equation
p of a circle with a radius of one. Let us plot x1 = 1 − x22
and x1 = − 1 − x22 . See figure 3.

2.3 Infinity Norm

kxk∞ = max |xi | (21)
1≤i≤n

The infinity norm is defined in equation 21. We apply it in two dimensions.



 x1 = 1, −1 ≤ x2 ≤ 1, or

x = −1,
1 −1 ≤ x2 ≤ 1, or
kvk∞ = max |xi | = 1 ⇔
1≤i≤2 

 x 2 = 1, −1 ≤ x1 ≤ 1, or
x2 = −1, −1 ≤ x1 ≤ 1,


We can plot this as a box with a width of two. See figure 3.

2.4 Code
No code except LATEX with the pgfplots package was used. See listings 1, 2, and
3.

Listing 1: Code for producing the plot with one-norm unit vectors in task 2.
\ begin { t i k z p i c t u r e }
\ begin { axis }[
width =5.9cm ,
h e i g h t =5.9cm ,
x l a b e l=$v 2$ ,
y l a b e l=$v 1$ ,
domain = −1:1 ,

6
minor t i c k num=4,
g r i d=both ,
g r i d s t y l e ={ l i n e width =.1 pt , draw=gray ! 1 0 } ,
major g r i d s t y l e ={ l i n e width =.1 pt , draw=gray ! 5 0 }
]
\ a d d p l o t [ red , s a m p l e s =100]{1 − abs ( x ) } ;
\ a d d p l o t [ blue , s a m p l e s =100]{−1 + abs ( x ) } ;
\ end { a x i s }
\ end { t i k z p i c t u r e }

Listing 2: Code for producing the plot with two-norm unit vectors in task 2.
\ begin { t i k z p i c t u r e }
\ begin { axis }[
width =5.9cm ,
h e i g h t =5.9cm ,
x l a b e l=$v 2$ ,
y l a b e l=$v 1$ ,
%domain = −1:1 ,
minor t i c k num=4,
g r i d=both ,
g r i d s t y l e ={ l i n e width =.1 pt , draw=gray ! 1 0 } ,
major g r i d s t y l e ={ l i n e width =.1 pt , draw=gray ! 5 0 }
]
\ a d d p l o t [ red , s a m p l e s =1001]{ s q r t (1−x ˆ 2 ) } ;
\ a d d p l o t [ blue , s a m p l e s =1001]{− s q r t (1−x ˆ 2 ) } ;
\ end { a x i s }
\ end { t i k z p i c t u r e }

Listing 3: Code for producing the plot with intfy-norm unit vectors in task 2.
\ begin { t i k z p i c t u r e }
\ begin { axis }[
width =5.9cm ,
h e i g h t =5.9cm ,
x l a b e l=$v 2$ ,
y l a b e l=$v 1$ ,
%domain = −1:1 ,
minor t i c k num=4,
g r i d=both ,
g r i d s t y l e ={ l i n e width =.1 pt , draw=gray ! 1 0 } ,
major g r i d s t y l e ={ l i n e width =.1 pt , draw=gray ! 5 0 }
]
\ a d d p l o t +[mark=none ] c o o r d i n a t e s {
( −1 , −1) ( −1 ,1) ( 1 , 1 ) (1 , −1) ( −1 , −1)
};
\ end { a x i s }

7
\ end { t i k z p i c t u r e }

8
3 Task 3
     
1 2 1 4 1 1
A1 = , A2 = , A3 = (22)
2 1 −4 1 1 1
In the figure below we have plotted the norms in cyan, and the A * norm
in magenta. Then there is the code for calculating and plotting the different
graphs. Code part 2 is repeated for all A’s and norms.

9
 Figure 4: The plots

Figure 5: Code part 1

10
 Figure 6: Code part 2

4 Task 4
Consider the matrices A1 and A2 as given in equation 22 in task 3.

4.1 Eigenvalues and Eigenvectors

Start with A1 by setting det (A1 − λI) = 0:
 
1 − λ 2 
 = 0 ⇐⇒ (1 − λ)(1 − λ) − 2 ∗ 2 = 0
 2 1 − λ
⇐⇒ λ2 − 2λ + 1 − 4 = 0
⇐⇒ λ2 − 2λ − 3 = 0
⇐⇒ (λ − 3)(λ + 1) = 0
(
λ1 = 3
⇐⇒
λ2 = −1

11
To find the eigenvector u, set (A1 − λI)x = 0. Start with λ1 = 3.
    
1−3 2 x1 0
=
2 1−3 x2 0
⇐⇒
(  
−2x1 + 2x2 =0 1
⇔ x1 = x2 ⇔ u1 = k
2x1 − 2x2 =0 1

Where k is some constant scalar. Then, for λ2 = −1.

    
1 − (−1) 2 x1 0
=
2 1 − (−1) x2 0
⇐⇒
(  
2x1 + 2x2 =0 1
⇔ x1 = −x2 ⇔ u2 = k
2x1 + 2x2 =0 −1

Where k is some constant scalar. Repeat for A2 . Set det (A2 − λI) = 0.
 
1 − λ 4 

 −4 = 0 ⇐⇒ (1 − λ)(1 − λ) − 4 ∗ (−4) = 0
1 − λ
⇐⇒ (1 − λ)(1 − λ) + 16 = 0
⇐⇒ (1 − λ)2 = −16
√
⇐⇒ (1 − λ) ± −16 (23)
(
1 − λ1 = 4i
⇐⇒
1 − λ2 = −4i
(
λ1 = 1 − 4i
⇐⇒
λ2 = 1 + 4i

To find the eigenvector u, set (A2 − λI)x = 0. Start with λ1 = 1 − 4i.

    
1 − (1 − 4i) 4 x1 0
=
4 1 − (1 − 4i) x2 0
⇐⇒
(  
4ix1 + 4x2 =0 i
⇔ x1 = ix2 ⇔ u1 = k
4x1 + 4ix2 =0 1

12
Where k is some constant scalar. Then, for λ2 = 1 + 4i.
    
1 − (1 + 4i) 4 x1 0
=
4 1 − (1 + 4i) x2 0
⇐⇒
(  
−4ix1 + 4x2 =0 i
⇔ x1 = −ix2 ⇔ u2 = k
4x1 + −4ix2 =0 −1

Where k is some constant scalar.

4.2 Condition Numbers

We can use the two-norm to get a nice expression for the condition number of
a symmetric matrix.
p
−1 1 λmax (AT A)
κ(A) = kAk2 · kA k2 = ρmax (A) · = p
ρmin (A) λmin (AT A)

We start with A1 . We first calculate AT1 A1 . A1 is symmetric, so AT1 A1 = A21 .

    
T 2 1 2 1 2 5 4
A1 A1 = A1 = =
2 1 2 1 4 5

Start by setting det (A21 − λI) = 0:

 
5 − λ 4 
 = 0 ⇐⇒ (5 − λ)(5 − λ) − 4 ∗ 4 = 0
 4 5 − λ
⇐⇒ λ2 − 10λ + 25 − 16 = 0
⇐⇒ λ2 − 10λ + 9 = 0
⇐⇒ (λ − 1)(λ − 9) = 0
(
λ1 = 1
⇐⇒
λ2 = 9

We can now calculate κ(A1 ).

p √
λmax (A21 ) 9
κ(A1 ) = p = =3
λmin (A21 ) 1

We continue with A2 . This matrix in not symmetric, but we can still use the
following: q
kAk2 = ρ(AT A) (24)

13
 ρ(A) = max |λ| (25)
We first calculate AT2 A2 .
    
1 −4 1 4 17 0
AT2 A2 = =
4 1 −4 1 0 17

We calculate the eigenvalues λ of AT2 A2 by setting det (AT2 A2 − λI) = 0

 
17 − λ 0 
 = 0 ⇒ (17 − λ)2 = 0 ⇒ λ = 17
 0 17 − λ
⇒ ρ(AT2 A2 ) = 17
√
⇒ kA2 k2 = 17

We now calculate A−1

2 .
 
1 1 1 −4
A−1
2 = adj(A2 ) =
det A2 1 ∗ 1 − (−4) ∗ 4 4 1
 
1 1 −4
=
17 4 1

Let A−1 T
2 = B. We now calculate B B.
     1 
T 1 1 4 1 −4 1 17 0 0
B B= 2 = 2 = 17 1
17 −4 1 4 1 17 0 17 0 17

We calculate the eigenvalues λ of B T B by setting det (B T B − λI) = 0

1 
 −λ
 17 0  1 2 1
 0 1
− λ = 0 ⇒ ( 17 − λ) = 0 ⇒ λ = 17
17
1
⇒ ρ(B T B) =
r 17
1
⇒ kBk2 =
17

We can now calculate κ(A2 ).

√
r
1
κ(A2 ) = kA2 k2 · kA−1
2 k2 = 17 ∗ =1
17

4.3 Power Method

Using a starting guess of x0 = (137, 42), the program was able to find the
eigen vectors of A1 with an error less than 10−4 in 7 or 8 iterations. For A2

14
however, the same starting guess did not converge to the correct result, even in
1000 iterations. Instead, it converged to λmax = 1 and λmin = 17. In fact, it
converged to those values for several different real x0 .
Different complex x0 produced different results, even with 1000 iterations.
For example, x0 = (−17 + 5i, 11 + 19i) produced λmax ≈ 1.00000 − 3.79899
and λmin ≈ 1.10158 − 4.18490. Others produced wildly incorrect results. The
program was able to converge to correct results in very few (two) iterations
when x0 = (1.1i + 0.1, −1.1 + 0.1i), but would only produce the same eigen
vector as both maximum and minimum. See listings 4, 5, 6, 7.

15
 Listing 4: Implementation of the Power Method.
1 % Based on the pseudo code in lecture 10
2 function lambda = power_method (A , x0 , iterations )
3 n = length ( A ) ;
4 x = zeros ( iterations , n ) ;
5 x (1 ,:) = x0 ;
6 u = zeros (n , iterations ) ;
7 l = zeros ( iterations , 1) ;
8 for j = 2: iterations
9 % normalize
10 u (: ,j -1) = x (j -1 ,:) / norm ( x (j -1 ,:) ,2) ;
11 % Power iteration for eigen vector
12 x (j ,:) = A * u (: ,j -1) ;
13 % Rayleigh quotient for eigen value
14 l(j) = u (: ,j -1) ' * A * u (: ,j -1) ;
15 end
16 % u (: , j ) = x (j ,:) / norm ( x (j ,:) ,2) ;
17 % A * u (: , j ) - l ( j ) * u (: , j )
18 lambda = l ( j ) ;

Listing 5: Implementation of the Inverse Power Method.

1 function lambda = i n v e r s e _ p o w e r _ m e t h o d (A , x0 ,
iterations )
2 lambda = power_method ( inv ( A ) , x0 , iterations ) ;
3 lambda = 1/ lambda ;

Listing 6: Main program for task 4.

1 % Task 4
2 % Program the Power Method and approximate both
eigenvalues of A_1 and A_2 with a maximum tolerance
of 10^( -4)
3 diary task4_output . txt
4 x0 = [137 42]
5 A =[1 2; 2 1]
6 l_max = power_method (A , x0 , 8)
7 l_min = i n v e r s e _ p o w e r _ m e t h o d (A , x0 , 8)
8
9 x0 = [137 42]
10 A =[1 4; -4 1]
11 l_max = power_method (A , x0 , 1000)
12 l_min = i n v e r s e _ p o w e r _ m e t h o d (A , x0 , 1000)
13
14 x0 = [ -17+5 i 11+19 i ]
15 l_max = power_method (A , x0 , 1000)

16
16 l_min = i n v e r s e _ p o w e r _ m e t h o d (A , x0 , 1000)
17
18 x0 = [1.1 i +0.1 -1.1+0.1 i ]
19 l_max = power_method (A , x0 , 2)
20 l_min = i n v e r s e _ p o w e r _ m e t h o d (A , x0 , 2)
21 diary off

Listing 7: Output of the program in listing 6.

1
2 x0 =
3
4 137 42
5
6
7 A =
8
9 1 2
10 2 1
11
12
13 l_max =
14
15 2.99 999787 995122 2
16
17
18 l_min =
19
20 -1.000008907247059
21
22
23 x0 =
24
25 137 42
26
27
28 A =
29
30 1 4
31 -4 1
32
33
34 l_max =
35
36 1
37

17
38
39 l_min =
40
41 17
42
43
44 x0 =
45
46 Column 1
47
48 -17.000000000000000 + 5. 000000 000000 000 i
49
50 Column 2
51
52 11 .0 00 00 00 00 00 00 00 + 1 9 .0 0 0 00 0 0 00 0 0 00 0 0 i
53
54
55 l_max =
56
57 1.0 000000 000000 00 - 3.7 989949 748743 71 i
58
59
60 l_min =
61
62 1.1 015811 511985 01 - 4.1 849012 578194 37 i
63
64
65 x0 =
66
67 Column 1
68
69 0.1 000000 000000 00 + 1.1 000000 000000 00 i
70
71 Column 2
72
73 -1.100000000000000 + 0. 100000 000000 000 i
74
75
76 l_max =
77
78 1.0 000000 000000 00 + 3.9 999999 999999 99 i
79
80
81 l_min =
82
83 1.0 000000 000000 00 + 4.0 000000 000000 01 i

18
 Appendix

Listing 8: Program for task 1.

1 % Task 1
2 As = task1_construct (15) ;
3 ks = zeros (10 ,1) ; % condition numbers
4 es = zeros (10 ,1) ; % relative errors
5 diary task1_output . txt
6 for n = 6:15
7 fprintf ( 'n = % d \ n ' , n ) ;
8 A = As (1: n ,1: n ) ;
9 % First subtask - compute infinity - norms
10 infinitynorm = task1_inftynorm ( A (1: n ,1: n ) ) ;
11 fprintf ( '\ tinfinity - norm = %.6 f \ n ' ,
infinitynorm ) ;
12 % Second subtask - solve for x
13 x0 = ones (n , 1) ;
14 b = A * x0 ;
15 warning ( ' off ' , ' MATLAB : n e a r l y S i n g u l a r M a t r i x ') ;
16 x=A\b;
17 warning ( ' on ' , ' MATLAB : n e a r l y S i n g u l a r M a t r i x ') ;
18 fprintf ( '\ tx = (%.6 f ' , x (1) ) ;
19 fprintf ( ' , %.6 f ' , x (2: n ) ) ;
20 fprintf ( ') \ n ') ;
21 % Third subtask - calculate condition number
22 warning ( ' off ' , ' MATLAB : n e a r l y S i n g u l a r M a t r i x ') ;
23 ks ( n ) = norm (A , inf ) * norm ( inv ( A ) , inf ) ;
24 warning ( ' on ' , ' MATLAB : n e a r l y S i n g u l a r M a t r i x ') ;
25 % Fourth subtask - calculate relative error
26 es ( n ) = norm ( x0 -x ,1) / norm ( x0 ,1) ; % relative
error using one - norm
27 end
28 % Third subtask - print table with condition numbers
29 diary t a s k 1 _ c o n d i t i o n _ n u m b e r s . txt
30 fprintf ( 'n \ tk ( A ) \ n ') ;
31 for n = 6:15
32 fprintf ( '% d \ t %.6 e \ n ' , n , ks ( n ) ) ;
33 end
34 % Fourth subtask - print table with relative errors
35 diary t a s k 1 _ r e l a t i v e _ e r r o r . txt
36 fprintf ( 'n \ trelative error \ n ') ;
37 for n = 6:15

19
38 fprintf ( '% d \ t %.6 e \ n ' , n , es ( n ) ) ;
39 end
40 diary off

Listing 9: Infinity-norms and approximations of x in task 1.

1 n = 6
2 infinity - norm = 6.125000
3 x = (1.000000 , 1.000000 , 1.000000 , 1.000000 ,
1.000000 , 1.000000)
4 n = 7
5 infinity - norm = 6.482143
6 x = (1.000000 , 1.000000 , 1.000000 , 1.000000 ,
1.000000 , 1.000000 , 1.000000)
7 n = 8
8 infinity - norm = 6.794643
9 x = (1.000000 , 1.000000 , 1.000000 , 1.000000 ,
1.000000 , 0.999999 , 1.000001 , 1.000000)
10 n = 9
11 infinity - norm = 7.072421
12 x = (1.000000 , 1.000000 , 1.000000 , 1.000003 ,
0.999986 , 1.000034 , 0.999955 , 1.000031 ,
0.999992)
13 n = 10
14 infinity - norm = 7.322421
15 x = (1.000000 , 1.000000 , 0.999998 , 1.000023 ,
0.999860 , 1.000467 , 0.999104 , 1.000986 ,
0.999423 , 1.000139)
16 n = 11
17 infinity - norm = 7.549693
18 x = (1.000000 , 1.000001 , 0.999955 , 1.000746 ,
0.993972 , 1.027000 , 0.928350 , 1.115212 ,
0.889912 , 1.057454 , 0.987397)
19 n = 12
20 infinity - norm = 7.758027
21 x = (1.000000 , 1.000002 , 0.999900 , 1.002080 ,
0.978802 , 1.121198 , 0.580855 , 1.909185 ,
-0.243305 , 2.039815 , 0.514647 , 1.096820)
22 n = 13
23 infinity - norm = 7.950334
24 x = (1.000000 , 0.999997 , 1.000190 , 0.995415 ,
1.054474 , 0.631676 , 2.535881 , -3.127155 ,
8.275503 , -7.358812 , 7.024454 , -1.472174 ,
1.440552)
25 n = 14
26 infinity - norm = 8.128906

20
27 x = (1.000000 , 1.000000 , 1.000001 , 1.000225 ,
0.993468 , 1.079399 , 0.466333 , 3.203143 ,
-4.870579 , 11.289551 , -10.783271 , 9.482357 ,
-2.482158 , 1.621531)
28 n = 15
29 infinity - norm = 8.295572
30 x = (1.000000 , 0.999989 , 1.000699 , 0.983112 ,
1.198485 , -0.305062 , 6.168525 , -11.722375 ,
20.284258 , -15.629808 , 6.699288 , 2.995919 ,
-0.750621 , 0.805904 , 1.271691)

21