Computational Methods 5CCYB070

Assignment 2: Linear Systems

INTRODUCTION
In this assignment, you will implement the matrix factorisation methods for solving systems of linear
equations. To get started with this assignment, download the .zip file from KEATS and extract to a
folder of your choice. Below is a list of files contained:

cm2.m – Main driver Matlab script to guide you through the programming exercises
[*] crout1.m – function that implements the Crout method (version 1)
[*] crout2.m – function that implements the Crout method (version 2)
[*] backsub.m – function that implements backward substitution
[*] forwardsub.m – function that implements forward substitution
[*] adjugateInv.m – function to implement adjugate matrix inversion algorithm
qritten_q3.m – Matlab script for question 3 of section 7
submit.m – Matlab script to package up your files for submission. Please use this!

Note that [*] indicates the files you will need to complete. They are partially written with function
definitions and input/output descriptions. These subroutines are called by the main driver script
cm2.m, which will step you through the exercises. Start by reading this whole document.

1 FORWARD SUBSTITUTION
The forward substitution solves the lower triangular linear system of the form:

𝑳𝒚 = 𝒃
As explained in the lecture, the algorithm simply solves each equation (row) from top to bottom.
Take the following system as an example:
2 0 0 0 𝑦1 1
𝑦
1 0 0 ] [ 2] = [ 0 ]
[−1
1 0 3 0 𝑦3 −1
−2 4 1 −1 𝑦4 0
In each row, we must isolate the current unknown of interest, which for row i, is 𝑦𝑖 . The precise
formulas are given in the box below. In the code, because we deal with each equation at a time
there must be a loop corresponding to the rows. Within each row, there is a summation to be
carried out (except for the first row, that is), which can be achieved using another loop or,
preferably, a vector operation. The code is partially implemented in forwardsub.m.

► Now implement the forward substitution algorithm in forwardsub.m:

𝑦1 = 𝑏1 /𝐿11
𝑗=𝑖−1
(𝑏𝑖 − ∑𝑗=1 𝐿𝑖𝑗 𝑦𝑗 )
𝑦𝑖 = , 𝑖 = 2. . . 𝑛
𝐿𝑖𝑖

At minimum, you will have to complete the two lines that are commented out though further
improvements could be made. Save it and return to cm2.m. Your implementation is automatically

1
Computational Methods 5CCYB070

checked in cm2.m - it multiplies 𝑳 with 𝒚, and verifies that it gives back 𝒃. In other words, if your
implementation is correct, the residual norm ||𝒓|| = ||𝒃 − 𝑳𝒚|| should equal (machine) zero.

1.1 Vector Orientations

Note that, the 𝒚 vector must be a column vector on exit, otherwise we would not be able to perform
the multiplication 𝑳𝒚 = 𝒃. This can be ensured at variable declaration, or the enforced at the end
just before exiting the subroutine (Avoiding extra steps is best for code efficiency).

2 BACKWARD SUBSTITUTION
► Now that you have the experience of making the forward substitution work, let’s implement the
backward substitution method for 𝑼𝒙 = 𝒚 in backsub.m. Your code will need to complete the
contents of the for-loop block.

This algorithm structure is very similar to the forward substitution. The only major difference here is
that we start at the final row i.e. 𝑥𝑛 and move back up to 𝑥1 , meaning your loop indexing should be
reversed. The formulas are provided below.

𝑦𝑛
𝑥𝑛 =
𝑈𝑛𝑛
𝑗=𝑛
(𝑦𝑖 − ∑𝑗=𝑖+1 𝑈𝑖𝑗 𝑥𝑗 )
𝑥𝑖 = , 𝑖 = 𝑛 − 1. . .1
𝑈𝑖𝑖
As before, cm2.m will test your implementation by ensuring that residual 𝒓 = 𝒚 − 𝑼𝒙 has a small
norm, so highlight the appropriate block in cm2.m and hit ctrl-enter to see the output.

3 CROUT FACTORISATION I
We will now work towards implementing the Crout factorisation algorithm. Recall that the algorithm
can be simply summarised as:

1) Normalising the pivot row (i.e. the diagonal), then

2) Zeroing the whole column below the pivot
The code provided sticks close to implementing these two steps (although you will have to figure out
which lines of code correspond to which). In this version of the code, all the loops are unrolled,
meaning that any iterations are handled via a for-loop instead of vectorisation.
► Now, open the file crout1.m and complete the code. The lines you have to write are commented
out i.e. there is no need for additional lines of code.

The results of your code are checked in cm2.m by multiplying 𝑳 and 𝑼 and verifying that it gives back
𝑨. If your code results in an error along the way, it may be easier to enable ‘Pause on Error’ under the
‘Run’ button in the menu to diagnose the problem as shown below (Ctrl-enter won’t work with this).
Pay attention to any error messages as they are very informative for debugging.

2
Computational Methods 5CCYB070

4 CROUT FACTORISATION II
► The next task will be performed on a different file, crout2.m. It implements the exact same Crout
method of factorisation, but with one difference – vectorisation should be used where possible in
order to cut down on the runtime of the code.
The rest should be very self-explanatory if you completed crout1.m already. Check your
implementation by using the built-in test in cm2.m.

5 ADJUGATE METHOD
For comparison, let’s also implement the adjugate method of matrix inversion. Remember, this
method involves calculating the cofactor of a given matrix. The formula for the method is
1 𝑇
𝐀−1 = 𝐂 , 𝐶𝑖𝑗 = (−1)𝑖+𝑗 𝑀𝑖𝑗
|𝐀|
The bulk of the method involves extracting a submatrix from 𝐀 (let’s call it 𝐁) and then calculating its
determinant, for which you should use the built-in function det().

► There are a few different ways of extracting a submatrix, and what you do here will likely make a
significant difference to the speed of the code. Remember that allocating an array unnecessarily,
“growing” an array instead of pre-allocating it, making copies of arrays unnecessarily etc will incur
additional runtime cost. Insert whatever code you need in order to complete the function.

When you’ve finished the implementation, test your code using cm2.m in the usual way. Note that in
order to earn credit for the next task, this function will have to contain no errors.

6 PERFORMANCE SCALING
We will now investigate the performance of your code. The three methods you implemented will be
compared in terms of their runtime speed. Advance to the next block in cm2.m.

The code generates a series of matrices of increasing size, and measures the time it takes to
factorise each one. The time measurement is performed using tic/toc statements, while the test
matrices are generated using an in-built Matlab functionality (‘lehmer’ type, which yields symmetric
positive definite matrices). The sizes of matrices built vary over [10 100 300 500 800 1000
1400 1600 2000], thus covering a large range. The final matrix (2000 by 2000) may take a while to
factorise, so be patient.

3
Computational Methods 5CCYB070

The adjugate method cannot invert matrices of the same size range within a reasonable length of
time. Therefore a smaller matrices are tested. The full results will be shown on the same plot if you
have correctly completed every task so far.

► There is nothing to write in this block, just run the code and wait for the plot to show up. Note
that, when you run this code you should not be running any other applications in the foreground
(e.g. browser, videos etc) otherwise they will compete for system resource and may affect your
timing results.

7 WRITTEN QUESTIONS
Provide short answers to the following questions as a single PDF file (*not* word) and include it in
your submission. Be sure to answer all parts of each question, and remember, it’s quality over quantity!

1) Include your plot from section 6 on computation times. Write a short paragraph to discuss
and compare each line on the plot, making sure to relate your observations to the known
theory. Additional credit may be awarded for providing quantitative evidence to back your
statements e.g. by further processing the scaling data.

2) Consider the following matrix:

4 −2 0 4
𝐀 = [2 −1 9 5 ]
−2 2 −1 1
−4 6 −1 11

What is the determinant of 𝐀? What is the condition number of 𝐀? Now use your Crout
routine to factorise the matrix. What happens, and why? Discuss these results collectively.

3) When solving a system 𝐀𝐱 = 𝐛, ill-conditioning and rounding error can interact and amplify
to degrade the accuracy of the computation. Denoting the numerically estimated solution as
𝐱̌ we will find that it contains error such that 𝐱̌ = 𝐱 + δ𝐱. As a consequence, the matrix-vector
product will not evaluate exactly to 𝐛:

𝐀𝐱̌ = 𝐛 + δ𝐛

One way to improve the solution is to use the above equation to estimate δ𝐱 which can then
be used to improve the solution. It is important that, when performing this calculation we use
increased numerical precision for 𝐱̌ as we are working near the limits of the Double type. This
method is implemented in written_q3.m where the correct solution should be

1 1 1 1 1T
𝐱=[ ] .
2 3 4 5 6

Figure out and explain what is going on in lines 9 and 10 of the code as you would to a
classmate, making reference to the outputs. Support your explanation with derivations and
clarify any unknown parts of the code.

4
Computational Methods 5CCYB070

SUBMISSION
Please use the submit function to create a zip file containing all your .m files. If it reports an error, it
may be because not all of the files to be submitted could be found in the folder. Please make sure to
submit this zip file, and only this file on KEATS. Due to a larger class size, all submissions via email will
be ignored.
Sometimes, for various reasons the submit script may fail on your system. It may be necessary in
this case to create a submission file manually. To do this, go inside the folder containing your .m
and .pdf files, select all, then right click to create a single zip file for submission. The content of this zip
file should look like

MARKING
Part of the evaluation of your code will involve solving various systems of linear equations, which will
not be made available to you at this time. For details, consult the detailed marking scheme below.

Item Marks are awarded for weighting

Forward sub
Backsub
Crout factorisation
Adjugate method
Written answers
Correct results with test problems, code accuracy, quality 10%
Correct results with test problems, code accuracy, quality 10%
Correct results with test problems. Even if the outputs are 20%
incorrect, partial marks could be awarded for the parts
implemented correctly
Correct results with test problems, code accuracy/efficiency 10%
and quality
Correct answers demonstrating the understanding of 50%
theoretical underpinnings of the questions

PLAGIARISM
You are reminded that we operate on zero tolerance policy in this module. If you google for solutions
and copy them, remember that we can google as well. Any submissions found to contain answers that
is not your own work will result in disciplinary action being taken.