ML-IDA Assignment 1 Marking Guide

These are some notes on what to look for when marking this assignment.
 There are 336 students registered in canvas, but there will be about 300 submissions
in total.
 Please grade the following scripts:
o Guy: Students 1 – 84
o Sadia: Students 85 - 168
o Michalina: Students 169 – 252
o Shishen: Students 253 – 336
You access the scripts through “Speedgrader”
 You should have time to spend up to 15 minutes on each submission – please use it
and give as good feedback as you can.
 I have supplied a model solution (attached). There may be significant deviations from
this, which is fine. There is not a single correct answer here. The notes below
indicate what you should be considering.
 I have graded students 12, 67, 91, 126, 177, 229, 266, 314 so you can see roughly
where you should be calibrated.
 Let me know when you have marked 10 submissions so I can give them a quick
check.
 I’ve created a Teams channel for discussion – please post queries in there – tag me if
you need a quick response.
 The assignment can be found at
https://canvas.bham.ac.uk/courses/56308/assignments/326159 and the submissions
accessed at https://canvas.bham.ac.uk/courses/56308/gradebook/speed_grader?
assignment_id=326159&anonymous_id=1Wpx3
 Download the scripts one-at-a-time – they are anonymous as required by the code of
practice on assessment and feedback so if you do a bulk
 “View Rubric” to see the marking and feedback form where you should enter your
scores and comments.
 Use the course Jupyterhub at https://jupyter.apps.okd.aws.cs.bham.ac.uk/ which
has all of the libraries you need to run students’ code. Make sure you have the data
file (attached) in the same folder

Part 1 [3 marks]
Points to consider
 Have they used cross-validation properly? Is there a hold-out test set? Have they
randomised the order of the data?
 Do they rely on just a single training run or do they run repeats to ensure the results
is not a statistical fluke?
 Is the conclusion sensible? A quartic polynomial is what I would use for this, but
some may conclude that a cubic or even a quadratic is sufficient. This is fine, if their
results justify their conclusions.
 Are the predictions reasonable? Does the line of best fit go where you would expect?
Note that the data is generated as t=log(x) + noise. Most solutions will deviate
significantly from this.
Part 2 [4 marks]
 Have they used the correct model from part 1? Note that this only needs the correct
model order, not the parameters as we now estimate those via Bayesian methods.
 Have they correctly computed the parameters of the posterior?
 Are these approximately the same as the estimated parameters from part 1?
 Do they identify why they might be different? Part 1 assumes a uniform prior whilst
here a Bayesian prior is used. The two also use different amounts of data (unless the
model in part 1 was retrained on the full x-val dataset. Both observations are valid. If
the width of the Bayesian prior is increase by reducing alpha, the two estimates
converge.
Part 3 [3 marks]
 The data is generated by a logarithm and therefore should increase, but at a
decreasing rate. Predictions which do not show this should not be penalised if the
choices that led to that outcome were sensible.
 The predictions of a polynomial model are likely to be over-confident. This is because
the support of the basis function is over all x, unlike a Gaussian basis where the
support is limited. This tends to make such predictions over-confident. Have they
made reasonable remarks about whether they believe this or not?