Understanding education research:

Numbers, narratives, knowledge &

nonsense
Live lecture 1: Introduction,
overview, country knowledge
JD Carpentieri
Overview Microphone! Check and
Topics
wear
1. Introductions: me, rest of the teaching team
2. What is this module about?
3. Quick tour of moodle
4. Nonsense

Activities:
1. Role of uni rankings
2. How much do you know about the UK/China?
Links
1. Role of uni rankings
https://docs.google.com/forms/d/e/1FAIpQLScncyheE0D9CEIBXPzkkjjEkEBO_aZXCBXh4chUMV-uM3byAg/vie
wform

a. Responses:
https://docs.google.com/forms/d/1wDE98W6h0HTzAkZU_3Js2vx9nC0GHOiWacrzewNv9lU/edit#response
s

2. How much do you know about the UK/China?

https://docs.google.com/spreadsheets/d/1mzwAl0E5zSQIiKGn4A0QwQeBZ_AC7KgvVDZ7-1UBlKY/edit#gid=0
Why did you choose
UCL?
1. How important was the
Institute of Education’s
number 1 ranking?
2. Individually, answer the 1st
question on this survey:
https://forms.gle/4PqaxSD6NvxCGoA18
a. Responses:
https://docs.google.com/forms/d/1wDE98W6h0
HTzAkZU_3Js2vx9nC0GHOiWacrzewNv9lU/edi
t#responses

3. Then discuss both survey

questions with your
tablemates
4. Finally, 1 person at your
table answer the 2nd
Nonsense
• What if the rankings and
reviews were lies?

• What if they
give you
nonsense
instead of
knowledge?

• Like at
Trump University?
Trump University
• Trump claimed that students gave 98% favourable
reviews
• But were the reviews high quality data?
• Trump University employees pressured students
to give positive reviews in order to graduate
• The evidence that potential students
were basing their decisions on was false –
it was nonsense, not knowledge
Which
magazine: ‘Higher education institutions c
•aught making
‘A number potentially
of universities misleading
are presenting claim
unverifiable
s marketing claims, making it difficult for prospective students
to make informed choices”
’• Newcastle University said it was in the top 1% in the QS
World University Rankings
– It actually came 141st, i.e. in the top 15%
• University of the West of Scotland claimed that it “ranked in
the top 3%”
– It actually came 501-600th out of 1,103 in the
Times Higher Education World University Rankings 2018
Trump University
Understanding education research
Key aims of this module:
• Help you recognise nonsense and understand how
knowledge is created
• Make you more confident and competent
consumers and users of research evidence and
arguments
• Not just in academia, but in the media and daily
life
• Help you learn how to use research evidence to
Understanding education research
• Primary focus isn’t to teach you how to do
research
• It’s about: understanding research
– How research evidence is produced
– How to understand that evidence
• How research influences policy and
practice
• Being a critical consumer of evidence,
not an unquestioning consumer
Understanding education research:
The information you encounter on this BA and in the world
at large:
• Is a product of human effort and human culture
– Evidence/information is produced, not found
– The process is not perfect
• May be right, but may be wrong
(or may be partly right and partly
wrong)
Understanding education research:
Numbers, narratives, knowledge and nonsense
1. Intro to nonsense and knowledge creation
2. Qualitative research (mostly)
3. Quantitative research (mostly)

Ongoing theme: How research is used and mis-used by

policy (the tension between nonsense and knowledge)
Let’s take a tour
of moodle
Week 1: https://moodle.ucl.ac.uk/course/view.php?id=34923&section=5#tabs-tree-start
Assessment
Brief discussion now
1. Portfolio: 30%
2. Writing assignment: 70%

See this week’s Asynchronous (pre-recorded)

lecture for a much fuller discussion
Asynchronous (pre-recorded)
1. Only about 2-3 over the course of the term
lectures
2. This week’s pre-recorded lecture has essential info about
assessment on this module

3. Other pre-recorded lecture will provide “extra info” on topics

that aren’t essential to the flow of ideas across the term,
but which you may find interesting
Readings
• Required reading some weeks (only about 3-4 times over
the term)
• Suggested readings other weeks – these are extras that
aren’t essential to understanding
• The more you read the more you’ll understand in the
lectures, when doing your assessments, etc.
• It’s your job to look at moodle, find the readings and do
them
Expectations
1. Uni is VERY different from secondary school
2. Strong expectation of engagement – not just listening but
speaking up
3. No one’s checking up on you, but we won’t spoonfeed you
either
4. It may seem easy at first: we’re not checking your homework
or giving you weekly quizzes, etc
5. First year counts much less, so you can develop yourself as a
uni student with “lower stakes”:
a. Year 1 counts for 11% of your overall grade (1/9)
Lectures
• Live lectures start 10:00, finish 11:20
– “90 minutes” = 80 minutes (to give students time to get to next
class)
• Mostly me talking, plus a few activities each week
• I design 9 lectures, which usually take 10 weeks to get
through

• We are using lecturecast (for lectures, not seminars)

– This is meant primarily to help you review lecture material, not
skip the lectures. You’ll get more out of being here live
– You can find lecturecasts on moodle: ‘Welcome: start here’
Seminars
• 1.5 hours (= 80 minutes)
• Be on time! See your timetable for rooms
• Portfolio activities during seminars – see
pre-recorded lecture 1.1 for a full discussion
• Your seminar leader will also discuss
portfolio activities
• Seminars: mostly you talking
Attendance
• Tap-in to seminar and lecture rooms
• LECTURES: There’s an expectation of 100% attendance (and you
should tap in), but I do not check your lecture attendance
– If you’re going to miss a lecture, you don’t need to message me.
If you do, I won’t respond (sorry!)

SEMINAR attendance is mandatory and is checked every week

– You’re allowed to miss a maximum of 3 seminar sessions per
module, but we traditionally have excellent attendance
– If you’re going to miss a seminar, email your seminar leader
(not me) – they’re in charge of recording your attendance
– Don’t email your seminar leader about missing a lecture
Come see me!
• Every Wednesday from 12:00-13:00 I will be in my office and
available for questions, discussion, etc
– If I’m not going to be available one week, I’ll announce it in the
lecture
– I’m away on 18 Oct

• You don't need to book an appointment: you

can just drop by
• Room G10, 55-59 Gordon Square
– But you can book an appointment during this
time period if you want to
Nonsense is everywhere
How much
water should
you
drink each day
?

Discuss this at
your table for a
minute
Sometimes we believe numbers without
questioning them
• 1945 US Food and Nutrition Board recommendation that people need
about 2 litres of water a day
– Also noted that we get most of this water through the food we eat

• The need for 2 litres of water (or more) water per day has been
stated/printed so often it’s become a ‘fact’ – we don’t
question it

• No one (originally) tried to mislead us, but

collectively we’ve all been misled
Big numbers can confuse us
• Former UK government (Labour) said it would spend
£300 million on childcare over 5 years to create extra
childcare places

• Is £300 million a lot of money?

• Equals @ £300 per child, over

a five-year period

• £60 per child per year, or

£5 per child per month
Statistics
can be
weaponised:

Correct but
misleading
figures
(link 1, 2)
Correct but misleading figures
• Didn’t count inflation: £1 in 2020 is worth less than £1 in
earlier years (= 88p in 2015)

• Misleading y-axis

• Didn’t look at spending per pupil

– Pupil numbers have risen by a large amount
– Spending per pupil is actually down
– Spending per pupil dropped by 8% in
real terms from 2009/10 - 2017/18
We are easily fooled by numbers
1. Numbers often seem like objective facts –
e.g. two litres of water per day

2. Doing maths can be a bit of a pain

3. Even if you feel very confident with numbers,

how do you know which numbers are accurate
and which aren’t?
a. Eg Talking baseball with Brits
But we are easily fooled by stories too
• Narratives (stories) can produce as much nonsense as
numbers, or even more

• We have evolved as a story-

producing, story-telling species
– Hardwired to create and believe
causal stories
We are programmed to be misled by
•stories
Causal story: If A happens and then B happens, A
must have caused B
• The cost of overlooking a
causal story was historically
much greater than the cost of
wrongly believing in one
• Why? Discuss at your tables
for 1-2 minutes
Misled by stories: Conspiracy theories about
illnesses and vaccines
Covid-19 conspiracy theories:
• Covid-19 denialism (“It doesn’t exist!”)
• Vaccine hesitancy (“Vaccines don’t work,
or are part of plot to control us”)
Old-school conspiracy theories: MMR vaccine
• Some people believe the MMR vaccine (Measles, Mumps
and Rubella) causes autism
– The age at which it’s best for kids get the vaccine is just before
the age kids start showing signs of autism
– This theory was advanced by a dodgy doctor/scientist who had a
financial interest in “disproving” the MMR vaccine

• The scientific evidence is overwhelming: the MMR vaccine

does not cause autism
• But compelling stories can be more powerful than scientific
evidence
Misled by stories: conspiracy theories
Quote from a mother: “What number [of children]
does it have to be for everyone to start listening to
what the mothers of children who have autism have
been saying for years, which is … We vaccinated our
baby and SOMETHING happened. SOMETHING
happened. Why won’t anyone believe us?”

These stories are very moving, and very

powerful – they have led to a multi-year
drop in vaccinations, and a rise in measles
in countries around the world
Some types of nonsense
1. Believable: Based in reality, sounds very
logical (eg 2 litres of water per day)
2. Fantastical: So extreme that it’s more a test of group loyalty
than anything else (eg Trump really won the 2020 US
election; Covid-19 isn’t real)
3. True but misleading: True information presented in a way that
reduces our understanding rather than increases it

Discuss at your tables: What makes a good lie, and/or a good

liar? Do you know any good liars – what makes them good at it?
Some factors contributing to nonsense &
1. There’s so much we don’t know, even if we’re very well
confusion
informed compared to most people
2. Some people want to mislead us, so they use information
in misleading ways
3. We are easily misled, both by numbers and stories (no
matter how bright we are)
4. Often we are willingly misled (perhaps
unconsciously), by ‘evidence’ that fits
our own view of the world, or our own biases
How much do you know about the
UK/China?
In table groups, come up with answers for these four questions.
Don’t look it up online – guess!

1. What is the average age of the UK’s population?

2. What is the average age of China’s population?)
3. What percentage of the UK population are immigrants to this
country (i.e. not born in the UK)?
4. What percentage of the China population are immigrants to that
country (i.e. not born in the China)?

Write your group’s guesses on this google doc:

https://docs.google.com/spreadsheets/d/1mzwAl0E5zSQIiKGn4A0QwQeBZ_AC7KgvVDZ7-1UBlKY/edit?us
p=sharing
 Don’t look at the next few slides, as they
contain the answers to the previous
questions. It’s more fun and interesting if
you don’t know the answers in advance

Guardian’s ‘How well do you really know your

country?’ quiz
https://www.theguardian.com/world/ng-interactive/2015/dec/02/how-well-do-you-really-know-your-country-take-our-quiz
Appendix
Correct but misleading figures
• “UK Govt announces extra £1bn for tutoring
and £400m more for teacher training, as part
of its programme to try and make up for lost
learning” during the pandemic
• Is £1.4bn a lot of money?
• This is roughly £100/pupil
• Estimated need to make up for
lost learning = £13.5bn