Professional Documents
Culture Documents
Lesson 1112
Lesson 1112
OVERVIEW
This chapter describes the various types of assessments used in medical education
before applying the general rules discussed in Chapter 10 to specific examples. It
should be noted that, while there are some excellent general books on exam technique
in higher education (see References and further reading), it is difficult to find any that
provide useful advice on the variety of assessments used specifically in medical
courses. Each medical school appears to add its own flavor to its assessments, so
supplement these rather generic pages with discussions with other students in your
year, those in years above, any past papers in the library, and any friendly faculty
members you meet.
Introduction
It would be tedious to end each paragraph with a disclaimer, so here's one:
good exam technique allows you to effectively demonstrate what you
know/can do. It does not, in any way, take the place of having first learned
the material! You should also remember from the first section of Chapter 10
that you should make every effort to learn the exact format of your school's
exams. As you will see further down, everyone has their own unique
variations based on common themes.
c11.qxd 1/9/09 10:44 Page 144
Types of exams
Multiple choice exams (MCQ)
We mean any assessment in which you are asked to choose between
different options in order to demonstrate that you know something. These
are popular exams, not least because they are simple to mark using
automatic systems such as optical readers of mark sheets (you mark in
pencil, and a computer scanner reads your answers and calculates your
marks, as well as a slew of exam statistics in seconds) and even direct-entry
exams where you interact with a computer (removing the need for
someone to stand over a computer scanner cursing every time it gets
jammed). Examples of various multiple choice formats are provided.
Q7 In terms of diabetes
A Insulin resistance is more common in type 2 (late-onset) diabetes than type 1 diabetes (T)
B A fasting blood glucose level of more than 5.0 is diagnostic (F)
C Women who develop diabetes during pregnancy have a higher risk of developing diabetes
later in life (T)
Best of five
In these questions, you must choose from a list of five items the one that is
most true or relevant to the question's lead in (or'stem'). This allows for
some uncertainty and judgment. Q8 demonstrates how to convert some
simple true-false questions to a best-of-five format (here best of three).
Both items B and C in Q9 may be true, but B is less clear and more debatable
than C, so C is the correct answer.
c11.qxd 1/9/09 10:44 Page 145
Each of the descriptions below corresponds to a different type of evaluation; select the
single most appropriate answer from the list above
Q10 This knowledge-based exam was given to a 1980s graduate who had to choose
whether or not each of 600 basic and clinical sciences items were true or false
Q11 A medical school decides to replace traditional assessments in the final year with this
method, which requires students to gather evidence from a variety of sources to
demonstrate that they possess the necessary characteristics to graduate.
c11.qxd 1/9/09 10:44 Page 146
Exams in writing
Essays
All students will be familiar with essays. You will be required to write essays
in medical school, most likely as assignments within coursework and, in
some schools, as part of formal exams. Essays are time-consuming to grade
and can be difficult to grade fairly. As a result, they appear to appear less
frequently in medical school assessments, and when they do, students may
feel underprepared. We did not include an essay section in this chapter, but
we did include some information about essays and academic writing in
Chapter 6.
Q13 Describe the classic acute symptoms of a heart attack (4 marks) Describe the key
findings that are likely to be discovered during an examination of a patient suffering
from a myocardial infarction (3 marks)
Clinical examinations
It is obvious that if you are to graduate as a doctor, you must be able to
demonstrate that you can apply your knowledge and skills directly with
patients. This was traditionally tested through long and short cases: long
cases in which you saw one or two patients, took their history, examined
them, and tried to figure out what was wrong before presenting your
findings to the examiners and answering questions, and short cases in
which the examiners took you to see 'bits' of patients to see if you could
spot what was wrong.
c11.qxd 1/9/09 10:44 Page 147
In one session, you could see three different types of hand arthritis, listen to
a couple of different heart murmurs, and feel a lump in someone's
abdomen, all while barely having time to introduce yourself to the patients
associated with these various clinical signs.
These exams had their advantages, but two students could have very
different experiences, with one having 'easier' patients than the other.
Long and short cases have been mostly (but not entirely) replaced by
Objective Structured Clinical Examinations (OSCEs). Consider a large hall
with 24 booths on each side of the walls. Each booth puts a different skill to
the test. In booth or'station' 1, a student may be expected to draw blood
from a rubber arm; in station 2, a student may be expected to ask a patient
about their chest pain in order to make a diagnosis; and in station 3,
students may be expected to examine a patient's arthritic hands, among
other things. Imagine a big hall, with 24 booths around the walls – six on
each side. Each booth tests a different skill. In booth or ‘station’ 1, a student
might be expected to take blood from a rubber arm, in station 2 a student
might be expected to ask a patient about their chest pain in order to come to
a diagnosis, in station 3 per-haps students examine a patient’s arthritic
hands and so forth.
Add some variations in timing (perhaps some stations last 10 minutes
and others last 5 minutes), design (perhaps it is made up of two 12-station
exams held at different times, with some'rest' stations where students can
sit and worry), and location (perhaps it is not in one large hall, but instead
students move in and out of pokey offices and tutorial rooms desperately
looking for the next station), and you have an OSCE. Fair because every
student has the same experience, safe because a wide range of skills are
tested, so unsafe students aren't able to slip through unnoticed, incredibly
boring for the examiners, and either quite fun or absolute torture for the
students depending on their perspective. Whether you like them or despise
them, OSCEs are still the gold standard for assessing clinical competence and
will be for some time.
Clearly, OSCEs are not very realistic, and various work-based
assessments have been proposed in response to calls for assessments that
truly assess what people can and do do in the real-life care of patients. The
Mini Clinical Evaluation Exercise is the most commonly used in the United
Kingdom (Mini CEX, pronounced mini C-E-X). This theme has many
variations, but in general, the candidate performs some kind of task,
usually with a patient, and is observed and graded by a senior using a semi-
structured pro forma. Many different people observe them over the
months and years as they perform many different clinical tasks with many
different patients, and an overall picture or judgement about the learner's
ability can be made.
c11.qxd 1/9/09 10:44 Page 148
The Mini CEX has the advantage of being used in a real-life setting, and the
assessor can provide rapid feedback to the learner to help them develop. As
you may recall from Chapter 3, this does not happen very often in the clinical
setting, so it must be a good thing. On the negative side, the original Mini
CEX form is probably a little too generic for detailed, focused feedback in
junior learners, and there have been some issues getting people to fill them
out.
Oral examinations
Oral exams are part of the history and tradition of medical education, and
you may find yourself taking a viva voce exam in front of two or three
examiners. Institutions now reserve these assessments for students on the
borderline or who may be eligible for a prize, though many have abandoned
them entirely. Even if you take an inter-calated BSc, you will almost certainly
have a thesis viva.
Historically, candidates have reacted with fear to this type of exam. The
format is rather elegant: the examiners ask the student questions about a
topic, each one more difficult than the last, until they determine the
candidate's upper limit of knowledge on that topic, at which point they
move on to another topic and work their way up the ladder again. They can
build a picture of the candidate's depth of knowledge from a variety of core
areas in a relatively short period of time. Unfortunately, whether they were
good or bad, the frustrated candidate leaves the exam remembering only
that they got the last question of every topic wrong, and thus feels terrible.
Thematic variations
So far, we have attempted to provide an overview of the major assessment
methods in this chapter. Of course, there are many variations, and your
institution's exams may differ. The OSCE, for example, appears differently
depending on where you see it, and sometimes people will take the OSCE
concept and apply it to something else, such as an anatomy spotter exam,
in which students rotate around anatomy specimens answering questions,
or data interpretation exams, in which students rotate around
radiographs, urine results, blood test results, and so on. In general, people
rename these exams anything that sounds like OSCE: OSPE, OSLER, OSTE—if
you're bored, try making up your own OSCE-like acronyms and entering
them into PubMed. You're likely to discover that someone else has already
coined your acronym! Mini-CEX, SAQs, and good old MCQs have all given
birth to a plethora of variations. Our call from Chapter 10 remains: make
sure you understand your institution's assessment program, what it entails,
what format it takes, and what purpose it serves.
c11.qxd 1/9/09 10:44 Page 149
There will occasionally be poorly written items that will provide you
with some hints. Look for items that are significantly longer or shorter
than their neighbors (in Q15, D is correct), and remember that 'never' is
always wrong and 'always' is never right.', only a guideline (very little in
medicine is
c11.qxd 1/9/09 10:44 Page 150
in Q16, you don't need to know much about items A, B, C, and E to guess
that they're false, leaving only item D as the correct answer.
A Gene on chromosome 1
B On chromosome x
C On chromosome 5
D On chromosome 7 and is in charge of coding for a protein that regulates ion
and fluid flux across cell membranes
E On chromosome 8
Q16 Concerning chest pain
Is it
Begin with: Do I Yes included on Yes Make a note of it
know the answer? the list?
No
No
s
Ye
Is there
anything
obviously incorrect
on the list?
No
Sort the items on the
list into impossible, Make an
improbable, educated
possible, and likely guess
Figure 11.1 Answering extended matching questions is a process.
friend who got them from someone who swore they were real exam
questions from a long time ago People make the mistake of attempting to
memorize every question they encounter. Your memory will not be able to
hold all of these questions, and if you get a rush of adrenaline during the
exam, you may recall seeing the question before, or at least a question
similar to it, but the answer, hmm, that's another question. This will result
in consternation and frustration. Even if you do have everything memorised,
the chances are that the questions will not be identical to the ones you learned
(examiners do not use books of exam questions to set the exams and they
tend to rewrite, write new and edit existing questions in the bank of questions
that your insti-tution uses).
Past papers and external questions can be extremely beneficial if used
correctly, specifically for two purposes. To begin, use them for timed mock
exams to familiarize yourself with the format and timing of the exam.
Second, they can be used as a diagnostic tool. Examine your answers, and if
you get a question wrong, jot down the area to which it relates. If you get a
couple of questions on similar topics wrong, it means you need to go back
over your notes and go over those topics again. Remember, instead of
memorizing answers to questions, try to understand the topic.
is likely that if there are three marks for 'Describing the key findings likely on
examination of a patient suffering from a myocardial infarction' (Q13
above), it is likely that the examiner is looking for three main points; unless
the mark scheme gives half a mark for each main point, implying that the
examiner is expecting six main points. Regardless of the difficulty, looking at
the marks should help you structure your response. If there is only one mark
for a definition, you may write slightly less than if there are five marks.
We've already talked about how important timing is and how you should
answer every question. We also discussed planning. It is worthwhile to
spend a minute or two planning your answer: most schools will encourage
you to include your rough plan in your answer, with a single line through it
to indicate that it is rough. If you run out of time, you may still receive one or
two points.
If you carefully read Chapter 2 (note taking), you will have a good idea of
how to prepare for this type of exam. You can incorporate your own
questions into your daily note taking and practice answering them as a way
to review your studies. Of course, you'll want to have full-length practice
exams, but you should also be well prepared. Perhaps you will use a
combination of your own and external questions. Maybe you could share
your concerns with other students and form a study group.
SAQ papers make it simple to assess factual recall and even
comprehension. Writing a SAQ that tests the more advanced skills discussed
in Chapter 2, such as application, analysis, and synthesis, is much more
difficult. Very skilled question writers may attempt to test these higher
skills, but these are more often than not reserved for other formats such as
debates, vivas, and, of course, the essay.
Students who work hard to become good doctors, to learn the knowledge
and skills needed for the job, and to become good at patient care, should
easily pass the OSCE (with one or two caveats discussed below).
Unfortunately, some students see the OSCE as a higher education hurdle
unrelated to patients or being a doctor, and they focus their learning on
passing the exam rather than becoming a good doctor. They devise
elaborate plans to 'perform well,' but they frequently fail.
Q17 You are an A&E senior medical student. The registrar has requested that you
collect a venous blood sample to test for anaemia. Please demonstrate how you
would take blood as if this rubber arm were your patient.
Good students will have sought out The exam-centered Students will have
numerous opportunities on the wards to mostly practiced in the skills lab, knowing
take blood (with appropriate consent, of that the rubber arms used in the exam are
course, see Chapter 3), occasionally not the same as real arms. They will have
asking doctors, nurses, or peers to memorized a list of steps, possibly from an
observe them and ensure that they are OSCE book, and will have practiced
doing it correctly. They will have become reciting the steps with robotic accuracy.
acquainted with the various pieces of Because their training and practice will be
equipment available, including which solely focused on the exam, they will be
bottles are used for which tests, they will unlikely to perform well in real life.
have developed rapport with patients
when taking their blood, and it is likely
that they once forgot to write the
patient's details on the sample and had to
redo them with many apologies to the
poor patient, and they will never, ever do
this again.
These students will perform this skill These students' performance in the
fluently and professionally in the exam, exam will be wooden, and they may
even on a rubber arm. It will be obvious miss steps that would be critical in real
that they have done it numerous times life but are less obvious in simulation
before. (for example, forgetting to write the
patient's details on the bottle, or not
knowing which bottle is used for which
test).
This student passes the exam, but This student will most likely fail the skill
more importantly, you would consent or barely pass it, but you would never let
to them drawing blood from your them near your favorite aunt with a
favorite aunt. needle.
c11.qxd 1/9/09 10:44 Page 154
Looking at the two examples in Q17, it should be clear that those who aim
solely to pass the exam, to be OSCE competents, tend to perform worse on
the exam than those who aim to become clinically (with patients)
competent and pass the exam as a'side-effect.'
This is a critical point that cannot be overstated. Ninety-five percent of
your clinical and communication skills training should be devoted to
becoming a great doctor, with the remaining 5% devoted to exam
technique. With a few exam technique tips and some practice, you'll be able
to breeze through the exam.
Suspended disbelief
You've probably seen a movie that engrossed you so much that you forgot
you were watching a movie and instead felt like you were a part of it. Other
times, you've watched a movie and all you could see were mediocre actors
on a screen. You might have been critiquing the script, the direction, or the
soundtrack in your head. It was far from 'authentic.'
You've probably tried to move from one extreme to the other - you
noticed that you were unengaged in the film and took steps to be drawn into
it, or the film was so disturbing (perhaps a horror film that was too scary, or
an erotic moment in a film you watched with your parents) that you
distanced yourself from what you were watching. Spend a minute making a
list of things you've done to draw yourself into the film and things you've
done to distance yourself from it (two columns). Go ahead, do it now; you're
probably tired of reading anyway.
The OSCE is comparable. Students who perceive it as an exam, something
synthetic, or even a game may struggle to engage with it. Students who can
imagine that each scenario is real, or who can'suspend their disbelief' and
approach each station as if the actor in front of them is a real patient,
perform significantly better.
c11.qxd 1/9/09 10:44 Page 155
Q18 You are a general practice medical student. The doctor has asked you to take a
history from the next patient, Mr Leo Hill, who has recently complained of chest pain.
Please take a detailed history before making a diagnosis. You only have 5 minutes.
Student A enters the station, shows the examiner her name badge, and sits down,
making sure the examiner is out of her line of sight. She introduces herself to the
patient and assumes he is a general practice patient. She inquires about his pain
and, as he speaks, wonders what is causing it. She naturally asks open ('could you
tell me a little more about the pain?') and closed ('does the pain ever move down
your arm?') questions, summarizing as she goes to ensure she understands. She is
interested in his thoughts on the cause, and she discovers that he is terrified that it is
coming from his heart. She understands how upsetting that must be, especially since
his brother died recently of a heart attack.
Overall, she obtains a detailed history from the patient in front of her. She gets a good
sense of how his symptoms are affecting him and believes she has ruled out a cardiac
cause, suspecting that his symptoms are caused by acid reflux into the esophagus or a
stomach ulcer.
When the bell rings, the student becomes disoriented for a brief moment. She had
forgotten she was taking a test. She gathers her thoughts, thanks the patient, and
moves on.
Student B enters the station and shows the examiner his name badge. He notices the
examiner holding the mark scheme and wonders what it says. He anticipates that the
first mark will be for introducing himself, so he turns to the actor and introduces
himself as he has practiced for the OSCE. From the corner of his eye, he watches the
examiner to see if he ticks anything off on the mark scheme, and sure enough, it
looks like he got a mark for the introduction - things are going well. He's learned a lot
about previous exam questions, and he's wondering if this is the station from 2007
about carpal tunnel syndrome or one from 2006 about acid reflux.
He asks, 'Can you tell me about the pain?' knowing that the mark scheme will almost
certainly reward using some open questions, and notices the examiner moving his
pencil on the mark sheet - another point. 'Yes, it is quite frightening,' says the actor. It
usually hurts just here,' he says, pointing up and down the middle of his chest with the
flat of his hand. The student understands that an actor would have been trained to use
a fist if the pain was angina or a heart attack, and he feels pretty good about himself for
identifying that this is the acid reflux station from 2006, so he smiles at the actor and
says 'good'.
c11.qxd 1/9/09 10:44 Page 156
He overhears one of his colleagues say loudly, 'I'd just like to examine your abdomen,'
and makes a mental note that there will be an abdominal examination station later in
the circuit. His smile transforms into a grin. He is aware that he has prepared for an
abdominal examination.
He asks the actor some additional questions to demonstrate to the examiner that he
understands the diagnosis: 'Is it worse when you lie flat?' 'Do you take milk, antacids,
and cimetidine to help it?' The actor appears distressed but nods, and the student
smiles, seeing an opportunity to gain a mark for empathy, and says, 'Yes, I know exactly
how distressing this must feel for you.'
The student finishes early and sits back to hear what the next station is about. He
smiles when he hears the word warfarin.
The examiner was paying close attention to the two performances. Her grade scheme
rewarded students for taking a patient-centered history, investigating symptoms,
investigating all possible causes of chest pain, and being empathic and appropriately
reassuring the patient. The first student did an excellent job, and while she appeared
to be less certain of a final diagnosis, she had thoroughly investigated all possible
causes of chest pain. She appeared to be genuinely concerned about how the patient
felt. She received high marks for being fluent and professional, as well as having a
patient-centered approach. The second student did not perform well; in fact, his
behavior was unusual. He didn't ask the standard questions to rule out potential
cardiac causes and seemed to jump right to one diagnosis, without asking nearly
enough questions to confirm it safely. He smiled in the most unexpected places. He
received no credit for investigating the nature of the pain or investigating other
possible causes. The examiner was convinced that this candidate should fail.
Each candidate was asked to rate the actor's empathy. Although candidate A did not
make any 'empathic statements,' it was clear to him that she could truly understand
the patient's point of view, and he gave her a high rating. Candidate B struck him as
odd: he smiled at odd times, and while he made an empathic statement, it was clear
that there was no empathy behind it at all. He gave him a zero.
('how ridiculous, that's not something you'd hear in real life') You may also
consider other things outside of the film to bring you back to reality (what
could you have for dinner?). To draw yourself in, you could try to forget
where you are by ignoring distractions and positioning yourself to minimize
distractions in your line of sight; you could also try to form an emotional
connection with what you are watching - put yourself in the shoes of the
lead actor and see how things affect him or her. In the OSCE, you can use the
same skills - ignore distractions, focus on the patient, and try to ignore the
examiner. If you can get them out of your line of sight, that's even better.
You should practice gaining this focus. Perhaps you and your friends could
act out some stations. When you've mastered throwing yourself into the
role at hand, you might want to try it in a less distracting environment, such
as the hospital canteen or public transportation. If you miss your train or bus
stop because you are preoccupied with determining what is wrong with your
'patient,' you will know you are doing well. Consider what might cause you
to lose focus, such as the examiner interrupting with a question, and
practice how you can learn to manage these distractions and quickly return
to your role.
Time management
Some students excel in clinical practice but struggle during the OSCE
because they are constantly running out of time. They may never have taken
a 5-minute focused history or examined the nervous system of the legs in 5
minutes in real life, so they become stuck.
'My name is Peter Muthos, and I am a third-year medical student.' I'm just getting
ready for an exam and was wondering if you wouldn't mind me practicing by
c11.qxd 1/9/09 10:44 Page 158
testing the nerves of your face. This would entail me simply looking at your face,
testing how strong the muscles around your head and neck are, and seeing how
sensitive your skin is to touch. I'd like to do this with a friend who will be timing me
for the exam. This is only for my education, so no worries if you say no.'
Although 95% of your practice time should be spent on becoming clinically
competent, you should also practice timed scenarios. You can still do this
with patients, and having a peer time you and provide feedback will help
you learn a lot. You will feel rushed at first, but if you become fluent in
taking a rapid, effective, focused, patient-centered history and conducting
appropriate focused, diagnostic examinations, you will quickly become a
popular junior doctor, particularly in outpatient clinics, A&E, and general
practice.
Q19 You're a senior medical student on a surgical rotation. The next patient will be
referred for an upper gastrointestinal endoscopy by your team. Please inform the
patient of the procedure.
Student 1: 'Good day, my name is... I understand you've been referred for an
endoscopy, and I've been asked to come and explain the procedure to you. I must
admit that I am not an expert on the subject, but I thought that if I could find out what
you already understand and what your main questions are, I could explain as best I
could, and if there are any outstanding questions, I will either find out and get back to
you, or I will ask one of my seniors to come and talk to you. 'Would that be okay?'
Student 2: (invents incorrect facts): 'Ah well, yes, hmm, an endoscopy - well, we will
pass a big telescope down your throat, erhm, but don't worry, you will be put to sleep
first, hmmm, and we will keep you in overnight to make sure you are ok afterward
[pause] ah yes, risk, hmm, well, no risk at all, very safe, unless...
For some reason, some (sometimes many) students believe that lying in
an OSCE is acceptable. True, saying 'I don't know' and sitting silent for the
next 4 minutes and 50 seconds won't help, but admitting uncertainty,
learning about the patient's perspective and prior knowledge, and
proposing a plan for getting the information you don't know back to the
patient will.
The examination
As previously stated, vivas are becoming less common. Knowing the
structure and format helps (the fact that they will always ask up to your level
of 'not knowing,' as mentioned above, means that you can remain confident
even if you make mistakes).
c11.qxd 1/9/09 10:44 Page 162
The ability to structure your answer well is probably the most important
skill in answering a viva.
Student 3: 'Hmm, well, erhm, it's where a pregnancy develops in the fallopian tube,
usually around 6 weeks of pregnancy, erhm, it can be quite dangerous...'
Student 4: 'Ruptured ectopic pregnancy is a gynaecological emergency that is treated
by making a rapid diagnosis through history, examination, and special investigations
while simultaneously resuscitating the patient, followed by emergency surgical
treatment.' The historical features that would identify this diagnosis are...'
In this example (Q20), student P actually provided more hard facts than
student Q, but student Q began with confidence (without saying much), set
the scene (making it clear that this is an emergency), and laid out a strategy
for continuing to answer the question (he will talk about history,
examination, special investigations, resuscitation and surgical treat-ment).
He will persuade the examiner that he is a clear thinker who understands
the information by providing this answer and signposting each time he
moves on to a separate part of the answer. Some examiners may even move
him on to another topic after that introduction, believing that he has already
demonstrated that he knows more than enough about this topic.
If vivas are likely in your curriculum, get plenty of practice by recording
yourself to practice fluency and structuring your answers, using your peers
to practice answering unexpected questions, and using any faculty who have
experience with 'vivaring' to practice how you would handle it in real life.
Practicing these skills will come in handy later in life for job interviews, etc.
Summary
Chapters 10 and 11 attempted to highlight some of the common pitfalls
encountered in medical school assessment and how to avoid them. Good
exam technique does not replace appropriate learning; rather, it
complements it. The tone of this book in general, and the section on OSCEs
in particular, should have made it clear that the goal of medical school is to
prepare you to become an excellent doctor. Exams are a part of that
journey, but they should not be viewed as the end goal.
c11.qxd 1/9/09 10:44 Page 163
A: MCQ papers can be difficult. Some students have a tendency to answer more
questions correctly the first time, then become indecisive and 'correct' them,
resulting in incorrect answers when they look back at the end. Checking helps other
students improve their grades. Perform some practice questions and keep track of
which answers you change. Add up your score before and after making changes to
see how much of a difference it makes for you. If you do this for a few mock exams,
you should notice a pattern emerge.
Q: I understand that I shouldn't copy someone else's text from their work without
attribution, but isn't it acceptable to paraphrase?
A: If you use someone else's ideas, you must credit them. 'I think, therefore I am,' said
Descartes. Even if you change it to 'I think, therefore I exist,' it is still his idea, not
yours. Plagiarism is more difficult to detect than 'cut and pasted' plagiarism, but it is
no less wrong.
Q: In a multiple choice question paper, should I leave the questions I'm unsure about
blank and come back to them later, or should I guess?
A: This is determined by the type of exam, the amount of time available, and your
personal preferences. In general, it is preferable to answer everything as you go,
leaving no questions unanswered. This makes it easier to time accurately and reduces
the possibility of you becoming out of sync with the question sheet and the mark
sheet. If you are permitted to rub out answers, mark the question sheet to highlight
the ones you have guessed so that you can go back and correct them.