JOURNAL OF PATHOLOGY, VOL.
179 343-345 (1996)
MEDICAL EDUCATION
THE SCENARIO PAPER: AN INTEGRATED
EXAMINATION FOR AN INTEGRATED COURSE
EMYR W. BENBOW AND ANTHONY J. FREEMONT
Department of Pathological Sciences, University of Munchesrer, Munchester, U.K.
BACKGROUND
In 1993, the General Medical Council made a number
of recommendations about undergraduate medical edu-
cation in the United Kingdom; amongst other things, it
recommended that curriculum overload be addressed,
that the traditional emphasis on rote learning (at the
expense of understanding) should be discarded, and that
there should be erosion of traditional interdisciplinary
boundaries. In anticipation of these recommendations,
major modifications were instituted in the third
M.B.Ch.B. course at Manchester, beginning in 1991.
The course became the first managed course in the
Faculty of Medicine, overseen by a management group
of individuals from various disciplines within pathology,
together with a pharmacologist and several clinicians,
including a clinical epidemiologist and a general
practitioner.
The management group instituted the development
of a carefully integrated structure, based on a series of
seminars about each major body system; there were
also seminars, of a more general nature, on the princi-
ples of microbiology, therapeutics, and chemical
pathology. Most of these seminars were based on the
pathological disciplines, with extensive integration of
topics from radiology, therapeutics, epidemiology,
medicine, surgery, gynaecology, paediatrics, and psy-
chiatry. A substantial reduction in didactic teaching
was achieved, with abandonment of traditional lectures
in favour of integrated talks, each lasting between 5
and 30 min, with very few contributions beyond these
limits.
A major upheaval in the fourth year of the course
compressed the time available for the third M.B.Ch.B.
examination, so traditional essay questions were aban-
doned in favour of a ‘scenario’ paper based loosely on
modified essay questions (MEQs) pioneered by the
Royal College of General Practitioners.2 This can be
marked quickly and better fulfils the educational objec-
tives of the course, testing a greater breadth of knowl-
edge than essays and a greater depth of understanding
than multiple choice questions (MCQs).
Addressee for correspondence: Dr E. W. Benbow, Department of
Pathological Sciences, Stopford Building, University of Manchester,
Oxford Road, Manchester, M13 9PT, U.K.
CCC 0022-3417/96/110343-03
CQ 1996 by John Wiley & Sons, Ltd.
THE SCENARIO PAPER
The scenario paper consists of a number of scenario
problems, each of which may generate a different
number of possible marks. The number of scenario
problems in any particular paper is not predetermined;
rather, scenarios are selected to provide a predetermined
number of overall marks. The typical scenario problem
commences with the description of a simple clinical case,
and as the history unfolds, a series of questions are
interspersed into the text. The number of marks avail-
able for each question is indicated alongside the question
(Appendix I). Experience shows that a paper generating
120 marks, using a mark scheme like that in Table A1 of
Appendix 11, can be completed within 3 h; such a paper
typically includes 12-14 scenario problems.
Problems are submitted by members of the manage-
ment group, together with a marking schedule for each
problem. Problems and marking schedules are scruti-
nized and modified by an emendations committee made
up from members of several traditional disciplines.
Problems are selected to give broad coverage of the
course, but with little overlap.
Various forms of preparation are organized for the
students. Problems, similar in structure to those used in
the examination, are linked to the didactic teaching.
They may be introduced during 1 week’s teaching and
discussed in an interactive theatre session the following
week; alternatively, they can be worked through in
small-group tutorials.
Marking of scripts is achieved in a single group
marking session, to which markers from various disci-
plines are invited, so that their combined knowledge and
experience allows informed discussion of any responses
not anticipated by the question-setter or the emendation
committee. Mark schemes are displayed on overhead
projector acetates during these sessions and can be
modified with the mutual agreement of the markers.
Typically a group of 15 markers can deal with 275
scripts in about 4 h; copious quantities of coffee are
essential.
ADVANTAGES AND DISADVANTAGES OF
SCENARIO PAPERS
Scenario papers have a number of practical advan-
tages over other forms of written examination. They can
Received 11 August 1995
Accepted 19 February 1996
344 E. W. BENBOW AND A. J. FREEMONT
be marked easily, reliably, and promptly, with very little
leeway for different interpretations between different
examiners, especially when marking schedules are pre-
pared ~ a r e f u l l y . ~Previous
-~ studies of MEQs indicate
high inter-rater reliability, especially for items presented
within the context of a case,s and show that they are a
valid measure of problem-solving skills.6 In addition,
they appear to be good predictors of clinical compe-
t e n ~ e .With
~ . ~ experience, it is possible to ensure a broad
spread of marks, both by varying the difficulty of the
material in the scenario problem and by modifying the
stringency applied to individual questions. For instance,
if there are four possible responses to a particular
question, marking can be made generous by allowing
two marks for each of any two correct responses or,
alternatively, the item may be made more searching by
ascribing a half-mark to each of four correct answers.
Questions can be written that test knowledge at different
cognitive levels, ranging from factual recall, to data
interpretation, to problem-solving.~8
Scenario problems allow easy integration of several
traditional disciplines; for instance, any demographic
data in the case history should be made consistent with
the epidemiology of the disease in question. The first
problem that the candidate may have to solve is one of
differential diagnosis of a medical or surgical problem,
followed by the selection of an appropriate investigation
and prediction of likely results; alternatively, data may
be provided for interpretation. There may then be
further questions on therapeutics, natural history, or
prognosis after treatment. Each individual problem
therefore includes coverage of a broad range of topics,
approaching that of an MCQ paper, without the oppor-
tunity for students to 'spot' likely topics, as they may do
with essay papers. Scenarios avoid the cueing effect of
MCQs, a feature thought to favour the less competent
student.
The scenario paper is particularly well suited to an
examination centred on the pathological disciplines and
serves to emphasize the central role of the pathological
sciences in clinical care. Carefully written scenarios
simulate some of the important intellectual tasks seen in
clinical practice, and should complement the examin-
ation of practical skills seen in OSCEs (Objective
Structured Clinical examination^).^.]^ They also prepare
students for MEQs used in postgraduate examinations
by some of the medical Royal Colleges.
There are also advantages for staff. Involvement in
marking gives members of the management team a clear
view of the strengths and weaknesses of the course
and there are opportunities for staff to learn from
other participants in multidisciplinary emendation and
marking teams.
The disadvantages of the scenario paper are few.
Preparation takes longer than for the essay paper, for
obvious reasons, and initially is more time-consuming
than the preparation of MCQ papers because there is
only a small bank of scenario questions; eventually, we
expect the preparation time to be similar." Marking is
more time-consuming than the marking of MCQs, but
much less time-consuming than the double marking
of essay papers. Choosing a time when a sufficient
number of markers can be accumulated at one time can
be difficult, but the marking sessions can be both
instructive and educational; most of those invited to the
marking sessions participate with enthusiasm and
no-one has yet refused the opportunity to repeat the
experience.
The most difficult problem is knowing how to ascribe
marks where a student makes an incorrect response
early within a particular problem, in a way that influ-
ences possible answers to later questions.'' Some argue
that weaker students are more likely to stumble at early
hurdles and should be penalized in any subsequent
answer that deviates from the marking scenario, whereas
others argue that marks should be awarded on the merit
of individual answers. The second course is more diffi-
cult to follow and subjectively seems fairer; however, the
first course rewards candidates who learn in the inte-
grated manner that we desire. The most important
aspect of dealing with this problem is to anticipate it,
choosing one of these two possible strategies by mutual
consent.
COMMENTS
The form of examinations governs the manner in
which students approach learning'2.'3 and may even
influence the form of teaching.I4 This phenomenon can
be detrimental to the educational experience, but it
is difficult to avoid. We seek to take advantage of it
by ensuring that our examinations closely match the
objectives of the course.*
Both MCQ and essay papers can encourage an
approach based entirely on the rote learning of facts,
designated surface learning by Newble and Entwistle. I s
We believe that the scenario paper encourages students
to combine the learning of facts with understanding of
the subject matter, so-called strategic learning.Is In our
hands, scenario papers provide junior clinical students
with an introduction to tests of clinical reasoning,
another laudable aim of modern medical education.I6
Pathologists may feel threatened by the horizontal
and vertical integration of newly-designed medical
courses, feeling that they will be squeezed out of their
traditional safe enclave in the third year of the conven-
tional course. Such course revisions should not be seen
as a threat, but rather as an opportunity to spread the
influence of one's discipline more widely. Scenario
papers are well suited to the examination of courses
which have integrated and problem-based learning for-
m a t ~ and
' ~ pathologists are ideally placed to take a
central role in the utilization of this tool. Written with
care, scenario papers can enhance students' perception
of pathology as central to clinical endeavour.
REFERENCES
1. General Medical Council. Tomorrow's Doctors: Recommendations on
Undergraduate Medical Education. London: GMC, 1993.
2. Hodgkin K, Knox JDE. Problem Centred Learning. London: Churchill
Livingstone, 1975.
3. Newble DI, Baxter A, Emslie RG. A comparison of multiple-choice tests
and free-response tests in examinations of clinical competence. Mrd Educ
1979; 1 3 263-268.
 THE SCENARIO EXAMINATION 345

4. Newble D1, Hoare J, Emslie RG. The validity and reliability of a new renal parenchyma. The surface of the brain is covered in
examination of the clinical competence of medical students. Med Educ 1981;
15: 4652. blood. The pathologist proceeds to a more detailed
5. Norman GR, Smith EKM, Powles ACP, Rooney PJ, Henry NL, Dodd PE. examination of the brain to establish a site for the
Factors underlying performance on written tests of knowledge. Med Educ bleeding.
1987; 21: 297-304.
6. Feletti GI. Reliability and validity studies on modified essay questions.
J Med Educ 1980: 5 5 933-941. Q2. What is the pathologist looking for? ( 1 murk)
7. Irwin WG, Bamber JH. The cognitive structure of the modified essay
question. Med Educ 1982; 1 6 326-331
8. Feletti GI, Smith EKM. Modified essay questions: are they worth the effort? Q3. Where are the most likely sites,for this lesion?
Med Educ 1966; 2 0 126-132. (2 marks)
9. Harden RM, Gleeson FA. Assessment of clinical competence using an
objective structured clinical examination (OSCE). Med Educ 1979; 13:
41-54. Q4. What is the most likely explanation of theJindings in
10. Selby C, Osman L, Davis M, Lee M. How to do it: set up and run an the kidneys? ( 1 mark)
objective structured clinical exam. Br Med J 1995; 310 1187-1190.
1 1 . Stratford P, Smeda J. Modified essay question. In: Shannon S, Norman G,
eds. Evaluation Methods: A Resource Handbook. Ontario: McMaster The patient’s wife, not knowing who else to talk to,
University, 1995.
12. Marton F, Saljo R. On qualitative differences in learning-11. Outcome as a
comes back to see you in the Accident and Emergency
function of the learner’s conception of the task. Br J Educ Psychol 1976; 4 6 Department. She is understandably distraught and con-
115-127. cerned about her future, particularly as she is 10 weeks’
13. Newble DI, Jaeger K. The effect of assessments and examinations on the
learning of medical students. MedEduc 1963; 17: 165-171. pregnant with their second child. In the light of this
14. Frederiksen N. The real test bias. Influences of testing on teaching and information, the autopsy findings become of even
learning. Am Prychol 1984; 3 9 193-202. greater importance.
15. Newble D1, Entwistle NJ. Learning styles and approaches: implications for
medical education. Med Educ 1986; 2 0 162-167.
16. van der Vleuten CPM, Newble DI. How can we test clinical reasoning? Q5. Why? (2 marks)
Lancet 1995; 345 1032-1034.
17. Schwarz RW, Donnelly MB, Sloan DA, Young 9. Knowledge gain in a
problem-based surgery clerkship. Arad Med 1994; 6 9 148-151 You suggest that the wife be given urgent counselling
which is over and above that usually offered to the
bereaved.
APPENDIX I. EXAMPLE OF A SCENARIO
QUESTION FROM A RECENT EXAMINATION Q6. Who should she be sent to see, and what kinds of
issues should this specialist discuss? ( 4 marks)
A man of 27 years is admitted to the Accident and
Emergency Department, where you are the casualty
officer. His wife says that he suddenly complained of APPENDIX I1
severe head and neck pain whilst eating lunch. He
became rapidly drowsy and confused and she called
for the ambulance. On admission to hospital he was
afebrile, confused, and moving all four limbs. He had Table AI-Marking schedule for the scenario problem in
no focal neurological signs and no papilloedema. A Appendix I. The first column of numbers is the total available
computed tomography scan was carried out, and was for the particular responses; the second column is a reminder
normal. An emergency lumbar puncture was performed of the total number of marks available for each individual
and the CSF was found to be uniformly bloodstained question
and xanthochromic. Serum electrolytes were normal,
but the blood urea was about three times the upper end Q1. Subarachnoid haemorrhage 1 1
of the normal range. 42. Berry aneurysm 1 1
4 3 . Circle of Willis 1
Ql. What to you think is the most likely diagnosis? Middle cerebral artery 1 2
( I mark) Q4. Polycystic kidneys-adult type 1
Polycystic kidneys-type not specified + 1
Q5. Inherited lesion 1
Following the lumbar puncture, his condition is stable Autosomal dominant 1 2
for 2 h and then rapidly deteriorates. Despite all Q6. Genetic counsellor 1
measures he dies 6 h after admission. Risk to current infant 1
At autopsy, the kidneys are large, each weighing Risk to unborn child 1
450 g; on sectioning, each contains numerous fluid-filled Possibility of termination 1 4
cysts up to 1 cm in diameter, separated by a little normal