Home About this Website About the Exam News Syllabus SAQs and answers Past paper SAQs

us SAQs and answers Past paper SAQs by specialty Useful links

Sensitivity and specificity  Search Site

A recent meta analysis of studies of the utility of the Mallampati score in the prediction of On this page 
a difficult airway found that it had a sensitivity of 60% and a specificity of 70%. 
Sensitivity and specificity 
(a) Outline what is meant by meta analysis and the factors that ensure a high quality ­ ©2014 Dr Charlene McDonnell 
conclusion from the process (10 marks) 
(b) Explain what is meant by sensitivity and specificity as applied to the interpretation of Clinical audit 
the Mallampati data given above. (6 marks)  ­ ©2013 Dr David Stephenson 
(c) Rank the levels of scientific proof used to grade medical evidence (4 marks) 
Null hypotheses, error and randomisation 
Click for model answer by Dr Charlene McDonnell ­ ©2013 Dr Emma Murray 

Outline what is meant by meta-analysis and the factors that ensure a high quality conclusion from the process. Evidence based medicine 
(10marks) ­ ©2013 Dr James Reid 
Meta-analysis is a statistical technique that combines the findings from several individual studies that address a similar
hypothesis in order to reach a single definitive conclusion using all the available evidence. Search strategies, study designs and bias 
The larger size of the meta-analysis gives it more power to detect small but clinically significant effects that could be ­ ©2013 Dr Christopher Wasson 
missed with the smaller individual study.
Central tendencies and statistical
Factors that ensure a high quality conclusion include: definitions 
- Systematic Review: A thorough systematic review with a comprehensive search strategy to ensure no suitable studies are ­ ©2013 Dr Patrick McAteer 
missed. A checklist such as QUOROM is then used to assess the quality of the review.
- Quality Assessment: Meta-analysis is only as good as the quality of the underlying studies. The quality of each study
should be assessed using a scale such as the one developed by Chalmers et al. Low quality studies can be excluded and
sensitivity analysis performed to assess the impact of this exclusion.
- Identifying Publication Bias: A funnel plot is created and examined. An asymmetric plot suggests that studies have been
missed, usually smaller studies with negative findings.
- Precision: The larger the number of subjects in the study the more power there is to detect an effect if it exists.
- Transparency: The reader should be able to recreate the meta-analysis if they desire, so all decision making should be
clear, again this reduces bias
- Heterogenicity: To determine if the studies used are actually combinable, heterogenicity should be assessed using the I2
statistical test
- Appropriate Method of Analysis: If there is no heterogenicity the appropriate method is the fixed-effects model and if
there is the random-effects model should be used.
- If all these factors are addressed the meta-analysis should provide the highest quality conclusions possible.

Explain what is meant by sensitivity and specificity as applied to the interpretation of the mallampati data given
above. (6marks)

+ means a difficult airway

– means not a difficult airway

Sensitivity is the ability of a test to correctly identify a positive outcome where one exists. This equates to A/ (A+C) in the
table above.
With respect to mallampati it is the ability to correctly identify a difficult airway when it is in fact difficult.
60% sensitivity for mallampati means that it will detect 60% of the patients who have a difficult airway (true positives) but
40% of the difficult airways will not be detected (false negative)

Specificity is the ability of a test to correctly identify a negative outcome where one exists. This equates to D/(B+D) in the
table above.
With respect to mallampati it is the ability to identify an airway that is not difficult when it is in fact not difficult.
70% specificity for mallampati means that 70% are correctly detected as not a difficult airway (true negative) and 30% are
incorrectly identified as a difficult airway when they are not difficult (false positive).

Rank the level of scientific proof used to grade medical evidence. (4 marks)
According to the NICE guidelines the level of evidence is as follows:

1a – systematic review or meta-analysis of one or more randomized controlled trial . 1b – at least one randomized
controlled trial
2a – at least one well designed, controlled, non-randomized study
2b – at least one well designed quasi-experimental study e.g. cohort study
3 – well designed non experimental study e.g case-control
4 – Expert opinion.

Level 1a evidence is ranked the highest in its usefulness and level 4 the least useful in medical evidence.

Clinical audit 

You have been newly appointed as the audit lead for your department and have been
asked to write a manual on audits and methodology for junior trainees. 

(a)  
(i) Define audit. What are the key features? (20%)  
(ii) Summarise the cycle (10%) 
(b) Describe some processes that are similar to and related to clinical audit that are
commonly mistaken for audit projects. (20%) 
(c) What are the advantages and disadvantages of the audit process? (20%) 
(d) What facts should be considered when gathering data for an audit project? (30%) 

Click for model answer by Dr David Stephenson

(a) (i) Define audit. What are the key features? (20%)

Definition:

Audit is a “quality improvement process that seeks to improve patient care and outcomes through systematic review of
care against explicit criteria and the implementation of change” (Principles of Best Practice in Clinical Audit, NICE, 2002)

Key features:
Focused goal / question to be addressed
Results compared a recognised standard
Designed to monitor current performance against the standard
Systematic, but not data not randomised

(a)(ii) Summarise the cycle (10%)

Audit Cycle:
Cycle used to have 5 steps (where re-audit was one such step)
Healthcare Quality Improvement Partnership (HQIP) in their Clinical Audit: A Simple Guide for NHS Boards and
Parners, 2010 simplifies this to 4 steps:
Preparation and planning (i.e. set out objectives and methods)
Measuring performance (i.e. data collection and analysis; drawing conclusions)
Implementing change
Monitoring improvement (including re-audit)
The cycle can continue in a loop, with re-audit starting the process over again

(b) Describe some processes that are similar to and related to clinical audit that are commonly mistaken for audit
projects. (20%)

Types of audit:
Local clinical audit
National clinical audit (e.g. NAP4)
Non-clinical / governmental / external audit processed

Non-audit processes, but may have audit-like components:

Peer review / critique of work (not measured against a recognised national standard)
Critical incident monitoring / significant event audit / morbidity and mortality meetings
Research (usually involves randomisation, and attempts to generate new knowledge or standards of practice with or
without hypothesis testing; i.e. does not necessarily measure current practice, it attempts to suggest what current practice
should be)
Patient experience surveys (data collection and evaluation)
Formal investigations (improvement not necessarily the sole aim – primary objectives are different from those of audit)
Registries (data collection tools)

(c) What are the advantages and disadvantages of the audit process? (20%)

Advantages:
Ethical approval not necessary
Should help to improve patient care, or at least identify opportunities for improvement
Usually multidisciplinary in extent

Disadvantages:
May be time consuming
May incur other costs, and if organisational changes are to be made they may incur further cost – but this may be offset by
the improved care and system efficiency
Does not necessarily dictate what best practice should be; however, large, well-executed audits may produce data from
which best practice points can be suggested and further studied

(d) What facts should be considered when gathering data for an audit project? (30%)

There should be a clear purpose, with recognised guidelines for comparison

The audit subject needs to be relevant, with the opportunity for quality improvement
The quality improvement strategies should be easy to implement, and realistic
The cost of the audit (time, monetary) should be outweighed by the value of the issues at stake, and the potential
improvements to be made following the audit
The improvements to be made should be sustainable, and lend themselves to re-audit
The data collection should be finite, and set within a clear time-frame
The data collected should be manageable and analysable
The data collected should be relevant to the primary objectives set out in the audit
The process should be carried out in a professional fashion, and should be completed 

Null hypotheses, error and randomisation 

You have been asked to become involved in the clinical trials of a new drug for the
treatment of postoperative nausea and vomiting. 

(a) What is a null hypothesis? List the logical steps in hypothesis testing. (25%) 
(b) What is a Type I error? What is a Type II error? How can they be avoided? (25%) 
(c) What is the purpose of randomisation? What is stratified randomisation? (25%) 
(d) Outline the phases in the clinical studies of new drugs and their introduction to the
market (25%) 

Click for model answer by Dr Emma Murray

(a)
Null hypothesis:
A type of hypothesis used in statistics that proposes that no statistical significance exists in a set of given observations.
Attempts to show no variation exists between variables of that a single variable is no different than zero.

- Form a null hypothesis

- Choose the appropriate statistical test e.g. if data in a normal distribution, a t-test can be used
- Obtain the p-value
- Accept or reject the null hypothesis

(b)
What is a Type I error?
Type 1 error is the incorrect rejection of a true null hypothesis (false positive)

What is a Type II error?

Type II error is the failure to reject a false null hypothesis (false negative)

How can they be avoided?

By setting the type I error as 0.5 or 0.1 (therefore 5 or 1 in 100 chance that variation due to chance), which is generally
reported as the p-value
For any given set of data, type I and II errors are inversely related; the smaller the risk of one, the higher the risk of the
other
By large sampling groups

(c)
What is the purpose of randomisation?
To distribute confounding factors equally across all treatment groups, also ensuring that each patient has a known chance
of receiving each treatment but that the treatment they receive cannot be predicted in advance.
Prevents a clinician from entering a patient into the early intervention group purely because he believes that their patient is
likely to respond well to a particular treatment
Ensure two groups are comparable and that the only difference between them is the intervention of interest

What is stratified randomisation?

The process of dividing members of the population into homogenous subgroups before sampling
This is to reduce the risk that the groups are unbalanced at the end of the study
Used if there are important baseline characteristics known to affect outcome of the treatment
Recruitment of sufficient numbers to all categories may limit studies

(d)
Outline the phases in the clinical studies of new drugs and their introduction to the market?

Phase 0: Pharmacodynamics and pharmacokinetics

First in-human trials
Single subtherapeutic doses of the study drug given to a small number of subjects to gather preliminary data on the agents
pharmacodynamics and pharmacokinetics
Phase 1: Screening for safety
Researchers test and experimental drug of treatment in a small group of people for the first time to evaluate its safety, and
determine the safe dosage range and identify side effects
Phase 2: Establishing the efficacy of the drug:
Usually against a placebo
Given to a larger group of people to see if it is effective
Further evaluate its safety
Phase 3: Final confirmation of safety and efficacy
Treatment given to a large group of people to confirm its effectiveness, monitor side effect, compare it to commonly used
treatments and collect information that will allow it to be used safely
Phase 4: Sentry studies during sales
Post marketing studies delineate additional information, including the treatment risks, benefits and optimal use.

Evidence based medicine 

The intensive care unit in your hospital is thinking of introducing a new standard sedation
regime. You have been asked to do some preliminary reading to establish best practice. 

(a) What is evidence based medicine? Describe its relevance at a national level, a trust
level and to the individual clinician. (25%) 
(b) What is critical appraisal? (25%) 
(c) What is the difference between a systematic review and meta­analysis? (25%) 
(d) How may levels of medical evidence be graded or arranged hierarchically? (25%) 
(e) What criticisms exist of evidence based medicine? (25%) 

Click for model answer by Dr James Reid

(a) What is evidence based medicine? Describe its relevance at a national level, a trust level and to the individual
clinician. (25%)
- Evidence-based medicine (EBM) promotes concept that one should apply evidence from medical research, particularly
RCTs to the treatment of patients. Integrates best research evidence with clinical expertise and patient values. Involves
systematically reviewing, appraising and using clinical research findings to aid the delivery of optimum clinical care to
patients.
- One of the pillars of clinical governance
- Nationally: Bodies such as the National
Institute for Health and Clinical Excellence
(NICE) use principles of EBM,
coupled with health economic analysis, in
directly commissioned health technology
assessments (HTAs) that inform guidance
as to which treatments should be available
within the NHS

- Trust level:Hospital trusts formulate and implement

formularies, care pathways and guidelines
based on assessments issued by EBM-focused organisations.

- Individually: An understanding of the

evidence base allows the clinician to tailor
treatment to the circumstances and
risk–benefit profile of the individual patient.

(b) What is critical appraisal? (25%)

- Process of systematically examining research to judge its trustworthiness, and its value and relevance in a particular
context
- Includes looking at exclusion/inclusion criteria, reproducibility, sources of bias, statistical methods. 2 crucial things are
power calculation and whether results analysed on intention to treat basis.

(c) What is the difference between a systematic review and meta-analysis? (25%)

- Systematic review means identifying, appraising and evaluating primary research studies using strict criteria to draw
conclusions about a specific issue.
- Meta-analysis is the statistical discipline of assimilating (pooling) data from multiple similar studies to measure an
overall effect and arrive at a statistical conclusion using all available evidence. Meta-analysis is therefore a type of
systematic review. Studies must be sufficiently similar that the pooled data arise from reasonably homogenous groups.
Meta-analyses are useful when large enough study hasn’t been undertaken or isn’t feasible

(d) How may levels of medical evidence be graded or arranged hierarchically? (25%)

- SIGN criteria or more recently GRADE methodology: essentially hierarchy as follows:
Systematic reviews and meta-analyses of RCTs
RCT
Observational studies - prospective e.g. cohort study
Observational study – retrospective e.g. case control study
Non-experimental studies e.g. case series
Expert opinion

(e) What criticisms exist of evidence based medicine? (25%)

- RCTs do not guarantee absence of bias (e.g. publication bias means negative results not published, sponsored trials by
pharmaceutical industry)
- Impossible to fully practice evidence based medicine (not possible to search literature and appraise evidence for
everything we do! Guidelines don’t exist for every situation)
- “Cookbook” medicine – suppresses clinical freedom – pts are individual
- RCTs not required for some situations e.g. in pt with tension pneumothorax clinical experience tells us to decompress –
trial not required

Search strategies, study designs, and bias 

The intensive care unit in your hospital is thinking of introducing a new standard sedation
regime. You have been asked to do some preliminary reading to establish best practice. 

You have been asked to design and conduct a study to investigate the wearing of PPE by
healthcare workers in intensive care and the prevention of hospital acquired infections. 

(a) What is meant by the term 'PICO question' with regards to a prospective study? What
is the purpose of formulating one? (30%) 
(b) Outline a search strategy for evidence based results with regard to your question.
(20%) 
(c) Outline the different type of study designs and indicate which one would you select.
(20%) 
(d) What are the common types of bias in clinical trials? What is the overall effect of bias?
(30%) 

Click for model answer by Dr Chris Wasson

(a) What is meant by the term 'PICO question' with regards to a prospective study? What is the purpose of
formulating one? (30%)
Patient/population
Intervention
Comparison
Outcome

- The Pico method is a technique of developing a researchable question.

- Using the Pico format for a prospective study will aide the author in identifying the patient population to be assessed, the
proposed intervention/exposure, the control group and outcomes to be measured.

- They are important to formulate as they will assist in the pre study design literature review, help decide on study design,
aide in developing the null hypothesis and make the study identifiable in future literature searches.

(b) Outline a search strategy for evidence based results with regard to your question. (20%)

- The PICO method often results in several questions being developed based on the problem identified. The best question
is selected to proceed to literature review.

- The question is inserted into PubMed, MedlinePlus, CINAHL or equivalent database of medical literature and the results
are viewed.

- The aim is to discover what is known and unknown already about the topic of interest and the levels of evidence based
medicine in existence.

- From this search we can complete the FINER (feasible, interesting, novel, ethical, and relevant) assessment of the
potential research topic and decide wether there is value/need in proceding with the project.

(c) Outline the different type of study designs and indicate which one would you select. (20%)

- Case study/series – uncontrolled study that describes outcomes in individual patients or groups of patients with similar
conditions.

- Retrospective study – collection and interpretation of data after events have occurred.

- Prospective study – Patient’s are selected in advance and observed based on a pre designed study protocol e.g;

- Observational – selection of 2 or more groups to be followed up for an appropriate period of time. Groups are selected
based on their exposure to different agents/conditions and are monitored for the specified end point e.g.
progression/development of disesase.

- Randomised controlled trial – similar patients are randomly allocated to a control and intervention group, ideally with
both patient and observer blinded to which. Intervention follows based on a strict protocol and outcomes are monitored.
Gold standard design for an intervention.

- For this study a single blind (observer) randomised control trial should be carried out comparing standard PPE/practice to
the proposed protocolised new PPE, adequately powered and designed to test superiority to current practice (rather than
non inferiority) with primary outcome being rate of hospital aquired infection (diagnosed by a validated tool).

(d) What are the common types of bias in clinical trials? What is the overall effect of bias? (30%)

- Selection bias – when patients are selected in a manner than results in systematic differences between the groups e.g.
poor randomisation

- Measurement bias – results from improper/poor measurement of outcomes e.g. due to uncalibrated equipment. Observer
bias is a form of measurement bias.

- Publication bias – scientific journals are often more interested in publishing positive or novel data, resulting in studies
showing negative data being excluded from systematic review/meta analysis. Individual researchers may choose to only
publish positive data.

- Commercial bias – bias introduced by constraints/study design influenced by the funding body e.g. pharmaceutical
company encouraging non inferiority outcome measure instead of superiority.

- Attrition bias – when data of subjects who do not complete the trial is not analysed, diminished by analysing data on an
intention to treat basis

- Participation bias – patients selected for RCTs are a often self selecting motivated people and several studies have shown
improved outcomes in placebo groups.

Any form of bias may alter results and lead to false assumptions/conclusions and may lead to intervention in groups that
may not be beneficial.

Central tendencies and statistical definitions 

A recent meta­analysis of the Mallampati examination found that it had a sensitivity of
about 60% and a specificity of about 70%. The diagnosis of a 'high' Mallampati grade had a
slightly positive correlation with diagnosis of other features of difficult intubation. 

(a) What are the different ways of expressing a central tendency of data? Which one, if
any, would be appropriate to describe Mallampati grade? 
(b) Explain what is meant by meta­analysis. What are the advantages and disadvantages? 
(c) Explain what is meant by the statistical terms: (i) sensitivity and (ii) specificity (iii)
positive predictive value 
(d) Distinguish between the terms correlation and regression. 

Click for model answer by Dr Patrick McAteer

(a)
- Different measures of central tendency attempt to determine the “typical” or “average” value of a dataset. The mode,
median and mean are used for most types of data.
- Mode: A measure of the most frequent value in a dataset, often not a particularly good indicator of central tendency. It is
the only means of measuring central tendency in a dataset containing nominal categorical values.
- Median: The central datum when all of the data are arranged (ranked) in numerical order. Used for ordinal categorical
data and for interval data. When analysing interval data, it is preferred to the mean when the data are not normally
distributed, as it is less sensitive to the influence of outliers.
- Mean: Arithmetic mean most commonly used. Mean = (X1+X2…+Xn)/n. Used to summarize interval data. As it may be
influenced by outlying data points, it is best used as a measure of central tendency when the data is normally
(symmetrically) distributed.
- Mallampati grades are categorical data (i.e. discrete and qualitative). As such, the mode is an appropriate method of
describing the central tendency of these data.

(b)
- Meta-analysis: the statistical discipline of assimilating data from multiple similar studies to measure an overall effect
using all of the available evidence.
- A quantitative systematic review in which the data from multiple smaller studies are pooled in an attempt to arrive at a
valid statistical conclusion based on all the available data.
- Systematic review: the formal process of identification, appraisal and evaluation of primary research studies and other
relevant research using strict criteria to draw conclusions about a specific issue.
- Advantages:
- Less subjective, less prone to bias and error when compared to narrative reviews.
- Able to be more focussed on a specific question.
- Useful when a large enough study has not been done or is not feasible. Allows conclusions to be drawn from the data
from smaller studies.
- Provides an overall estimate of the effect of a treatment expressed as a Relative Risk, Odds Ratio, Number Needed to
Treat or Number Needed to Harm.
- Disadvantages:
- Not as statistically robust as a large RCT.
- Potential for replication bias (same data published in multiple articles) or selection bias (increased likelihood for trials
with positive or statistically significant results to be published).

(c)
(i) Sensitivity: The ability of the test to correctly identify those patients with the disease.
Sensitivity = True positives/(True positives + False negatives)
(ii) Specificity: The ability of the test to correctly identify those patients without the disease.
Specificity = True negatives/(True negatives + False positives)
(iii) Positive Predictive Value: How likely it is that a patient has the disease when the test result is positive.
PPV = True positives/(True positives + False positives)
PPV (and NPV) are dependent on the population being tested and are influenced by the prevalence of the disease, so
selection of the population to be tested may increase (or decrease) the PPV and NPV. This is not the case with sensitivity
and specificity.

(d)
- Correlation refers to the relationship between two sets of paired interval data. Correlation quantifies the strength of the
linear relationship between a pair of variables. The correlation coefficient is calculated, and lies within the range -1
(negative correlation) and +1 (positive correlation), with a value of 0 indicating no correlation at all.
- If a statistically significant correlation exists between two variables, linear regression analysis may be used to calculate
the equation for the straight line relationship. A regression equation allows us to express the relationship between two (or
more) variables algebraically, and indicates the extent to which some variables are associated with others. This is often
represented on a scatterplot by a regression line. Confidence intervals for the slope and position of the line may also be
determined.

Back To Top ↑

Home About this Website About the Exam News Syllabus SAQs and answers Past paper SAQs by specialty Useful links

Page last updated: 10/01/2018 
©2012­17 Claire Shevlin Contact Me