Using Run Charts

Complex Care Management Toolkit Resource

Reasons to plot data over time

(not pre-post!)
• Make performance of the process visible
• Determine if change is an improvement by
comparing data before and after test
– Aggregate measures alone do not lead to predictions
about future performance or insights to explain past
variations
– Displaying data over time allows us to make informed
predictions, and thus manage effectively
• Determine if holding the gain

Supported by the 2
CALIFORNIA HEALTHCARE FOUNDATION

1
 Time series graphs
100
90
80

Wait Time (Minutes)

70

• Line graph
60
50
40
30 Change
20 Implemented

– series of data over time connected by lines 10

0

l
ug
n
pr
n

ov
ar

ay

ct

ec
pt
b

Ju
Ju
Ja

Fe

O
M

Se
A

D
M

N
• Run chart
– line graph with a median
– allows application of rules to detect signals
• Statistical process control chart (SPC)
– series of data over time with a mean center line,
upper and lower control limits
– allows identification of common or
special cause variation

Supported by the 3
CALIFORNIA HEALTHCARE FOUNDATION

Run Charts

• Display ordered sequence of data and provide

running record over time
• Can be used for any data that are sequenced
over time (trending)
• Require no statistics
• Visually illustrate progress toward goal
• Allow us to detect signals of improvement or
degradation in a process over time
Adapted from, NHS Scotland Tutorial Guide on Statistical
Process Control.
Supported by the 4
http://www.indicators.scot.nhs.uk/SPC/SPC.html CALIFORNIA HEALTHCARE FOUNDATION

2
 Run Chart Anatomy
Individual Center line =
measures median

Time-Ordered Data
Supported by the 5
Source: Richard Scoville, PhD
CALIFORNIA HEALTHCARE FOUNDATION

Run Chart Anatomy

Annotations tell the story Title names the
measure
2/5 Project Start
= baseline data
collected
2/6 New data
system, no
data available
6/6 New triage
procedure

Appropriate Scale:
• Data fills most of scale
Supported by the 6
Source: Richard Scoville, PhD
CALIFORNIA HEALTHCARE FOUNDATION

3
 Sample Run Chart
% Patients with Chlamydia Screen Go-live;
process
changes
100% New
team mtg
90% MD/staff 60% Target
% Patients Screened during

80% educ

70%
60%
Week of Visit

50%
40%
30%
20%
10%
0%

May9-13
Feb7-11

Mar7-11
Jan17-21

Jan24-28
Jan31-

Jun13-17
Jan10-14

May2-6

May16-20

May23-27
May30-
Feb14-18

Feb21-25

Mar14-18

Mar21-25

Jun6-10
Feb28-

Mar28-

Apr4-8

Apr18-22
Apr11-15

Apr25-29
Feb4

Mar4

Jun3
Apr1
Week

Supported by the Slide used with permission from Medical Group

CALIFORNIA HEALTHCARE FOUNDATION 7

Understanding Variation
• All data demonstrate variation
• Sources of variation
– People, methods, environment, materials,
measurements
– Methods: measuring, collecting, analyzing,
interpreting
• Two types of variation
– Random / Common cause
– Non-random / Special cause
Adapted from, NHS Scotland Tutorial Guide on Statistical Process Control.
http://www.indicators.scot.nhs.uk/SPC/SPC.html

Supported by the 8
CALIFORNIA HEALTHCARE FOUNDATION

4
 Random Variation
• Aka “unassigned” variation
• Is present in all processes – reflects “business as usual”
• Does not judge whether the process is “good” or “bad”
• Is predictable
• Example: Arrival time to work

Supported by the 9
CALIFORNIA HEALTHCARE FOUNDATION

Non-random Variation
• Aka “assignable” variation
– Is assignable to a specific cause
– Is a special circumstance that is not part of the
process – not “business as usual”
• Helps you determine if your change is an
improvement

Supported by the 10
CALIFORNIA HEALTHCARE FOUNDATION

5
 Rules of Probability
• Run Charts
– Statistically significant evidence that there is a <
5% probability that would occur by chance
– Signals = Runs, trends, shifts
• Control Charts
– Statistically significant evidence that < 1%
probability of occurring by chance
– Common cause
– Special cause
Supported by the 11
CALIFORNIA HEALTHCARE FOUNDATION

Constructing a Run Chart

1. Identify your measure for which want to display data

2. Horizontal x-axis: time units (days, weeks, months)

3. Vertical y-axis: outcome units

– Wait time of appointment: minutes
– Wait time to get appointment: days
– Blood pressure: mmHg
– LDL: mg/DL
Adapted from, NHS Scotland Tutorial Guide on Statistical Process Control.
http://www.indicators.scot.nhs.uk/SPC/SPC.html
Supported by the 12
CALIFORNIA HEALTHCARE FOUNDATION

6
 Constructing a Run Chart (cont)
4. Scale both axes for current and future data points
5. Plot and join data points over time (ideal of 15 pts)
– Plot baseline data
6. Calculate median as the center line
– Median is the data point where half the data points are
above and half are below – aka the 50th percentile
– Sort all of your data points and take the middle value – if
you have an even number of data points, average the
middle two

Adapted from, NHS Scotland Tutorial Guide on Statistical Process Control.

http://www.indicators.scot.nhs.uk/SPC/SPC.html and Richard Scoville, PhD

Supported by the 13
CALIFORNIA HEALTHCARE FOUNDATION

A sidenote about the Median

• Why not the mean? The median is used because:
– It is the point at which half of the data points are
expected to be above and below (this allows us to apply
the rules we will discuss in a bit)
– Not influenced by extreme points in the data set

• Which points do I use to calculate the median?

– Ideally, calculate the median using baseline data and
extend into the future

• When do I recalculate the median?

Supported by the 14
CALIFORNIA HEALTHCARE FOUNDATION

7
 Rules for Detecting Signals of
Non-Random Variation (aka, a
change!)
Four decision rules:
1. Astronomical point – One point wildly
different from all the others
2. Number of Runs
3. Shift
4. Trend

Supported by the 15
CALIFORNIA HEALTHCARE FOUNDATION

Rule 2: Number of Runs

• Run = sequence of one or more consecutive useful
observations on same side of centerline
– Useful observation = data point that does NOT fall on
centerline
– Total # observations – number on centerline = # useful
observations
• Too many or too few runs suggest non-random
variation
– If number of runs < lower limit or > upper limit, indicates
non-random variation

Supported by the 16
CALIFORNIA HEALTHCARE FOUNDATION

8
 Number of Runs:
Lower and Upper Limits
Total # of Lower limit for Upper limit for Total # of Lower limit for Upper limit for
data points on the number of the number of data points on the number of the number of
the run chart runs (< than this runs (> than this the run chart runs (< than this runs (> than this
that do not fall number runs is number runs is that do not fall number runs is number runs is
on the median ‘too few’) ‘too many’) on the median ‘too few’) ‘too many’)
10 3 9 26 9 19
11 3 10 27 10 19
12 3 11 28 10 20
13 4 11 29 10 20
14 4 12 30 11 21
15 5 12 31 11 22
16 5 13 32 11 23
17 5 13 33 12 23
18 6 14 34 12 24
19 6 15 35 12 24
20 6 16 36 13 25
21 7 16 37 13 25
22 7 17 38 14 26
23 7 17 39 14 26
24 8 18 40 15 27
25 8 18 41 15 27

Source: The run chart: a simple analytical tool for learning from variation in healthcare processes, Rocco J
Perla, Lloyd P Provost, Sandy K Murray, BMJ Qual Saf 2011;20:46-51.

Supported by the 17
CALIFORNIA HEALTHCARE FOUNDATION

Rule 3: Shift
There is a shift if:
• 6 or more consecutive points either all above or all
below the median
– Values that fall on the median neither add to nor break a
shift. Skip values that fall on the median and continue
counting.

Source: The Improvement Guide by Langley

Supported by the 18
et al., 2nd edition, page 436
CALIFORNIA HEALTHCARE FOUNDATION

9
 Rule 4: Trends

There is a trend if:

• 5 or more consecutive points all going up or all going
down.
– If the value of 2 or more successive points is the same,
ignore one of the points when counting; like values do not
make or break a trend

Source: The Improvement Guide by Langley

Supported by the 19
et al., 2nd edition, page 436
CALIFORNIA HEALTHCARE FOUNDATION

10