Topics in Stroke Rehabilitation

ISSN: 1074-9357 (Print) 1945-5119 (Online) Journal homepage: http://www.tandfonline.com/loi/ytsr20

Development and Validation of the Stroke

Knowledge Test

Karen Sullivan & Natalie J. Dunton

To cite this article: Karen Sullivan & Natalie J. Dunton (2004) Development and Validation of
the Stroke Knowledge Test, Topics in Stroke Rehabilitation, 11:3, 19-28

To link to this article: http://dx.doi.org/10.1310/RED5-V47T-8MJN-JY9H

Published online: 30 Nov 2015.

Submit your article to this journal

View related articles

Citing articles: 15 View citing articles

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=ytsr20

Download by: [The University Of Melbourne Libraries] Date: 02 January 2016, At: 14:06
TSR.11;3.QX 8/4/04 9:19 PM Page 19

Grand Rounds

Elliot J. Roth, MD, Editor

Development and Validation of the Stroke

Knowledge Test
Karen Sullivan and Natalie J. Dunton

Purpose: The purpose of this study was to develop a measure of stroke knowledge (the Stroke Knowledge Test [SKT])
using a systematic test construction process and to investigate the psychometric properties of this test. There are rela-
Downloaded by [The University Of Melbourne Libraries] at 14:06 02 January 2016

tively few published measures of stroke knowledge, and, of those that exist, relatively little is documented about test con-
struction or psychometric properties. Such tests are important for evaluation of stroke education programs. Method: Test
construction involved systematic generation of pilot test items, expert review of pilot items, and calculation of pilot item
properties. After final item selection, two experiments were conducted to determine if the SKT was sensitive to varying
levels of stroke knowledge and to estimate the reliability of the test. Results: The final version of the test included 20
items with good content coverage, acceptable item properties, and positive expert review ratings. Results from psycho-
metric investigations suggest that SKT has relatively good reliability (internal consistency and test-retest reliability) and
construct validity (i.e., SKT scores significantly increased after stroke education [cf. nonstroke education], and communi-
ty-dwelling older adults who had a relative with stroke [and more prior exposure to stroke information] scored higher on
the SKT than those without a stroke relative). Conclusion: Findings provide preliminary support for the SKT as a valid
and reliable tool for assessing stroke knowledge. The SKT may be used to identify individual information needs of stroke
survivors and their caregivers or as a tool to evaluate group- or community-based stroke education programs. Key
words: patient education, rehabilitation, stroke education, stroke knowledge

programs, including video.13 The general pattern

S
troke affects more than 40,000 Australians
annually and is the most common cause of
disability in the elderly.1–4 The effects of
stroke are substantial and impact extensively upon
the quality of life of both stroke survivors and
caregivers.5 Further, the prevalence of stroke is
expected to increase with the aging of the popula-
tion.3
Stroke is clearly a major public health issue;
however it is important to note that the risk of
stroke may be reduced by healthy lifestyle change
and the extent of stroke-related disability may be
minimized by early intervention and treatment.
For example, early recognition of stroke symp-
toms is crucial for appropriate diagnosis and treat-
ment.6–8 Additionally, adherence to treatment rec-
ommendations may be improved by addressing
some of the common misconceptions articulated
by noncompliers, such as unwillingness to engage
in gentle exercise because this is perceived as
increasing stroke risk.9
Previous strategies used to improve awareness of
stroke risk reduction factors and stroke knowl-
edge in general have included public education
programs,3,10,11 stroke education tailored to
patients’ and carers’ needs,12 and group-based
information programs.8 In addition, a variety of
media have been used to deliver stroke education
of results from patient- and carer-based education
studies suggests there are a number of benefits of
providing stroke education, such as reduced carer
burden (e.g., ref. 14; for an exception, see ref. 15).
This general trend in patient education evaluation
studies is consistent with findings in other areas
that suggest illness knowledge is an important
precursor to healthy lifestyle change in conditions
such as diabetes16 and cardiovascular disease.17
The precise nature and extent of the beneficial
education effect found in patient and carer educa-
tion studies is somewhat difficult to characterize
given that there are a number of important
methodological issues that complicate the inter-
pretation of results from such studies. First, there
is substantial variation in outcome measures used
to define effective stroke education. That is, stud-
ies examining the efficacy of education interven-
Karen Sullivan, PhD, is Senior Lecturer, School of
Psychology & Counselling, Queensland University of
Technology, Queensland, Australia.
Natalie J. Dunton, BPsych (Hons), was a research student,
Queensland University of Technology, Queensland, Australia.
Top Stroke Rehabil 2004;11(3):19–28
© 2004 Thomas Land Publishers, Inc.
www.thomasland.com
19
TSR.11;3.QX 8/4/04 9:19 PM Page 20
20 TOPICS IN STROKE REHABILITATION/SUMMER 2004
tions typically attribute change in behavior, such
as a decrease in carer stress, to the effects of edu-
cation without directly assessing change in under-
standing of stroke. Outcome measures have
included the extent to which stroke-related educa-
tion influences the stability of family function-
ing,18 adherence to treatment recommendation,19
reported anxiety,20 perceived health status,12 and
level of satisfaction with health services.21 The
general findings from the majority of these studies
Downloaded by [The University Of Melbourne Libraries] at 14:06 02 January 2016
have indicated that the provision of stroke educa-
tion results in improvements on such measures
(e.g., increased satisfaction with health services,21
reduced anxiety,20 more stable family function-
ing,18 and consequently greater adherence with
treatment recommendations19). Essentially then,
stroke knowledge (which can be defined as
patients or carers understanding of stroke) is
implicated as the underlying mechanism associat-
ed with mediating such improvements.
Second, in the few studies in which change in
stroke knowledge has been assessed directly as an
outcome measure (either as the single outcome
measure or in combination with one of the meas-
ures identified earlier [e.g., refs. 15, 20–22]),
results have been complicated by the use of a
range of different, relatively unstandardized out-
come measures, including those with undocu-
mented psychometric properties. Indeed, this has
led to recognition of the absence of (and conse-
quent need for) formal psychometric testing of
stroke knowledge measures that could be used to
accurately evaluate the efficacy of existing stroke
education programs.15,21
The third factor that has complicated the inter-
pretation of stroke education evaluations has been
the failure to assess outcomes pre- and posteduca-
tion15,18 or to include an appropriate control or
comparison group.19,23 In the absence of these
design features, it is difficult to accurately deter-
mine the magnitude of genuine education effects. A
related point is that the method used to deliver
stroke education potentially confounds the process
of education itself, such that it is difficult to attrib-
ute an apparent increase in stroke knowledge to the
specific effects of stroke education interventions.12
A growing body of literature suggests there is
increasing recognition of the need to conduct
stroke education evaluations using optimal out-
come measures. Ideally, stroke education evalua-
tion studies should include multiple outcome
measures across multiple domains, but at least one
measure should assess stroke knowledge. That is,
to evaluate the nature and extent of change in
stroke knowledge, stroke evaluation studies
should include a measure of stroke knowledge
that has been systematically constructed with
good psychometric properties and is appropriate
for administration in a before and after fashion or
with an appropriate comparison group(s).
The overall aim of this study was to develop and
assess the psychometric properties of a new
instrument, the Stroke Knowledge Test (SKT),
which was intended as a tool for assessing level of
stroke knowledge among the general public or
stroke survivors and their carers. To achieve this
aim, two sets of investigations were undertaken:
test construction and preliminary psychometric
investigation. The first set of investigations (test
construction) involved generating test items from
a literature review, conducting an expert review of
test items, and investigating item properties. The
second set of investigations was primarily aimed at
investigating the validity of the test in terms of its
sensitivity to varying levels of stroke knowledge
and characterizing the reliability of the SKT.
Test Construction
Item generation
As recommended by Nunnally and Bernstein,24
more pilot test items (30) were generated than the
number anticipated for the final version of the test
(approximately 20). The content of items was
selected on the basis of literature review and
included issues identified previously as important
for stroke patient education.6,8,25 Items covered
potential risk factors, signs and symptoms of
stroke,4 and issues related to (or perceived to be
related to) the prevention, prevalence, treatment,
and rehabilitation of stroke.10 Items were also writ-
ten in plain language to maximize readability.26
A five-alternative multiple-choice format was
adopted for this test. Items consisted of one correct
option, three distractors, and an I-don’t-know
TSR.11;3.QX 8/4/04 9:19 PM Page 21
option to reduce the likelihood of guessing.27 The
decision to construct the SKT in this way was based
on consideration of test construction guidelines.24,28
Recommendations of when to use multiple-choice
formats include applications such as those that
require broad sampling of content domain within
reasonable time limits (see ref. 24), and these con-
straints applied to the SKT. An example of an item
from the SKT is shown in Figure 1.
Downloaded by [The University Of Melbourne Libraries] at 14:06 02 January 2016
Pilot SKT investigations
Two investigations of the pilot SKT were under-
taken prior to final item selection given that using
a combination of indicators of the quality and per-
ceived utility of items is recommended for optimal
test construction.24,28,29 First, the pilot SKT was
subject to expert review. Second, an investigation
of the item properties of the pilot-SKT (item diffi-
culty and item discrimination) was undertaken.
Each of these steps is detailed below.
Expert review
After a 30-item test with appropriate coverage
was generated, the pilot SKT was subject to expert
review. The purpose of the expert review was to
generate information that might be used to refine
the test on the basis of potential user input. Forty
health professionals working in stroke-related
areas were invited to comment on the pilot SKT.
Each participant was sent an SKT evaluation kit.
If someone has a stroke, when should you ring
for an ambulance?
(a) Only ring if the symptoms stay after 24
hours.
(b) Always ring for an ambulance straight
away.
(c) Just see your doctor when you can.
(d) You don’t need to ring an ambulance.
(e) I don’t know.
Figure 1. Sample item from the Stroke Knowledge
Test. The correct answer is shown in bold. Options
a, c, and d provide an illustration of the type of dis-
tractors developed for SKT items.
Stroke Knowledge Test 21
The kit comprised a cover letter, demographics
information sheet, and the pilot SKT (evaluation
version). The evaluation version of the pilot SKT
was constructed to include questions for reviewers
about aspects of the test on which their comments
were sought. Specifically, expert reviewers were
asked to evaluate each of the 30 items on four
dimensions using a dichotomous response scale
(e.g., “clear” vs. “unclear”). The four dimensions
were (a) consistency of item to content area, (b)
clarity of item wording, (c) perceived item diffi-
culty, and (d) whether (and why) they thought the
item should be included or excluded from a
revised version of the test. In addition, reviewers
were asked to rate the perceived usefulness of an
item using a 5-point Likert scale ranging from not
useful at all to very useful. Finally, expert reviewers
were asked to indicate how likely they were to use
the SKT in their workplace (“likely use” question)
and to rate the overall perceived utility of the test.
Of the 40 kits distributed, 15 were returned,
which yielded a 30% response rate. The final sam-
ple of expert reviewers comprised a broad cross-
section of professionals (females = 79%), includ-
ing neurologists, occupational therapists, a
geriatrician, nurses, volunteers from a stroke asso-
ciation, and clinicians from the National Stroke
Foundation. The majority of respondents (86%)
ranged in age from 31 to 50 years, with an average
of 9 years experience working in a stroke-related
area (M = 9.35, SD = 6).
Data from expert reviewers were analyzed to
determine the percentage of agreement among
health professionals along each dimension on bina-
ry categorical items (i.e., content, clarity, difficulty,
and inclusion) and on general utility and likely use
questions. The general consensus among health
professionals was that the majority of items on the
pilot SKT were consistent with the content area (M
= 92%), clearly worded (M = 80%), and recom-
mended for inclusion (M = 78%) in the final ver-
sion of the test. Overall, the average rating of the
usefulness of the items was quite high (M = 3.7). A
slightly lower rating of perceived utility of the test
in the workplace was reported (M = 3.09), sug-
gesting that on average reviewers were unsure if
they would use the test. Given that reviewers made
these ratings of the pilot version of the test and in
TSR.11;3.QX 8/4/04 9:19 PM Page 22
22 TOPICS IN STROKE REHABILITATION/SUMMER 2004
the absence of results about the psychometric
properties of the SKT, the likely use ratings can be
interpreted as cautious but appropriate.
Estimating item properties
The second investigation of the pilot SKT under-
taken prior to final item selection involved admin-
istration of the test to explore item properties.
Fifty-one undergraduate psychology students from
Downloaded by [The University Of Melbourne Libraries] at 14:06 02 January 2016
Queensland University of Technology (QUT) vol-
unteered to participate in this part of the study.
This number of participants satisfies recommend-
ed minimum standards for pilot study sample size
(30). Participants were mostly female (N = 42) with
an average age 24.3 years (range, 17–43; SD = 8.2).
Participants were asked to complete the pilot SKT
and results were used to generate item difficulty
and item discrimination indices for all pilot SKT
questions. Item difficulty was established by calcu-
lating the proportion of people obtaining the cor-
rect answer on an item. Following recommenda-
tions outlined by Allen and Yen,30 the optimal range
of item difficulty was defined as between 30% and
70%. An item was deemed too difficult if less than
30% of responses were correct and too easy if the
proportion of correct responses exceeded 70%.
Item discrimination provides an assessment of
how well an item differentiates between high scorers
(upper third) and low scorers (lower third) in a sam-
ple. For example, on an item with good discrimina-
tion, a high scorer on the test should perform well,
whereas a low scorer should perform poorly.31 Item
discrimination indices range from -1 to 1, with a
positive index being indicative of this discriminating
quality.31,32 Given the format of the pilot SKT and
allowing for the effects of guessing, the index pro-
viding the optimal discrimination between items
would be approximately .375, and this cut-off was
used when selecting items for the final test.30,31
SKT final item selection
The results of item analyses and expert review
were considered together to inform final item
selection. Of the 19 items found to be outside the
recommended range of item difficulty, 7 were dis-
carded from the revised test because most of these
were also identified for exclusion by at least 40%
of expert reviewers. A further three items were dis-
carded despite adequate item properties, because
the majority of reviewers rated these items as lack-
ing consistency and clarity. Based on results of
item analyses, one item was considered subopti-
mal but was retained because it covered important
content and was not rated poorly by expert
reviewers. The content of this item concerned the
prevalence of stroke; because this may have a
bearing on normalizing the experience of rehabil-
itation poststroke, this item was retained. Eight
items were reworded because less than 80% of
reviewers rated these items as clearly worded.
Overall, 10 pilot SKT items were discarded and 8
were reworded.
Preliminary Investigations of the
Psychometric Properties of the SKT
Having selected the best 20 SKT items, we per-
formed three investigations of the psychometric
properties of the SKT to assess the readability, reli-
ability, and validity of the SKT. The method and
results for these investigations are detailed below.
Readability analysis of the SKT
A readability analysis of SKT (final item set) was
conducted to check the comprehensibility of the
20-item test.26 Results revealed the readability of
the SKT was at a “standard” level, based on formal
criteria.33 This suggests that the test should be able
to be read by people with a marginal literacy lev-
el, equating to approximately 8 or 9 years of for-
mal education.34,35
Estimating the reliability and validity of Stroke
Knowledge Test (SKT)
One means of assessing the validity of tests such
as the SKT is to determine if the test is capable of
discriminating between varying levels of knowl-
edge. If so, the test is said to exhibit construct
validity.36 Therefore, to explore the sensitivity of
the SKT to varying levels of stroke knowledge, two
investigations were conducted. In the first investi-
gation (referred to as the education test), varying
TSR.11;3.QX 8/4/04 9:19 PM Page 23
levels of stroke knowledge were “induced” in a
subset of participants by exposing them to infor-
mation about stroke, while other participants
received irrelevant (i.e., nonstroke) information.
In the second investigation (the relative test),
varying levels of stroke knowledge were “pre-
sumed” on the basis of whether participants had a
relative with stroke. Estimates of reliability (inter-
nal consistency) were calculated as part of both
investigations. The method and results for these
Downloaded by [The University Of Melbourne Libraries] at 14:06 02 January 2016
investigations are detailed below.
Education Test: Method
Participants
Fifty undergraduate psychology students from
QUT (females = 39) volunteered to participate in
this part of the study. They received course credit
for participation. Participants involved in the pilot
testing stage of this project were not allowed to
participate in this stage to prevent confounds. The
age of participants ranged from 17 to 43 (M =
25.02, SD = 7.52) and education ranged from 10
to 12 years (M = 11.8, SD = .57). Participants were
randomly allocated to one of two groups: stroke
information (SI) or nonstroke information (NSI).
To ensure the groups had equivalent characteris-
tics, independent samples t tests were performed.
With alpha set at .05, no significant group differ-
ences in age or education were evident.
Additionally, there were no significant group dif-
ferences in the level of prior exposure to stroke
education materials (i.e., books, pamphlets, or
videos; p > .05). That is, of the group receiving
stroke information, one participant reported hav-
ing read a book about stroke (compared to zero
NSI participants), five SI participants had previ-
ously read pamphlets about stroke (compared to
six NSI participants), and five SI participants had
previously watched videos about stroke (com-
pared to seven NSI participants).
Materials
The final version of SKT (20-item test) was used
in this phase of the study. Incorrect responses on
multiple-choice items were scored zero, and correct
Stroke Knowledge Test 23
responses were awarded one point each. The total
SKT score was calculated by summing across items
(maximum possible score of 20), such that higher
SKT scores indicated greater stroke knowledge.
The participants received information consisting
of a pamphlet about stroke (i.e., “The Top Ten
Questions About Stroke?”37) or banking (nonstroke
information, i.e., “How to Run a Business
Account”38). These brochures were selected because
they contained a similar amount of information
(i.e., approximately the same number of words).
Further, readability statistics were calculated for
both brochures to estimate (and approximately
equate for) the reading level required to compre-
hend information provided to participants. Both
brochures were rated as “hard reading”33 (NSF
brochure: 49.1 [reading ease] and 10.3 [grade lev-
el]; Suncorp-Metway brochure: 41.5 [reading ease]
and 11.8 [grade level]). Finally, to ensure that the
content of the brochure containing stroke informa-
tion was sufficient to answer SKT questions, two
independent evaluators were asked to indicate how
many questions they thought could be answered
after exposure to the brochure. Raters indicated an
average of 85% of SKT items could be answered
after reading the stroke brochure.
Design and procedure
There were two independent variables: type of
information (stroke information or nonstroke
information manipulated between groups), and
education (pre- and posteducation tested within
groups). The dependent variable was the total
number of correct answers on the SKT. In the pre-
education phase, participants completed the SKT.
A week later, participants returned for the posted-
ucation phase; at this time, they were asked to
read the information provided to them (either SI
or NSI) and then complete the SKT again.
Education test: results
To assess the validity and reliability of the 20-
item SKT, four analyses were performed. First, a 2
x 2 mixed analysis of variance (ANOVA) was used
to ascertain whether SI participants would score
higher on the SKT than NSI participants after edu-
TSR.11;3.QX 8/4/04 9:19 PM Page 24
24 TOPICS IN STROKE REHABILITATION/SUMMER 2004
cation. Second, test-retest reliability was calculat-
ed to assess stability of SKT scores across time.
Third, internal consistency of the SKT was calcu-
lated, and, finally, an item-by-item analysis of the
SKT was undertaken to examine additional item
performance parameters.
ANOVA results revealed a significant main effect
of information type, F(1, 48) = 22.84, p < .001, η2 =
.32, with power of .997, and a significant main effect
of education, F(1, 48) = 99.38, p < .001, η2 = .67,
Downloaded by [The University Of Melbourne Libraries] at 14:06 02 January 2016
with power of 1. Consistent with predictions, a sig-
nificant interaction was found between information
type and education, F(1, 48) = 80.098, p < .001, η2
= .63, with power of 1.00. Simple effects of educa-
tion for each level of information type were per-
formed using a Bonferroni adjustment to control for
family-wise error rate (α = .025). These tests
revealed that when NSI was provided to partici-
pants, education had no effect on SKT scores, F(1,
48) = .52, p > .025. However, when SI was given, a
significant effect of education emerged, F(1, 48) =
178.97, p < .025, with posteducation scores being
enhanced (M = 16.64, SD = 1.35) compared to pre-
education scores (M = 9.96, SD = 2.8). Figure 2
shows this interaction. Note that the mean number
of correct responses posteducation for the SI group
is approximately equal to the estimated maximum
Figure 2. Mean SKT scores showing effect of information type before and after education.
SKT score based on rater estimates of the number of
items answerable from reading the stroke brochure
(i.e., 17 out of 20).
To explore test-retest reliability of the SKT and
further examine the construct validity of the SKT,
a Pearson product moment correlation coefficient
was calculated using NSI participants’ mean scores
before and after exposure to education because
these scores did not show a significant education
effect. The high correlation resulting from this
analysis (r = .82, p < .001) suggests that a low lev-
el of individual error variance was introduced to
SKT scores as a consequence of repeat testing. The
demonstration that the SKT has adequate tempo-
ral stability can be interpreted as a further indica-
tion of the construct validity of the SKT.32,39
An estimate of internal consistency of pre-edu-
cation SKT scores was calculated using scores
from all participants, given nonsignificant group
differences in SKT scores pre-education. These
estimates were in the low to modest range
(Cronbach’s α = .65).24
Finally, an item-by-item analysis was conducted
to determine which SKT items respondents
answered correctly after education and to explore
other items’ performance parameters, in particu-
lar, changes in the rate of endorsement of I-don’t-
TSR.11;3.QX 8/4/04 9:19 PM Page 25
know options. A series of paired samples t tests
was conducted to compare correct and incorrect
responses for each SKT item before and after edu-
cation. Of the 6 out of 20 items found to be non-
significant after exposure to education, 5 were still
in the expected direction. This suggests that in
addition to overall improvement in SKT total
scores after education, performance across almost
all test items showed at least a trend in the expect-
ed direction after education. In addition, results
Downloaded by [The University Of Melbourne Libraries] at 14:06 02 January 2016
from an analysis of the pattern of endorsement of
I-don’t-know (option e) responses showed a
decrease in the percentage of such responses after
education (option e responses accounted for 39%
of responses pre-education and only 5.4% posted-
ucation in the stroke education group).
Relative Test: Method
Participants
Forty-one community-dwelling older adults
(females = 23) volunteered to participate in this part
of the study. Initially, the community sample includ-
ed 29 adults from a bowling club, with the remain-
ing 12 adults from the Cleveland Stroke Association
(CSA). Data from three participants were excluded,
however, as more than 90% of responses were miss-
ing from their protocols, which left a final sample of
38. The average age of participants was 67.34 years
(SD = 6.02), with approximately 9 years of formal
education (M = 9.28, SD = 1.98).
Participants were assigned to one of two groups
(stroke relative [SR] or no stroke relative [NSR])
according to whether they reported knowing
someone who had sustained a stroke. A relative
was defined as parent, sibling, grandparent,
aunt/uncle, or brother/sister-in-law, and friend. To
test for group differences between participants
with and without a stroke relative, we calculated
independent samples t tests. Results showed no
significant group differences in age or education (p
> .05). Perhaps unsurprisingly, there were signifi-
cant group differences in the level of previous
exposure to stroke materials (books, pamphlets,
or videos); however, participants in the SR group
reported more prior exposure to stroke informa-
tion (p < .05) than those in the NSR group.
Stroke Knowledge Test 25
Design and procedure
A between-groups design was used to assess
whether the stroke knowledge test was capable of
differentiating between individuals in SR and NSR
conditions on the expectation that these groups
would have varying levels of stroke knowledge.
The independent variable was knowing a relative
with stroke or not, with the dependent variable
being total SKT score. Participants received the
20-item SKT by mail with a reply paid envelope
and were asked to return these within 2 weeks of
receipt.
Relative test: results
A one-way independent groups ANOVA was
conducted to test the prediction that participants
in the SR group would score higher on the SKT
than NSR participants (n = 38). As predicted,
results revealed a significant between-subjects
effect, F(1, 36) = 7.132, p < .05, η2 = .17, with
power of .74, showing higher average SKT per-
formance among SR participants (M = 12.17, SD =
3.10, n = 28) compared to those in the NSR con-
dition (M = 9.3, SD = 2.31, n = 10).
An estimate of internal consistency in this sam-
ple was calculated and was found to be at least
modest, according to standard criteria (Cronbach’s
α = .7).24 This estimate was slightly higher than
the level of reliability estimated from the educa-
tion test sample.
Additional examination of the data revealed a
reasonably low level of I-don’t-know responses
(14.6%) across both groups. Furthermore, results
showed the highest proportion of responses on
any one item was always the correct item
response, suggesting that item distractors on this
test were written appropriately (see ref. 24).
Discussion
The aim of this study was to follow a systematic
test development procedure to generate a measure
of stroke knowledge with a range of potential
applications. To achieve this aim, we performed a
series of steps comprising literature review (to
identify content that items should cover), expert
TSR.11;3.QX 8/4/04 9:19 PM Page 26
26 TOPICS IN STROKE REHABILITATION/SUMMER 2004
review of items (to enable potential user input
during test construction), investigation of item
properties of the SKT in a student sample, and a
readability analysis of test. Overall, the outcome of
the item selection and review process resulted in a
20-item version of the SKT with adequate read-
ability, good coverage of relevant content, and
acceptable item properties.
The need to develop standardized measures of
stroke knowledge with good psychometric prop-
Downloaded by [The University Of Melbourne Libraries] at 14:06 02 January 2016
erties has been articulated previously, and only rel-
atively recently have attempts been made to pub-
lish such measures.11,13,15 The publication of this
test increases the number of stroke education eval-
uation tools that researchers and stroke education
providers might use to evaluate current stroke
education programs and to identify individual
learning needs of stroke survivors and their carers.
In addition, the test construction process that was
undertaken to develop this test is a feature of the
SKT, as is the fact that this test comes with pre-
liminary data on its psychometric properties.
However, there are several limitations of this
study that should be noted. First, the response
rate from expert reviewers (30%) was less than
optimal. The significance of this response rate
needs to be considered in the context of the pur-
pose of the expert review phase of this study. The
primary reason for including reviewers in this
study was to elicit comment on potential test
items. While it could be argued that this response
rate limits generalizability (i.e., our feedback was
based on a small number of reviewers and may
not represent the views of most health profession-
als), this response rate seems acceptable given the
purpose of this stage of the study and the fact that
responses were received from a variety of health
professionals.
A second limitation of the expert review phase
of this study relates to the timing of questions
regarding the perceived utility of the SKT and the
likely use of this test. Note that average ratings
indicated that experts were “unsure” if they would
use this test, even though the overall rating of the
usefulness of items was quite high. Given that
experts made their ratings in the context of a pilot
test review, these ratings may not accurately reflect
experts’ views of the final version of the SKT.
Indeed, higher ratings may be expected given that
the final version of the SKT incorporates changes
based on reviewers’ comments. We recommend
further investigation of health professionals’ per-
ceptions of the utility of the SKT and their likely
use of this test.
A third potential limitation of this study is the
use of university students in the education test.
Given the format of education provided (i.e., writ-
ten information), it could be argued that this
medium is particularly suitable to the learning
style of students, but that older adults may not
benefit as much from information provided this
way. That is, the extent to which knowledge
increased after education in this sample could be
a result of the students possessing a greater degree
of familiarity with written information sources and
“test” procedures, and the same magnitude of
change may not be present in a sample compris-
ing older adults. Future investigations are needed
to determine whether results from the education
test generalized to older samples and alternate
educational formats. Despite this potential limita-
tion, these initial findings suggest that the SKT is
sensitive to changes in stroke knowledge induced
by exposure to written stroke information.
Finally, comment should be made on the level of
internal consistency of the SKT. In both education
and relative tests, this was found to be in the low-
to-modest range, raising the question of whether
(and how) this could be improved. Although this
level of internal consistency is acceptable for a
new test24 and should not be a barrier to its use, a
larger scale study is needed to determine if SKT
items load on a single stroke knowledge factor (or
separate factors [e.g., treatment or risk factors]) as
this will impact the interpretation of internal con-
sistency estimates. In addition, before attempts to
improve internal consistency by item modifica-
tion, we recommend independent verification of
these results given the careful initial steps taken to
select and refine items.
In conclusion, although further independent
investigations of the psychometric properties are
needed to fully characterize this test, the results of
preliminary investigations undertaken to docu-
ment its psychometric properties suggest that the
SKT has adequate internal consistency and test-
TSR.11;3.QX 8/4/04 9:19 PM Page 27
retest reliability and is sensitive to varying levels of
stroke knowledge. Potential applications of this
test include its use in stroke education programs
(including those delivered to groups of carers, sur-
vivors, or professionals) and as a tool to guide
information giving to individuals seeking to
understand more about stroke.
Acknowledgments
Downloaded by [The University Of Melbourne Libraries] at 14:06 02 January 2016
The Queensland University of Technology
Human Research Ethics Committee granted ethi-
cal clearance for this project. Parts of this project
have been presented as posters at conferences held
by the Australian Psychological Society in 2001
REFERENCES
1. Bakas T, Champion V. Development and psycho-
metric testing of the Bakas caregiving outcomes
scale. Nurs Res. 1999;48:250–259.
2. National Health and Medical Research Council.
Prevention of Stroke: A Consumer’s Guide.
Commonwealth Department of Health and Family
Services: Australian Government Publishing
Service; 1996.
3. National Stroke Foundation. Asia Pacific Consensus
Forum on Stroke Management, Melbourne [On-line].
Available at: http:/www.natstroke.asn.au/asia-
pac/melbdec.html.
4. Van Veenendaal H, Grinspun DR, Adriaanse HP.
Educational needs of stroke survivors and their fam-
ily members, as perceived by themselves and by
health professionals. Patient Education Counseling.
1996;28:265–276.
5. Hanger HC, Fogarty B, Wilkinson TJ, Sainsbury R.
Stroke patients’ views on stroke outcomes: death
versus disability. Clin Rehabil. 2000;14:417–424.
6. Kothari R, Sauerbeck L, Jauch E, Broderick J.
Patients’ awareness of stroke signs, symptoms, and
risk factors. Stroke. 1997;28:1871–1877.
7. Pancioli AM, Broderick J, Kothari R, et al. Public per-
ception of stroke warning signs and knowledge of
potential risk factors. J Am Med Assoc. 1998;279:
1288–1292.
8. Weltermann BM, Homann J, Rogaleewski A, Brach
S, Voss S, Ringelstein EB. Stroke knowledge among
stroke support group members. Stroke. 2000;31:
1230–1233.
9. Wells A. Family support systems: their role in apha-
sia. Aphasiology. 1999;13:911–914.
10. Hux K, Rogers T, Mongar K. Common perceptions
about strokes. J Community Health. 2000;25:47–65.
11. Yoon SS, Heller RF, Levi C, Wiggers J, Fitzgerald PE.
Knowledge of stroke risk factors, warning symp-
toms, and treatment among an Australian urban
population. Stroke. 2001;32:1926–1930.
Stroke Knowledge Test 27
and 2002. This project was funded by a grant
awarded to Karen Sullivan by the School of
Psychology and Counselling, Research and
Postgraduate Committee, QUT. Writing up of this
research was funded by a Manuscript Completion
Grant awarded to Karen Sullivan by the School of
Psychology and Counselling Research and
Postgraduate Committee, QUT.
We thank the Australian National Stroke
Foundation for granting permission to use their
brochure in this study and for volunteering to
allow staff to participate as expert reviewers of the
SKT. We also thank participants from the Bowling
Club and the Cleveland Stroke Association who
kindly agreed to take part in this study.
12. Rodgers H, Atkinson C, Bond S, Suddes M, Dobson
R, Curless R. Randomized controlled trial of a com-
prehensive stroke education program for patients
and caregivers. Stroke. 1999;30:2585–2591.
13. Stern EB, Berman ME, Thomas JJ, Klassen AC.
Community education for stroke awareness: an
efficacy study. Stroke. 1999;30:720–723.
14. Le Dorze G, Croteau C, Brassard C, Michallet B.
Research considerations guiding interventions for
families affected by aphasia. Aphasiology. 1999;
13:922–927.
15. Mant J, Carter J, Wade DT, Winner S. The impact of
an information pack on patients with stroke and
their carers: a randomized controlled trial. Clin
Rehabil. 1998;12:465–476.
16. Whittemore R. Strategies to facilitate lifestyle
change associated with diabetes mellitus. J Nurs
Scholarship. 2000;32:225–241.
17. Eriksson S, Kaati G, Bygren LO. Personal resources,
motives and patient education leading to changes
in cardiovascular risk factors. Patient Education
Counseling. 1998;34:159–168.
18. Evans RL, Matlock AL, Bishop DS, Stranahan S,
Pederson C. Family intervention after stroke: Does
counseling or education help? Stroke. 1988;19:
1243–1249.
19. Evans RL, Bishop DS, Matlock AL, Stranahan S,
Smith GG, Halar EM. Family interaction and treat-
ment adherence after stroke. Arch Phys Med
Rehabil. 1987;68:513–517.
20. Dzau RE, Boehme AR. Stroke rehabilitation: a fami-
ly-team education program. Arch Phys Med Rehabil.
1978;59:236–239.
21. Wellwood I, Dennis M, Warlow C. Patients’ and
carers’ satisfaction with acute stroke management.
Age Ageing. 1995;24:519–524
22. Korner-Bitensky N, Tarasuk J, Nelles J, Bouchard JM.
The impact of interventions with families post-
stroke: a review. Top Stroke Rehabil. 1998;5:69–77.
TSR.11;3.QX 8/4/04 9:19 PM Page 28
28 TOPICS IN STROKE REHABILITATION/SUMMER 2004
23. Evans RL, Held S. Evaluation of family stroke edu-
cation. Int J Rehabil Res. 1984;7:47–51.
24. Nunnally JC, Bernstein I. Psychometric Theory, 3rd
ed. New York: McGraw Hill; 1994.
25. Zarit SH, Anthony CR, Boutselis M. Interventions
with care givers of dementia patients: comparison
of two approaches. Psychol Aging. 1987;2:225–232.
26. Le Bas J. Comprehensibility of patient education lit-
erature. Aust N Z J Psychiatry. 1989;23:542–546.
27. Courtenay BC, Weidemann C. The effects of a
“don’t know” response on Palmore’s facts on
aging quizzes. Gerontological Soc Am. 1985;25:
Downloaded by [The University Of Melbourne Libraries] at 14:06 02 January 2016
177–181.
28. Haladyna TM. Developing and Validating Multiple-
Choice Test Items, 2nd ed. Mahwah, NJ: Lawrence
Erlbaum Associates; 1999.
29. Ferketich S. Focus on psychometrics: aspects of
item analysis. Res Nurs Health. 1991;14:165–168.
30. Allen M J, Yen W M. Introduction to Measurement
Theory. Monterey, CA: Brooks/Cole Publishing
Company; 1979.
31. Gregory RJ. Psychological Testing: History, Principles,
and Application, 2nd ed. Boston: Allyn and Bacon;
1996.
32. Friedenberg L. Psychological Testing: Design,
Analysis, and Use. Needham Heights, MA: Allyn &
Bacon; 1995.
33. Alki Software Corporation. Word info how to:
Readability statistics dialog box [On-line; accessed
07.04.2002; created 1996]. Available at:
http://www.wordinfo.com/how_to/dialogs/Mwdia
log00000328.html.
34. Estrada CA, Hryniewicz MM, Higgs VB, Collins C,
Byrd JC. Anticoagulant patient information materi-
al is written at high readability levels. Stroke.
2000;31:2966–2970.
35. Power J. Readability helps the level [On-line]. Available
at: http://www.csun.edu/~vcecn006/read1.htm.
36. Dieckmann L, Zarit SH, Zarit JM, Gatz M. The
Alzheimer’s disease knowledge test. Gerontologist.
1988;28:402–407.
37. National Stroke Foundation. Answers to the top 10
questions about stroke [brochure]. Melbourne,
Australia: National Stroke Foundation; 2000.
38. Suncorp-Metway. Expertise business accounts
[brochure]. Australia: Suncorp-Metway Ltd; 2000.
Brochure No. 229.
39. Whitla DK. Handbook of Measurement and
Assessment in Behavioral Sciences. London: Addison-
Wesley Publishing Company; 1968.