Disability and Rehabilitation

ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/idre20

A comparison of test-retest reliability of four

cognitive screening tools in people with dementia

Ya-Chen Lee, Yi-Te Lin & En-Chi Chiu

To cite this article: Ya-Chen Lee, Yi-Te Lin & En-Chi Chiu (2021): A comparison of test-retest
reliability of four cognitive screening tools in people with dementia, Disability and Rehabilitation,
DOI: 10.1080/09638288.2021.1891466

To link to this article: https://doi.org/10.1080/09638288.2021.1891466

Published online: 09 Mar 2021.

Submit your article to this journal

Article views: 87

View related articles

View Crossmark data

Citing articles: 1 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=idre20
DISABILITY AND REHABILITATION
https://doi.org/10.1080/09638288.2021.1891466

ORIGINAL ARTICLE

A comparison of test-retest reliability of four cognitive screening tools in people

with dementia
Ya-Chen Leea
, Yi-Te Linb
and En-Chi Chiuc
a
Department of Occupational Therapy, College of Medical and Health Science, Asia University, Taichung, Taiwan; bDepartment of Neurology,
Cardinal Tien Hospital, New Taipei City, Taiwan; cDepartment of Long-Term Care, National Taipei University of Nursing and Health Sciences,
Taipei, Taiwan

ABSTRACT ARTICLE HISTORY

Purpose: This study aimed to compare the test-retest reliability and minimal detectable change (MDC) of Received 20 August 2020
the Mini-Mental State Examination (MMSE), the Short Portable Mental Status Questionnaire (SPMSQ), the Revised 11 February 2021
Montreal Cognitive Assessment (MoCA), and the Saint Louis University Status Examination (SLUMS) in a Accepted 13 February 2021
single sample of people with dementia.
KEYWORDS
Methods: Sixty people with dementia were assessed twice two weeks apart, and the test-retest reliability Cognition; screening tools;
was examined using the intraclass correlation coefficient (ICC) for four screening tools. The MDC95 value reliability; minimal
was calculated based on the standard error of measurement to estimate the random measurement error. detectable change; people
Results: The ICC values for screening tools were 0.86–0.90. The MDC95 values (MDC95%) were 5.0 (17.2%), with dementia
2.74 (27%), 4.71(20%), and 6.26 (24%) for the MMSE, SPMSQ, MoCA, and SLUMS, respectively.
Conclusions: Overall, the four screening tools were similar in test-retest reliability which imply that the
MMSE, MoCA, SPMSQ, and SLUMS were reliable in monitoring cognitive function in people with demen-
tia. The results of the direct comparisons of test-retest reliability of the four screening tools provide useful
information for both clinicians and researchers to select an appropriate cognitive screening tool.

ä IMPLICATIONS FOR REHABILITATION

 The MMSE, MoCA, SPMSQ, and SLUMS are equally reliable and thus they could be used to monitor
the cognitive function in people with dementia.
 The MDC values are useful in determining whether a real change has occurred between repeated
assessments for people with dementia.

Introduction
With the rapid growth of the aging population, the number of
people aged over 60-years old at risk of developing dementia is
rapidly increasing. People with dementia could have problems
with memory, language, thinking, and judgement, which affect
their ability to perform activities of daily living and might have
impact on quality of life of themselves and those that care for
them [1]. Although no cures currently exist, early detection is cru-
cial for ensuring the best targeted treatment effect or out-
come [2].
Four commonly-used cognitive screening tools have been
used for the early detection of dementia, including the Mini-
Mental State Examination (MMSE) [3], the Short Portable Mental
Status Questionnaire (SPMSQ) [4], the Montreal Cognitive
Assessment (MoCA) [5], and the Saint Louis University Status
Examination (SLUMS) [2]. Among these four screening tools, the
MMSE has been the most extensively used in clinical and research
settings due to its practicality [6]. The MMSE is easy to administer
and requires no specialized equipment or training [3,7]. However,
the MMSE has been reported to be less sensitive to mild cognitive
impairment [2,8]. The SPMSQ is a brief cognitive screening tool
that only has 10 items and has been reported to have high
specificity but also with lower sensitivity [9]. The MoCA measures
an important component of dementia that is not measured by
the MMSE, namely executive function [10,11]; however, it has its
own weakness, which is the time required for administration [12].
Compared with the other three screening tools that could be fin-
ished within around 3–10 min, the MoCA takes approximately
15 min to complete could be considered rather lengthy. The
SLUMS only takes about seven minutes to administer [2,13]; how-
ever, it has been less researched for psychometric properties
(such as minimal detectable change and responsiveness) than the
other three screening tools [14]. Summarizing the above-men-
tioned aspects, each one offers its own set of pros and cons, and
it is important to determine the appropriate screening tool based
on the patient’s cognitive status.
Some supportive evidences of psychometric properties have
been found for the MMSE, SPMSQ, MoCA, and SLUMS in people
with dementia [8,10,15,16]. However, to our knowledge, no stud-
ies have simultaneously examined all four screening tools to
determine their psychometric properties (particularly the test-
retest reliability and random measure error) in people with
dementia. As all studies employed different methodologies in the
examination of psychometric properties, it is difficult for users to

CONTACT En-Chi Chiu enchichiu@ntunhs.edu.tw No. 83-1, Nei-Chiang Street, Wan-Hwa District, Taipei 10845, Taiwan

Ya-Chen Lee and Yi-Te Lin contributed equally to this work.
ß 2021 Informa UK Limited, trading as Taylor & Francis Group
2 Y.-C. LEE ET AL.
decide which screening tools to select for people with dementia.
Thus, a direct comparison of these screening tools in a single
sample is indeed required, as the results could assist clinicians
and researchers in selecting a proper cognitive screening tool.
As there are many situations where cognitive screening tools
must be administered on more than two occasions [17], the abil-
ity of a screening tool to reflect the extent of agreement [18],
namely, test-retest reliability [19] between repeated assessments
have become extremely important. The clinicians must have confi-
dence that any changes in scores on repeated assessments repre-
sent a true change in the cognitive function of people with
dementia, and they are not due to measurement errors [19].
However, no previous studies have examined whether these four
cognitive screening tools have similar test-retest reliability for
people with dementia. While clinicians and researchers may be
faced with a greater range of choices, there is limited information
regarding which to select. Therefore, the purpose of this study
was to compare the test-retest reliability of the MMSE, SPMSQ,
MoCA, and SLUMS in a single sample of people with dementia.
Methods
Participants
A convenience sample of people with dementia were recruited
from geriatric outpatient clinics from one hospital in northern
Taiwan. All patients attending the clinics between March and
December 2019 were approached by a research assistant to par-
ticipate in the study. The inclusion criteria were: (1) diagnosis of
dementia based on the Diagnostic and Statistical Manual of
Mental Disorder, fifth edition; (2) age 65; and (3) having a stable
condition with a stable dose of medication within the past month.
Participants were excluded if they (1) had history of severe brain
injury; (2) were diagnosed with intellectual disability; (3) different
scores on the Clinical Dementia Rating (CDR) over these two
repeated tests (any change in score of CDR was considered as
unstable cognitive condition).
With a 10% drop out rate, a sample size of at least 55 partici-
pants was estimated for reliability with a power of 0.8 at a signifi-
cance level of 0.05 [20,21]. The study was approved by the
Institutional Review Board of a teaching hospital. Both the partici-
pants and their caregivers were required to sign the informed
consent to confirm their willingness to participate in this study.
Procedure
A trained rater administered all screening tools, including the
MMSE, SPMSQ, MoCA, and SLUMS, to participants twice, 2 weeks
apart. The four screening tools were administered randomly to
avoid possible order effects. The participants were assessed in a
quiet room to avoid distraction that might affect their perform-
ance. Between each test, the participants were allowed to have a
rest in order to minimize fatigue. The CDR was administered in
the test and retest sessions to confirm that the participants had
stable cognitive condition. In addition, the participants’ demo-
graphic data such as age and years of education were collected
from chart review.
Measures
The MMSE was developed to test the cognitive functions of the
elderly [3]. It tests an individual’s orientation, attention, memory,
language, and visual spatial skills. The total scores range from 0
to 30, where higher scores indicate better cognitive functioning
[3]. This study used the education adjusted cut-off scores of the
MMSE. If the participants attained education level is 8th grade or
some high school, a MMSE score of 24 or below is expected,
while those with some college or higher, a MMSE score of 26 or
below may imply a cognitive impairment [22]. Administration
time of the MMSE has been reported to be about 10 min [23].
The SPMSQ has been widely used to screen for cognitive dys-
function [4,9], and consists of 10 items, including testing orienta-
tion to time and place, memory, current event information, and
calculation. The total number of errors is computed and the range
of the total score is 0–10; where 0–2 errors imply normal mental
function; 3–4 errors imply mild cognitive impairment; 5–7 errors
imply moderate cognitive impairment [4]. Administration time of
the SPMSQ has been reported to be about 3 min [24].
The MoCA is a cognitive screening test designed for detecting
cognitive impairment [25]. It consists of 30 items for assessing
multiple cognitive domains, including orientation, memory, visuo-
spatial skills, executive functioning, language, and attention [25].
The total scores range between 0 and 30, and higher scores indi-
cate better cognitive functioning [11]. In this study, the total score
and the addition of one point for examinees with 12 or fewer
years of education was used for analysis. Administration time of
the MoCA has been reported to be about 15 min [26].
The SLUMS was developed as a screening tool for Alzheimer’s
disease or other kinds of dementia [2]. The SLUMS consists of 11
items that assess various aspects of cognition, including orienta-
tion, short-term memory, calculations, the naming of animals, the
clock drawing test, and recognition of geometric figures [13]. The
scores range from 0 to 30; a score of 27 to 30 is considered nor-
mal for a person with a high school education and a score of
0–20 indicates dementia [2]. Administration time of the SLUMS
has been reported to be about 7 min [14].
The CDR was developed to stage dementia severity. The CDR
consists of six domains, including orientation, judgment and prob-
lem solving, community affairs, home and hobbies, and personal
care [27]. A global score can be obtained from the six domains to
quantify the severity of dementia on a 5-point scale: 0 (healthy),
0.5 (questionable dementia), 1 (mild dementia), 2 (moderate
dementia), and 3 severe dementia) [27]. We used the CDR to
examine whether the symptom severity of the participants was
stable during the study period [28].
Data analysis
The intraclass correlation coefficient (ICC2, 1) was calculated on
the basis of a two-way random model with absolute agreement
type. ICC values 0.80 indicate high agreement, 0.60–0.79 indi-
cate moderate agreement, and <0.59 indicates poor agree-
ment [1,29].
In addition, the minimal detectable change (MDC95) was calcu-
lated to estimate the random measurement error of all four
screening tools. The MDC95 is smallest detectable threshold of
change score, and can be used to determine whether a real
change (either improvement or deterioration) has occurred
between two assessments or whether the change is due to the
measurement error [30]. This study calculated the MDC95 value
based on the standard error of measurement (SEM) using the fol-
lowing formula [31]:
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
SEM ¼ SDall testing scores  1  ICC
pffiffiffi
MDC95 ¼ z-scorelevel of confidence  2  SEM
In these formulae, the SDall testing scores denotes the standard
deviation of all scores of the two assessments. The z-score
 RELIABILITY OF FOUR COGNITIVE SCREENING TOOLS 3

represents the confidence interval from a standard normal distri- Results

bution (i.e., 1.96 for 95% confidence level in this study).
This study also calculated the MDC95% by dividing the MDC95 by
the maximum score, and then, multiplying by 100 to manifest a
relative amount of random measurement error. An MDC95% of less
than 30% was considered as an acceptable random measurement
error [1,28].
In addition, this study plotted the Bland-Altman plots with
95% limits of agreement (LOA) to visualize the agreement
between test-retest assessments. In these plots, the difference
of each pair of scores were against the mean scores of the
two assessments [32,33]. The LOA provides insight into the
amount of variation between assessments and was estimated
as the mean difference ±1.96  SD, where SD represents the
standard deviation of differences [32]. The plots were also
used to examine the possibility of heteroscedasticity. If hetero-
geneity is detected, there is a possibility that a systematic
trend (e.g., the higher the score, the larger the difference)
exists [33]. If heteroscedasticity exists, the MDC95 value should
not apply for different levels (i.e., level of impairment) of
patients [30]. Heterogeneity was examined using Pearson’s r
between the mean values and the absolute difference of the
two assessments. The data were considered to display hetero-
scedasticity when r > 0.30 [19]. In addition, the relative limits
of agreement (LOA%) was used to describe the absolute differ-
ence of screening tools measured at pre-test vs post-test as a
percentage of the group mean of the screening tools [34].
Furthermore, two approaches have been used to check
whether the practice effect exists. First, paired t-test (tow-tailed,
a ¼ 0.05) was performed to determine whether a statistically
significant difference existed between two assessments. Second,
the effect size (Cohen’s d) was calculated to estimate the size
of the practice effect. The value of Cohen’s d  0.80 is large,
0.50–0.79 is moderate, and 0.20–0.49 has small effect size [35].
Table 1. Demographic and clinical characteristics of the participants (n ¼ 60).
Characteristic
Gender, n (%)
Male
Female
Age, years, mean (SD)
Education, n (%)
Elementary school and below
Junior high school
Senior high school
College and above
MMSE, mean of two assessments (SD)
SPMSQ, mean of two assessments (SD)
MoCA, mean of two assessments (SD)
SLUMS, mean of two assessments (SD)
SD: standard deviation; MMSE: Mini-Mental State Examination; SPMSQ: Short
Portable Mental Status Questionnaire; MoCA: Montreal Cognitive Assessment;
SLUMS: Saint Louis University Mental Status Examination.
A total of 60 people with dementia participated in this study; the
mean age was 81.5 years, 55% of the participants were female.
Fifty-eight percent of participants’ education attainment level was
elementary school and below. The demographic and clinical char-
acteristics of the participants are shown in Table 1.
The ICC values for the MMSE, SPMSQ, MoCA, and SLUMS were
0.90, 0.89, 0.88, and 0.86, respectively. The MDC95 values
(MDC95%) were 5.0 (17.2%), 2.74 (27.0%), 4.71(20.0%), and 6.26
(24.0%) for the MMSE, SPMSQ, MoCA, and SLUMS, respectively
(Table 2).
The Bland-Altman Plots for the MMSE, SPMSQ, MoCA, and
SLUMS are shown in Figure 1. The LOAs of the four screening
tools ranged from 5.63 to 4.33 for the MMSE, 2.73 to 2.83 for
the SPMSQ, 5.21 to 4.11 for the MoCA, and 6.80 to 6.17 for the
SLUMS. Correlations between the mean and the difference of the
two assessments for these four screening tools were r ¼ 0.05–0.26.
The LOA%s were 13.89%, 22.14%, 21.62%, and 30.20% for the
MMSE, SPMSQ, MoCA, and SLUMS, respectively, indicating that
the MMSE tended to be the most reliable screening tool and the
SLUMS the least reliable one.
In addition, no significant difference (p ¼ 0.052–0.786) was
found in all four screening tools. The practice effects of these
three screening tools were trivial-to-small (d ¼ 0.04–0.26) (Table 2).
Discussion
This study was the first one to simultaneously compare the test-
retest reliability and minimal detectable change of the MMSE,
SPMSQ, MoCA, and SLUMS in a single sample of people with
dementia. The results of direct comparison could provide an
informed choice regarding the screening tools most appropriate
for clinical practice and research.
The ICC represents the extent of agreement between two
assessments [19]. This study found that the ICC values for the
test-retest reliability of the four screening tools were high (ICC >
0.86). Our findings indicate that all four screening tools are
equally reliable when administered to people with dementia. A
high test-retest reliability of these four screening tools imply that
27 (45.0) they were all useful for monitoring change in cognitive function
33 (55.0) of people with dementia over time.
81.5 (7.8) Regarding the estimation of the random measurement error
between repeated assessments, the MDC95 is used to examine
35 (58.3)
the variability of the score in individual response [30]. Our find-
8 (13.3)
10 (16.7) ings suggest that a change of 5.0 points for the total scores of
7 (11.7) the MMSE (highest possible score is 30), 3 points (30) for the
14.0 (5.7) MoCA, 5 points (10) for the SPMSQ, and 7 (30) points for the
4.8 (3.0) SLUMS in a person with dementia, can be interpreted as a real
8.3 (4.9)
8.3 (6.1) change (either improvement or deterioration) with 95% confi-
dence, and such change is not due to a measurement error. The
abovementioned MDC95 values provide a threshold for clinicians
to determine whether a real change has occurred between
repeated assessments for people with dementia.

Table 2. Results of reliability of four cognitive screening assessments.

Measures Mean1 (SD1) Mean2 (SD2) Difference mean (SD) ICC (95% CI) SEM MDC95 (MDC95%) Pearson’s r t-test (p-value) Cohen’s d
MMSE 13.7 (5.8) 14.4 (5.7) 0.65 (2.54) 0.90 (0.83, 0.94) 1.80 5.0 (17.2%) 0.05 1.99 (0.052) 0.26
SPMSQ 4.8 (3.0) 4.78 (3.0) 0.05 (1.42) 0.89 (0.82, 0.93) 0.99 2.74 (27.0%) 0.18 0.27 (0.786) 0.04
MoCA 8.1 (4.6) 8.6 (5.3) 0.55 (2.38) 0.88 (0.81, 0.93) 1.70 4.71 (20.0%) 0.25 1.79 (0.078) 0.23
SLUMS 8.2 (5.8) 8.5 (6.5) 0.32 (3.31) 0.86 (0.77, 0.91) 2.26 6.26 (24.0%) 0.26 0.74 (0.462) 0.10
MMSE: Mini-Mental State Examination; SPMSQ: Short Portable Mental Status Questionnaire; MoCA: Montreal Cognitive Assessment; SLUMS: Saint Louis University
Mental Status Examination; SD: standard deviation; ICC: intra-class correlation coefficient; CI: confidence interval; SEM: standard error of measurement; MDC95: min-
imal detectable change.
4 Y.-C. LEE ET AL.
Figure 1. Bland-Altman plots. (A) Mini-Mental State Examination; (B) Short Portable Mental Status Questionnaire; (C) Montreal Cognitive Assessment; (D) Saint Louis
University Mental Status Examination. The bold line is the mean difference. The two dotted lines are the 95% limits of agreement.
In terms of heteroscedasticity, the Bland-Altman plots show no
obvious systematic trend for all four screening tools. In addition,
the Pearson’s r between the absolute difference scores and mean
scores of three screening tools were all less than 0.30, implying
that no heteroscedasticity exists. Our findings suggest that both
clinicians and researchers could use a fixed MDC95 value as the
threshold to judge whether a difference between repeated meas-
urement is real change [30]. These results further support the reli-
ability of the MMSE, SPMSQ, MoCA, and SLUMS in monitoring the
changes of the cognitive function of people with dementia.
However, although the values of the MDC95% of the four
screening tools were <30%, it indicates an acceptable random
measurement error. The MDC95% is independent of the unit
measurements and can be used to compare the quantities of ran-
dom measurement errors between screening tools [30].
Compared to the other three screening tools, the SPMSQ had the
highest MDC95% value (27.0%), which implies that a high random
measurement error exists. Although the SPMSQ had the least
numbers of items among the four screening tools, the high ran-
dom measurement error might have hindered its ability to detect
any real changes in the cognitive functions of people in clin-
ical settings.
Interestingly, compared to the other three screening tools, the
SPMSQ had the lowest practice effect (d ¼ 0.04). The most likely
explanation for this discrepancy might be due to the participants
in our study rarely remembered their own phone number (about
62.0% of participants), as most of them use a communication
app, such as LINE, to make free voice calls. Thus, people have no
need to exchange phone numbers, as they just scan a QR code to
obtain contact information. It was observed that our participants
were less motivated to respond to the question of: “What is your
phone number?” in SPMSQ, which might have reduced the prac-
tice-related phenomenon of the SPMSQ in repetitive testing. In
contrast, regarding both the MMSE and MoCA, the extent of prac-
tice effects observed here (Cohen’s d ¼ 0.26 and 0.23, respectively)
were higher than SPMSQ and SLUMS. It was reported that the
high practice effect might have obscured any true changes in
cognitive function [36,37]. In order to reduce the practice effect
of the MMSE and MoCA, it has been suggested that at least
two assessments should be conducted to stabilize perform-
ance [36,38].
To be clinically useful, a screening tool must be scientifically
sound in test-retest reliability to demonstrate the specific treat-
ment effects, and ensure such effects are not due to measure-
ment inconsistency. Overall, the four screening tools were similar
in test-retest reliability. Particularly, the results showed that the
MMSE had high test-retest reliability with acceptable random
measurement error and small practice effect. Although the MMSE
has its own weakness, which is lower sensitivity in detecting peo-
ple with mild cognitive impairment [2,8], the new version of
MMSE, the Mini-Mental Examination, 2nd edition (MMSE-2) has
been developed to resolve these issues. The MMSE-2, as sug-
gested by Folstein et al. [39], has the ability to measure cognitive
function in more detail, and thus, it can measure a greater variety
of cognitive functions than the MMSE [8]. Future research examin-
ing the test-retest reliability of the MMSE-2, SPMSQ, MoCA, and
SLUMS simultaneously in people with dementia might be needed.
Two limitations need to be addressed in this study. First, par-
ticipants were recruited only from one hospital and the limited
education attainment level (58.3% of participants had elementary
school and below) variability may limit the generalizability of our
results. Second, participants were recruited using convenience

sampling and thus, a potential selection bias might have
occurred. In addition, no information was available for participants
who did not consent to participate. Therefore, we cannot com-
pare the differences between participants who participated in the
study and those who did not.
Conclusions
Overall, the four screening tools were similar in test-retest reliabil-
ity which imply that the MMSE, MoCA, SPMSQ, and SLUMS were
reliable in monitoring cognitive function in people with dementia.
The results of the direct comparisons of the test-retest reliability
of the four screening tools can provide useful information for
both clinicians and researchers to select an appropriate cognitive
screening tool.
Acknowledgments
We would like to thank the participants for their work during
data collection.
Disclosure statement
The authors declare no conflict of interest.
Funding
This study was supported by Cardinal Tien Hospital [Grant no.
CTH108A-2N02].
ORCID
Ya-Chen Lee http://orcid.org/0000-0003-0917-3859
En-Chi Chiu http://orcid.org/0000-0002-6052-0871
References
[1] Chiu EC, Yip PK, Woo P, et al. Test-retest reliability and
minimal detectable change of the Cognitive Abilities
Screening Instrument in patients with dementia. PLoS One.
2019;14(5):e0216450.
[2] Howland M, Tatsuoka C, Smyth KA, et al. Detecting change
over time: a comparison of the SLUMS examination and
the MMSE in older adults at risk for cognitive decline. CNS
Neurosci Ther. 2016;22(5):413–419.
[3] Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A
practical method for grading the cognitive state of patients
for the clinician. J Psychiatr Res. 1975;12(3):189–198.
[4] Pfeiffer EA. A short portable mental status questionnaire
for the assessment of organic brain deficit in elderly
patients. J Am Geriatr Soc. 1975;23(10):433–441.
[5] Rademeyer M, Joubert P. A comparison between the Mini-
Mental State Examination and the Montreal Cognitive
Assessment Test in schizophrenia. S Afr J Psychiatr. 2016;
22(1):890–890.
[6] Werner P, Heinik J, Mendel A, et al. Examining the reliabil-
ity and validity of the Hebrew version of the Mini Mental
State Examination. Aging. 1999;11(5):329–334.
[7] Cao L, Hai S, Lin X, et al. Comparison of the Saint Louis
University Mental Status Examination, the Mini-Mental
State Examination, and the Montreal Cognitive Assessment
RELIABILITY OF FOUR COGNITIVE SCREENING TOOLS 5
in detection of cognitive impairment in Chinese elderly
from the geriatric department. J Am Med Dir Assoc. 2012;
13(7):626–629.
[8] Baek MJ, Kim K, Park YH, et al. The validity and reliability of
the Mini-Mental State Examination-2 for detecting mild
cognitive impairment and Alzheimer’s Disease in a Korean
population. PLoS One. 2016;11(9):e0163792.
[9] Malhotra C, Chan A, Matchar D, et al. Diagnostic perform-
ance of short portable mental status questionnaire for
screening dementia among patients attending cognitive
assessment clinics in Singapore. Ann Acd Med Singapore.
2013; 42:315–319.
[10] Bruijnen CJWH, Dijkstra BAG, Walvoor SJW, et al.
Psychometric properties of the Montreal Cognitive
Assessment (MoCA) in healthy participants aged 18–70. Int
J Psychiatry Clin Pract. 2020;4:1–8.
[11] Wong A, Yiu S, Nasreddine Z, et al. Validity and reliability
of two alternate versions of the Montreal Cognitive
Assessment (Hong Kong version) for screening of mild neu-
rocognitive disorder. PLoS One. 2018;13(5):e0196344.
[12] Chiti G, Pantoni L. Use of Montreal Cognitive Assessment
in patients with stroke. Stroke. 2014;45(10):3135–3140.
[13] Szczesniak D, Rymaszewska J. The usfulness of the SLUMS
test for diagnosis of mild cognitive impairment and
dementia. Psychiatr Pol. 2016;50(2):457–472.
[14] Kaya D, Isik A, Usarel C, et al. The Saint Louis University
Mental Status Examination is better than the Mini-Mental
State Examination to determine the cognitive impairment
in Turkish elderly people. J Am Med Dir Asso. 2016;17:
370.ee11–370.ee15.
[15] Roccaforte WH, Burke WJ, Bayer BL, et al. Reliability and
validity of the Short Portable Mental Status Questionnaire
administered by telephone. J Geriatr Psychiatry Neurol.
1994;7(1):33–38.
[16] Zhang S, Wu YH, Zhang Y, et al. Preliminary study of the
validity and reliability of the Chinese version of the Saint
Louis University Mental Status Examination (SLUMS) in
detecting cognitive impairment in patients with traumatic
brain injury. Appl Neuropsychol Adult. 2019;24:1–8.
[17] Falleti MG, Maruff P, Collie A, et al. Practice effects associ-
ated with the repeated assessment of cognitive function
using the CogState battery at 10-minute, one week and
one month test-retest intervals. J Clin Exp Neuropsychol.
2006;28(7):1095–1112.
[18] Hobart JC, Lamping DL, Thompson AJ. Evaluating neuro-
logical outcome measures: the bare essentials. J Neurol
Neurosurg Psychiatry. 1996;60(2):127–130.
[19] Atkinson G, Nevill AM. Statistical methods for assessing
measurement error (reliability) in variables relevant to
sports medicine. Sports Med. 1998;26(4):217–238.
[20] Walter SD, Eliasziw M, Donner A. Sample size and optimal
designs for reliability. Statist Med. 1998;17(1):101–110.
[21] Bonett DG. Sample size requirements for estimating intra-
class correlations with desired precision. Stat Med. 2002;
21(9):1331–1335.
[22] Tombaugh TN, McIntyre NJ. The mini-mental state examin-
ation: a comprehensive review. J Am Geriatr Soc. 1992;
40(9):922–935.
[23] Steenland NK, Auman CM, Patel PM, et al. Development of
a rapid screening instrument for mild cognitive impairment
and undiagnosed dementia. J Alzheimers Dis. 2008;15(3):
419–427.
6 Y.-C. LEE ET AL.
[24] Villareio A, Puertas-Martin V. Usefulness of short tests in
dementia screening. Neurologia. 2011;26:425–433.
[25] Nasreddine ZS, Phillips NA, Bedirian V, et al. The Montreal
Cognitive Assessment, MoCA: a brief screening tool for
mild cognitive impairment. J Am Geriatr Soc. 2005;53(4):
695–699.
[26] Roalf DR, Moore TM, Wolk DA, et al. Defining and validing
a short form Montreal Cognitive Assessment (s-MoCA) for
use in neurodegenerative disease. J Neurol Neurosurg
Psychiatry. 2016;87(12):1303–1310.
[27] O’Bryant SE, Waring SC, Cullum CM, et al. Staging dementia
using Clinical Dementia Rating Scale Sum of Boxes scores:
a Texas Alzheimer’s research consortim study. Arch Neurol.
2008;65(8):1091–1095.
[28] Hughes CP, Berg L, Danziger WL, et al. A new clinical scale
for the staging of dementia. Br J Psychiatry. 1982;140:
566–572.
[29] Landis JR, Koch GG. The measurement of observer agree-
ment for categorical data. Biometrics. 1977;33(1):159–174.
[30] Huang SL, Hsieh CL, Wu RM, et al. Minimal detectable
change of the timed “up & go” test and the dynamic gait
index in people with Parkinson disease. Phys Ther. 2011;
91(1):114–121.
[31] Haley SM, Fragala-Pinkham MA. Interpreting change scores
of tests and measures used in physical therapy. Phys Ther.
2006;86(5):735–743.
[32] Lu WS, Wang CH, Lin JH, et al. The minimal detectable
change of the simplified stroke rehabilitation assessment
of movement measure. J Rehabil Med. 2008;40(8):
615–619.
[33] Bland JM, Altman DG. Statistical methods for assessing
agreement between two methods of clinical measurement.
Lancet. 1986;1(8476):307–310.
[34] Almarwani M, Perera S, VanSwearingen JM, et al. The test-
retest reliability and minimal detectable change of spatial
and temporal gait variability during usual over-ground
walking for younger and older adults. Gait Posture. 2016;
44:94–99.
[35] Cohen J. Statistical power analysis for the behavioral sciences.
2nd ed. Hillsdale (NJ): Lawrence Erlbaum Associates; 1988.
[36] Goldberg TE, Harvey PD, Wesnes KA, et al. Practice effects
due to serial cognitive assessment: Implications for preclin-
ical Alzheimer’s disease randomized controlled trials.
Alzheimers Dement. 2015;1(1):103–111.
[37] Collie A, Maruff P, Darby DG, et al. The effects of practice
on the cognitive test performance of neurologically normal
individuals assessed at brief test-retest intervals. J Int
Neuropsychol Soc. 2003;9(3):419–428.
[38] Cooley SA, Heaps JM, Bolzenius JD, et al. Longitudinal
change in performance on the Montreal Cognitive
Assessment in older adults. Clin Neuropsychol. 2015;29(6):
824–835.
[39] Folstein MF, Folstin SE, White T, et al. Mini-mental state
examination. 2nd ed. Lutz (FL): Psychological Assessment
Resources; 2010.