Journal of the International Neuropsychological Society (2021), 1–11

Copyright © INS. Published by Cambridge University Press, 2021.

doi:10.1017/S1355617721000679

Comparison of Neuropsychological Assessment by Videoconference

and Face to Face

Richard Gnassounou1, Bénédicte Defontaines1, Séverine Denolle1, Stéphanie Brun1, Raphaël Germain2, Déborah Schwartz2,
Stéphane Schück2, Agnès Michon3, Catherine Belin4 and Didier Maillet4,5,*
1
Association Réseau Mémoire Aloïs, 75 rue de Lourmel, Paris, France
2
Kappa Santé, 4 rue de Cléry, Paris, France
3
Département des Maladies du Système Nerveux, Institut de la mémoire et de la maladie d’Alzheimer (IM2A), Hôpital de la Pitié-Salpêtrière, AP-HP, Paris,
France
4
Service de Neurologie, Hôpital Saint Louis, AP-HP, 1 avenue Claude Vellefaux, Paris, France
5
Université de. Lille, ULR 4072 – PSITEC – Psychologie: Interactions, Temps, Emotions, Cognition, Lille, France

(RECEIVED May 5, 2020; FINAL REVISION March 25, 2021; ACCEPTED April 17, 2021)

Abstract
Objective: To compare the administration of neuropsychological tests by teleneuropsychology (TeleNP) and face to
face (F-F) in order to determine the feasibility and reliability of TeleNP. Method: At the inclusion visit, all participants
underwent a traditional F-F neuropsychological assessment as part of their standard care. Four months after inclusion,
they were randomized to undergo an additional neuropsychological assessment either by F-F administration or by
TeleNP. Results: A total of 150 adults with cognitive complaints, but with no major cognitive or sensorial impairment
were included. At 4 months, 69 participants were randomized in the F-F arm and 71 in TeleNP arm (10 lost in the
follow-up). The overall satisfaction was high: 87.1% in the TeleNP arm were “very satisfied”, and 82.9% indicated no
preference between F-F and TeleNP. In agreement with previous data from the literature, neuropsychological
assessments gave similar results across both administration conditions for a large majority of tests [Mini-Mental State
Examination (MMSE), Free and Cued Selective Reminding Test (FCSRT) French version, Mahieux gestural praxis
battery, Frontal Assessment Battery (FAB), time of completion of the Trail making Test (TMT) A and B, number of
errors of the TMT B, Rey complex figure test, categorical et phonological verbal fluency tests] and minor differences
for others [80-picture naming test (DO-80), FAB, Digit Span forward and backward and number of errors in the TMT
A]. Conclusions: TeleNP is a promising method to be able to test patients as an alternative to F-F condition. Before this
procedure can be generalized, it is now necessary to standardize the adaptation of certain tests and to test them in
populations with more significant cognitive disorders.
Keywords: Cognitive assessment, Dementia, Alzheimer’s disease, Neuropsychological tests, Teleneuropsychology,
Videoconference

INTRODUCTION number of people with Alzheimer’s disease is expected to

surpass 1.3 million in 2020.
In France, as in many countries, the population is aging. The
Facilitating access to the diagnosis of dementia is, there-
French National Institute of Statistics and Economic Studies
fore, an obvious need. However, the capacity for making
(INSEE) estimates that there are more than 6 million people
these diagnoses is limited in medical deserts with insuffi-
over the age of 75, or 9% of the French population (INSEE,
cient healthcare coverage where, for lack of an expert opin-
2014). As the number of older adults increases, so does the
ion, patients are often underdiagnosed (Chodosh et al.,
incidence of pathologies that cause major neurocognitive
2004). With the rapid advances in telemedicine in recent
impairment interfering with autonomy and requiring the
years, new digital and communication technologies can
intervention of a caregiver. And as a consequence, the
offer a solution to the growing exodus of doctors from cer-
tain regions in France, facilitating access to neuropsycho-
logical assessments and providing clinicians with new
*Correspondence and reprint requests to: Didier Maillet, Service de
Neurologie, Hôpital Saint Louis, 1 avenue Claude Vellefaux, Paris F-75010,
tools in line with evolving practices (Jollivet, Fortier,
France. Email: didier.maillet@aphp.fr Besnard, Le Gall, & Allain, 2018).
1
Downloaded from https://www.cambridge.org/core. University of Glasgow Library, on 16 Aug 2021 at 17:10:32, subject to the Cambridge Core terms of use, available at
https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355617721000679
 2 R. Gnassounou et al.
Teleneuropsychology (TeleNP) (neuropsychological
assessment by videoconference) now allows clinicians to
administer neuropsychological tests remotely (Grosch,
Weiner, Hynan, Shore, & Cullum, 2015) and thereby help
with diagnosis (Harrell, Wilkins, Connor, &Chodosh,
2014). The feasibility and reliability of this new method
of administering neuropsychological tests have been dem-
onstrated in adults with and without cognitive impairment
in an urban setting (Castanho et al., 2016; Cullum, Hynan,
Grosch, Parikh, & Weiner, 2014; Cullum, Weiner,
Gehrmann, & Hynan, 2006; Galusha-Glasscock, Horton,
Weiner, & Cullum, 2016; Hildebrand, Chow, Williams,
Nelson, & Wass, 2004; Jacobsen, Sprenger, Andersson, &
Krogstad, 2003; Vestal, Smith-Olinde, Hicks, Hutton, &
Hart, 2006) as well as in a rural population (Adjorlolo,
2015; Wadsworth et al., 2016). The videoconference-based
neuropsychological assessment procedure was also shown
to distinguish between cognitively impaired and non-
impaired subjects similar to a face-to-face (F-F) assessment
(Wadsworth et al., 2018). This new assessment method seems
to be well accepted by a large part of healthy older adults and
patients with cognitive impairment, but a higher percentage
of subjects with mild cognitive impairment or early stage
Alzheimer’s disease indicated that they preferred to be tested
by traditional F-F administration (Parikh et al., 2013).
In France, no studies evaluating the reliability of neuro-
psychological assessment by TeleNP versus a standard F-F
assessment have been conducted to date. Our primary objec-
tive was to evaluate the feasibility and the reliability of
a videoconference-based neuropsychological assessment
procedure performed in accordance with the guidelines
of the Joint Task Force for the Development of
Telepsychology Guidelines for Psychologists (2013). To
this end, we compared the neuropsychological scores
obtained with a battery of tests in two conditions: admin-
istered by F-F and by TeleNP, in a large sample of urban
adults with cognitive complaints. This study includes a
battery of tests widely used in France during neuropsycho-
logical assessment for adults. We hypothesized that there
would be no difference between the neuropsychological
test scores obtained by TeleNP and those by traditional
F-F administration.
We will also assess the level of satisfaction of participants
with the TeleNP assessment procedure.
METHODS
Population
A total of 150 participants were included. The participants all
consulted in memory clinics and the course of the assessment
was based on usual clinical practice. The study was con-
ducted at the Pitié-Salpêtrière hospital memory clinic and
in the Aloïs Memory Network in the greater Paris area.
Inclusion criteria were: (1) men or women aged 60–80; (2)
consulting at the memory clinic for a cognitive complaint;
(3) Mini-Mental State Examination (MMSE; Folstein,
Downloaded from https://www.cambridge.org/core. University of Glasgow Library, on 16 Aug 2021 at 17:10:32, subject to the Cambridge Core terms of use, available at
https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355617721000679
Folstein & McHugh, 1975) score greater than 20/30; (4) hav-
ing at least primary school education; (5) fluent in French; (6)
able to have a second neuropsychological test 4 months later;
(7) having received and signed the informed consent form.
All subjects were tested by a single experienced examiner
under both conditions (TeleNP and traditional F-F).
At the inclusion visit, all participants underwent a tradi-
tional F-F neuropsychological assessment as part of their
standard care. The medical history (neurological, psychiatric,
cardiovascular, hematological, respiratory, digestive, hepatic,
renal, dermatological, endocrine, and metabolic) of the partici-
pants were collected during this visit.
Four months after study inclusion, they were randomized
to have an additional neuropsychological assessment either
by F-F administration or by TeleNP (see Figure 1). By the
end of the second assessment, participants in the videoconfer-
ence arm were asked to rate the method of assessment via a
five-level Likert scale (very satisfied/moderately satisfied/
rather unsatisfied/very unsatisfied/unknown), as well as a
series of statements about their satisfaction regarding this
new mode of testing, their anxiety during the tele-evaluation,
their preference between the two methods of testing, and how
they felt regarding the presence of the assistant (see below for
the role of the assistant).
The same neuropsychological tests were used at the inclu-
sion (assessment 1) and 4 months after inclusion (assessment
2) and the scores were compared test by test under both con-
ditions (F-F vs. TeleNP). Whenever alternative versions of
tests existed, they were used during assessment 2.
This multicenter, prospective, randomized, compara-
tive study was conducted in accordance with the Helsinki
Declaration and the rules governing telemedicine (teleme-
dicine decree 2010-1229 of 19 October 2010) and was
approved by an Ethics Committee (CPP Ile-de-France X –
Hôpital Robert Ballanger – 93602 Aulnay-sous-Bois
Cedex – N° ID-RCB : 2014-A01971-45). Participants
signed an informed consent for this study on forms approved
by this Ethics Committee.
Material
The F-F assessment took place at the Aloïs Memory Network
centers and at Pitié-Salpêtrière Hospital. The assessment
lasted 2 h. Independently of the administration condition
(F-F or TeleNP), the questionnaire and the tests were admin-
istered in a specific order: State-Trait Anxiety Inventory
(STAI-Y; Spielberger, Gorsuch, Lushene, Vagg, & Jacobs,
1983), MMSE, Digit Span forward and backward
(Wechsler, 1997), learning phase of the Free and Cued
Selective Reminding Test (FCSRT; French version of the
FCSRT; Van der Linden et al., 2004), Mahieux gestural
praxis battery (Mahieux et al., 2009), Frontal Assessment
Battery (FAB; Dubois, Slachevsky, Litvan, & Pillon,
2000), Trail Making Tests (TMT) A and B (Godefroy
et al., 2010), delayed recall phase of the RL/RI-16 items
(Van der Linden et al., 2004), Rey complex figure test
(Meyers & Meyers, 1995), category (animals) and letter
 Teleneuropsychology and face-to-face testing
Inclusion visit
Face-to-face neuropsychological
assessment
N = 150
Lost for follow-up
N=10
Randomisaon
For mounth later
Face-to-face neuropsychological
assessment
N = 69
Fig. 1. Flowchart of the patients analyzed in the study.
(letter “P”) verbal fluency tests (Godefroy et al., 2010), Rey
complex figure recall (Meyers & Meyers, 1995), 80-picture
naming test (DO 80, Deloche & Hannequin, 1997). The
neuropsychological tests used in this study are the tests most
commonly used in France for the cognitive assessment of
dementia. Some are encountered very frequently in the
American and international literature (TMT-A and -B,
Digit Span forward and backward, FAB), while others are
specific to cognitive assessments in France and are validated
in a French population in F-F mode of testing (the French
version of the FCSRT, Mahieux gestural praxis battery).
The TeleNP assessment used a MacBook pro system with
an Internet connection (download speed of 13.14 Mbit/s and
an upload speed of 0.90 Mbit/s), a microphone, and speaker.
Participants were seated in a chair 2 m away from a 40-inch
LCD monitor. A mobile pivoting camera was placed above
the screen and could film the entire room, while another cam-
era (handled by the assistant) could be used when needed to
film anything the subjects wrote or drew. Through a video-
conference interface on a 14-inch LCD monitor, the neuro-
psychologist examiner could simultaneously see the entire
room and the writings and drawings of the subjects. In the
F-F situation, the assistant was not present during the
assessment.
In the TeleNP condition, the subjects did not handle any of
the equipment. The assistant is a secretary who has undergone
1 h of training in computer equipment management (switch
on the computer, connect and open the videoconference inter-
face, check that the loudspeakers are working properly). The
assistant had no training in neuropsychology. She/he inter-
vened at the request of the neuropsychologist to hand over
to the subject the parts of the testing material that we did
not seem desirable for the participants to handle themselves
(especially in the case of a participant with a cognitive impair-
ment that would not allow him to fully understand when and
how to handle the testing material). The assistant welcomed
Downloaded from https://www.cambridge.org/core. University of Glasgow Library, on 16 Aug 2021 at 17:10:32, subject to the Cambridge Core terms of use, available at
https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355617721000679
3
Videoconference-based
neuropsychological assessment
N= 71
the participants, positioned them, and explained to them that
they would not have to handle the computer equipment. The
assistant stayed at their side during all the tests. Above all, he/
she provided practical help with computer videoconferencing
equipment. For the neuropsychological assessment itself, the
assistant intervened for the following tests: (1) the learning
phase of RL/RI-16 by showing and hiding successively the
four boards of the four words to learn; (2) the Rey’s complex
figure test in which the assistant provided a sheet of paper for
the participant to draw a copy of the complex drawing on, and
one to recall it; (3) the subtests 'Prehension behavior’ (the as-
sistant touches the participant’s hands) of the FAB; and (4)
the TMT A and B for which the assistant provided the sheet
on which the participant must perform the test. When errors
were observed in the Trail-Making Test A and B, the assistant
was not allowed to intervene; only the examiner intervened.
After each evaluation, the data were recorded in an anony-
mous database. Tests that were not completed due to cogni-
tive impairment were coded as missing data. In the event of
hardware malfunctions (e.g., important problems with
Internet speed, with image and sound out of sync, etc.), the
data from these tests were not used. Any incident likely to
disturb the conduct of the videoconference assessment was
recorded: delay, out of sync, image or sound frozen, echo,
stress, anxiety, poor comprehension, or behavioral problem.
Statistical Analysis
Data management and statistical analyses were performed
with SAS software (version 9.4). For the descriptive analysis,
qualitative variables were expressed as number and fre-
quency for each modality, and quantitative variables as the
mean, standard deviation (SD), median, interquartile range
(IQR), and minimum/maximum (Min-Max). In order to mea-
sure the effect between neuropsychological test scores at
 4 R. Gnassounou et al.

Table 1. General characteristics of patients according to administration condition

Face-to-face arm (n = 69) Video arm (n = 71) Analysis set (n = 140)

Age at inclusion
Analyzed 67 69 136
Mean (±SD) 69.9 (± 6.1) 71.5 (± 5.9) 70.7 (± 6)
Median 70 72.7 71.8
IQR 64.9–74.7 66.8–76.3 65.8–75.2
Min-Max 60–80.4 60.1–81.3 60–81.3
Sex
Male 26 (37.7%) 32 (45.1%) 58 (41.4%)
Female 43 (62.3%) 39 (54.9%) 82 (58.6%)
Mother tongue
French 62 (91.2%) 71 (100%) 133 (95.7%)
Other 6 (8.8%) 0 (0%) 6 (4.3%)
Missing value 1- 0- 1-
Sociocultural category
Primary school 3 (4.3%) 3 (4.2%) 6 (4.3%)
BEPC or CAP (professional diplomas) 13 (18.8%) 9 (12.7%) 22 (15.7%)
End of high school or manual trade/Craftsman 5 (7.2%) 10 (14.1%) 15 (10.7%)
with responsibilities
Baccalaureate or highly qualified manual trades 10 (14.5%) 7 (9.9%) 17 (12.1%)
University 38 (55.1%) 42 (59.2%) 80 (57.1%)

assessments 1 and 2 according to the administration method
(F-F vs. TeleNP), the Cohen’s d was calculated.
The Student’s t test and the Pearson’s chi-squared test
(used when all expected numbers were greater than 5, other-
wise the Fisher’s exact test was applied when relevant) were
used to compare means and proportions, respectively,
between the F-F and TeleNP arms (characteristics at baseline
and neuropsychological results at each assessment). The
Student’s t test for paired data and the McNemar-Bowker test
were used to compare distributions of neuropsychological
test scores and symmetry for categorical endpoints, between
assessments 1 and 2, for each administration method (F-F and
TeleNP). In addition, weighted kappa coefficients were pro-
vided to measure the agreement (very low: 0–0.21; low:
0.21–0.4; moderate: 0.41–0.60; high to almost perfect:
0.61–0.80; almost perfect: 0.81–1; Landis & Koch, 1977)
between ordinal endpoints as STAI-Y scores (anxiety state
and trait). Moreover, ANCOVA models were performed to
compare neuropsychological test scores between the F-F
and TeleNP arms at assessment 2 controlling results at assess-
ment 1. In order to consider the multiplicity of tests, the
Benjamini–Hochberg method (Benjamini, & Hochberg,
2000) was used to control the false discovery rate (FDR)
and to provide adjusted p-values. The level of significance
was set at 5% for all statistical tests.
RESULTS
Sample Characteristics
Across the whole sample, 6.7% (n = 10) of participants left
the study before the second assessment. For the 140
participants who had assessments 1 and 2 (regardless of
administration condition), the mean MMSE score was
27.4 ± 2 at assessment 1. At assessment 2, there were
69 participants (49.2%) in the F-F arm and 71 (50.7%)
in the TeleNP arm. Demographic characteristics are shown
in Table 1 and were similar between both arms.
Both arms were also comparable in terms of medical
history and more specifically neurological (around 30% of
patients, p = 0.53), psychiatric (around 37% of patients,
p = 0.9), and vascular history (around 50% of patients,
p = 0.53), with the exception of a significant difference in
terms of the presence of hypertension (44.9% vs. 28.2%,
p = 0.04), between the F-F and TeleNP arms, respectively,
(p-values from chi-square tests).
Neuropsychological Test Scores at Assessments 1
and 2 in Each Arm
The results indicate that the cognitive test scores from assess-
ments 1 and 2 were stable in both arm (F-F and TeleNP)
for almost all of the assessment tests, respectively : MMSE
(p = 0.50 vs. 0.37), learning phase of the RL/RI-16 items
(p = 0.21 vs. 0.61), total of three free recalls of the RL/RI-
16 items (p = 0.98 vs. 0.42), total of three recalls of the
RL/RI-16 items (p = 0.18 vs. 0.62), Mahieux gestural praxis
battery (p = 0.61 vs. 0.49), TMT A (time only, p = 0.26 vs.
0.10) and B (time p = 0.92 vs. 0.26, and number of errors
p = 0.70 vs. 0.72), Rey complex figure test (p = 0.75 vs.
0.10), fluency task category “animals” (p = 0.54 vs.
0.08) and letter “P”( p = 0.48 vs. 0.18). A few significant
difference was initially noted in the F-F arm (DO 80 picture

Downloaded from https://www.cambridge.org/core. University of Glasgow Library, on 16 Aug 2021 at 17:10:32, subject to the Cambridge Core terms of use, available at
https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355617721000679
 Teleneuropsychology and face-to-face testing 5

Table 2. Comparison of neuropsychological test scores at assessments 1 and 2, according to administration method (face to face and video) and
between both methods.

Name of neuropsychological Analysis set Face-to-Face Assessment 1 Assessment 2

test arm or Video arm Mean (± SD) Mean (± SD) p* Cohen’s d Adjusted p**

Mini-Mental State Face to face 27.4 (± 2) 27.5 (± 1.8) 0.50 0.08 0.95
Examination Video 27.5 (± 1.9) 27.3 (± 2.4) 0.37 0.11 0.56
(MMSE) (/30) p*** 0.69 0.61
Cohen’s d 0.07 0.09
p**** (ANCOVA) 0.31
Digit span (/19) Face to face 8.7 (± 3.1) 8.5 (± 3.6) 0.61 0.06 0.95
Video 8.9 (± 3.1) 8.1 (± 3.5) 0.02 0.28 0.15
p*** 0.66 0.53
Cohen’s d 0.07 0.11
Adjusted p**** (ANCOVA) 0.18
RL/RI-16: learning phase Face to face 14.8 (± 1.9) 14.6 (± 1.6) 0.21 0.15 0.88
(/16) Video 15 (± 1.7) 15 (± 1.2) 0.61 0.06 0.62
p*** 0.66 0.07
Cohen’s d 0.07 0.31
Adjusted p**** (ANCOVA) 0.05
Adjusted p*** 0.55
RL/RI-16: Total of three Face to face 27.4 (± 7.5) 27.4 (± 8.4) 0.98 0 1.00
free recalls (/48) Video 26.8 (± 8.6) 27.4 (± 8.1) 0.42 0.1 0.57
p*** 0.64 0.99
Cohen’s d 0.08 0
Adjusted p**** (ANCOVA) 0.62
RL/RI-16: Total of three Face to face 44,3 (± 6,1) 43,8 (± 6,4) 0.18 0.16 0.88
recalls (/48) Video 43,4 (± 6,9) 43,8 (± 7,2) 0.62 0.06 0.62
p*** 0.43 0.97
Cohen’s d 0.13 0.01
Adjusted p**** (ANCOVA) 0.33
Frontal Assessment Battery Face to face 15.2 (± 1.8) 15.7 (± 1.9) 0.01 0.33 0.17
(FAB) (/18) Video 15.4 (± 1.8) 15.4 (± 1.8) 1.00 0 1
p*** 0.49 0.33
Cohen’s d 0.12 0.17
Adjusted p**** (ANCOVA) 0.09
Mahieux gestural praxis Face to face 22.8 (± 0.9) 22.9 (± 0.7) 0.61 0.06 0.95
battery (/23) Video 22.8 (± 1.1) 22.9 (± 0.6) 0.49 0.08 0.61
p*** 0.90 0.87
Cohen’s d 0.02 0.03
Adjusted p**** (ANCOVA) 0.9
Trail Making Test A – time Face to face 49.5 (± 29.3) 52.5 (± 39.6) 0.26 0.14 0.88
(in seconds) Video 53.7 (± 31.2) 49.7 (± 30.4) 0.10 0.21 0.30
p*** 0.42 0.65
Cohen’s d 0.14 0.08
Adjusted p**** (ANCOVA) 0.1
Trail Making Test A – Face to face 0.1 (± 0.3) 0.1 (± 0.3) 1.00 0 1.00
number of errors Video 0.4 (± 1.3) 0 (± 0.1) 0.01 0.35 0.15
p*** 0.03 0.09
Cohen’s d 0.38 0.3
Adjusted p*** (ANCOVA) 0.55
Trail Making Test B – Face to face 124.7 (± 74.8) 123.2 (± 64.7) 0.92 0.01 1.00
time (in seconds) Video 124.4 (± 73.9) 113.4 (± 64.4) 0.26 0.15 0.45
p*** 0.98 0.39
Cohen’s d 0 0.15
Adjusted p**** (ANCOVA) 0.36
Trail Making Test B – Face to face 0,7 (± 1,2) 0,6 (± 0,9) 0.70 0.05 0.95
number of errors Video 0,6 (± 1) 0,6 (± 0,9) 0.72 0.05 0.72
p*** 0.85 0.86
Cohen’s d 0.03 0.03
Adjusted p**** (ANCOVA) 0.8
(Continued)

Downloaded from https://www.cambridge.org/core. University of Glasgow Library, on 16 Aug 2021 at 17:10:32, subject to the Cambridge Core terms of use, available at
https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355617721000679
 6 R. Gnassounou et al.

Table 2. (Continued )
Name of neuropsychological Analysis set Face-to-Face Assessment 1 Assessment 2
test arm or Video arm Mean (± SD) Mean (± SD) p* Cohen’s d Adjusted p**
Rey/Taylor complex figure Face to face 34.3 (± 2.8) 34 (± 3.6) 0.75 0.04 0.95
copy score (/36 points) Video 33.8 (± 3.6) 34.3 (± 3.1) 0.10 0.21 0.30
p*** 0.43 0.66
Cohen’s d 0.14 0.08
Adjusted p**** (ANCOVA) 0.23
Animal verbal fluency Face to face 26 (± 8.8) 26.3 (± 8.4) 0.54 0.07 0.95
(number of items) Video 26.6 (± 8.8) 25.6 (± 8.1) 0.08 0.21 0.30
p*** 0.66 0.59
Cohen’s d 0.08 0.09
Adjusted p**** (ANCOVA) 0.1
Letter « P » verbal fluency Face to face 19.4 (± 6.7) 19.8 (± 7.8) 0.48 0.09 0.95
(number of item) Video 19.8 (± 7.7) 19.1 (± 7.7) 0.18 0.16 0.45
p*** 0.74 0.58
Cohen’s d 0.06 0.09
Adjusted p**** (ANCOVA) 0.17
Oral picture naming test – Face to face 77.8 (± 4.3) 78.2 (± 4.2) 0.05 0.24 0.43
80 items (/80) Video 78.3 (± 2.6) 78.6 (± 2) 0.22 0.15 0.45
p*** 0.36 0.43
Cohen’s d 0.16 0.14
Adjusted p****(ANCOVA) 0.77

*Student’s t test for paired data; **Adjustment according to the Benjamini–Hochberg method; ***Student’s t test; ****ANCOVA model: comparison of assess-
ment 2 value between F-F and video arms controlling assessment 1 value (see Table 3 for adjustment according to the Benjamini–Hochberg method for adjusted
p-value ANCOVA).

naming p = 0.05 vs. 0.22, and FAB p = 0.01 vs. 1.00) and in
TeleNP arm (Digit Span forward and backward p = 0.61 vs.
0.02, and TMT A number of errors p = 1.00 vs. 0.01)
between assessments 1 and 2 with the use of tests for paired
data (Student’s or McNemar-Bowker’s) (Tables 2 and 3).
No significant difference exists when controlling the FDR
with the Benjamini–Hochberg method (Table 3). Scores
were not significantly different between assessments 1
and 2 for the DO 80 picture naming test (initially p = 0.05
vs. adjusted p = 0.43) and the FAB (initially p = 0.01 vs.
adjusted p = 0.17) in the F-F arm, and for the Digit Span
forward and backward (initially p = 0.01 vs. adjusted
p = 0.17) and TMT A number of errors (initially p = 0.01
vs. adjusted p = 0.15) in the TeleNP arm (Tables 2 and 3).
Moreover, the Cohen’s d values showed small-to-medium size
effects with a maximum for the FAB (Cohen’s d = 0.33) in the
F-F arm and for the TMT A number of errors (Cohen’s
d = 0.35) in the situation of videoconference after first F-F
assessment (Table 2).
ANCOVA models showed that score results were similar
between the two administration conditions at time 2, control-
ling assessments results at time 1 (confirmed controlling FDR,
Tables 2 and 3). All assessment results were not significantly
different between both administration modes (at time 2).
Level of Anxiety during Test Administration
Fisher’s tests used indicated an absence of significant differ-
ence between the two types groups in terms of anxiety level
felt by patients, both in assessment 1 (F-F assessments) and
assessment 2 (F-F vs. TeleNP) (Table 4). A total of 63.1% and
70.4% of patients judged their anxiety state (STAI-Y-A) low
or very low at assessment 2 in, respectively, F-F and TeleNP
conditions, higher proportion in TeleNP, but not significantly
(p = 0.26). These proportions of low anxiety state were
74.2% and 79.7% at the first F-F assessment in F-F and
TeleNP arm, indicating in both groups a trend of a lower level
of anxiety at the second assessment, but not significantly
(Bowker’s test for paired categorical data in each arm,
p-values ≥0.27). Low-to-moderate agreements are measured
with Kappa’s coefficients K between assessments 1 and 2 in
each arm (K≤0.61). These trends are also observed with
regards to level of Anxiety Trait (STAI-Y-B) (Table 4).
Satisfaction with Videoconference Administration
Overall satisfaction with assessment by videoconference was
87.1% (participants “very satisfied”), and, respectively,
82.9% and 74.2% of participants assessed the F-F testing
as effective and tiring as TeleNP testing, while 70% reported
not feeling anxiety with this new administration method.
Only 2 participants indicated that they were rather unsatisfied
and 0 very unsatisfied.
Administration or Hardware Malfunctions Across
the Study
In total, eight incidents occurred in our study. Six involved
minor problems that were rapidly resolved (slow connection,
sound/image out of sync) and two were “participant” events

Downloaded from https://www.cambridge.org/core. University of Glasgow Library, on 16 Aug 2021 at 17:10:32, subject to the Cambridge Core terms of use, available at
https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355617721000679
https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355617721000679
Downloaded from https://www.cambridge.org/core. University of Glasgow Library, on 16 Aug 2021 at 17:10:32, subject to the Cambridge Core terms of use, available at

Teleneuropsychology and face-to-face testing

Table 3. P-values adjustments (FDR) for neuropsychological tests scores

Assessment 1 versus assessment 2, Assessment 1 versus assessment 2, in TeleNP F-F arm versus TeleNP arm, assessment 2
in F-F arm (n = 17 tests) arm (n = 17 tests) adjusted for assessment 1 (n = 15)
Neuropsychological tests p-values* Rank Adjusted p-values p-values* Rank – TelePN Adjusted p-values p-values (ANCOVA) Rank Adjusted p-values
MMSE 0.50 7 0.95 0.37 10 0.56 0.31 7 0.6
Digit span 0.61 9 0.95 0.02 2 0.15 0.18 5 0.54
RL/RI-16 learning phase 0.21 4 0.88 0.61 14 0.62 0.55 10 0.75
RL/RI-16 total of three free recalls 0.98 16 1 0.42 11 0.57 0.62 12 0.78
RL/RI-16 total of three recalls 0.18 3 0.88 0.62 15 0.62 0.33 8 0.6
Frontal Assessment Battery (FAB) 0.01 1 0.17 1.00 17 1 0.09 1 0.5
Mahieux gestual praxis battery 0.61 10 0.95 0.49 12 0.61 0.90 15 0.9
Trail Making Test (TMT) A – time 0.26 5 0.88 0.10 4 0.3 0.1 2 0.5
Trail Making Test (TMT) A – number of errors 1.00 17 1 0.01 1 0.15 0.55 11 0.75
Trail Making Test (TMT) B – time 0.92 15 1 0.26 8 0.45 0.36 9 0.6
Trail Making Test (TMT) B – number of errors 0.70 11 0.95 0.72 16 0.72 0.80 14 0.86
Rey/Taylor copy score 0.75 13 0.95 0.10 5 0.3 0.23 6 0.58
Animal verbal fluency 0.54 8 0.95 0.08 3 0.3 0.1 3 0.5
Letter « P » verbal fluency 0.48 6 0.95 0.18 6 0.45 0.17 4 0.54
DO 80-picture naming test 0.05 2 0.43 0.22 7 0.45 0.77 13 0.86
Anxiety state 0.73 12 0.95 0.27 9 0.45 Not including – –
Anxiety trait 0.78 14 0.95 0.55 13 0.62 – –

* Student’s t test for paired data except for all scores except for anxiety state and trait tested with Bowker’s tests of symmetry.

7
 8 R. Gnassounou et al.

Table 4. Comparison of the State-Trait Anxiety Inventory (STAI-Y) at assessments 1 and 2, according to administration method (face to face
and video)

Analysis set Kappa coefficient

face-to-face arm Level of (Assess.1 vs. 2)
Name of the test or video arm anxiety Assessment 1 % Assessment 2 % (IC 95 %) p* Adjusted p**

State-Trait Anxiety Face to face Very low 39.4 38.5 0.50 [0.34; 0.67] 0.73 0.95
Inventory (STAI-Y) – Low 34.8 24.6
part A (anxiety Average 15.2 23.1
state) (/80) High 7.6 7.7
Very high 3 6.2
Video Very low 62.3 49.3 0.37 [0.18; 0.55] 0.27 0.45
Low 17.4 21.1
Average 10.1 21.1
High 8.7 8.5
Very high 1.4 0
p** 0.06 0.26
State-Trait Anxiety Face to face Very low 20.9 21.9 0.61 [0.48; 0.74] 0.78 0.95
Inventory (STAI-Y) – Low 35.8 31.3
part B (anxiety Average 20.9 31.3
trait) (/80) High 14.9 9.4
Very high 7.5 6.3
Video Very low 22.9 28.2 0.52 [0.38; 0.66] 0.55 0.62
Low 35.7 33.8
Average 24.3 25.4
High 15.7 12.7
Very high 1.4 0
p*** 0.58 0.24

*Bowker’s tests of symmetry: Assessment 1 versus Assessment 2, **Fisher’s tests: F-F versus video; ***Adjustment according to the Benjamini–Hochberg
method.

Table 5. Frequency of administration or hardware malfunctions across the study (n = 8)

Resolution at end of
Type Description Action undertaken videoconference
1 Equipment incident Poor connection choppy sound/video None, assessment continued Resolved
with no impact on the assessment
2 Equipment incident Poor connection choppy sound/video None, assessment continued Resolved
with no impact on the assessment
3 Equipment incident Two failed connections Rapid reconnection Resolved
4 Equipment incident Choppy sound None, assessment continued Resolved
5 Equipment incident Sound/image out of sync Deconnection, reconnection Resolved
6 Equipment incident Sound/image out of sync Deconnection, reconnection Resolved
7 Patient event Patient disturbed by the camera Assessment stopped Non-resolved
8 Patient event Patient disturbed by computer noise Assessment continued Non-resolved

(one subject was disturbed by the presence of the camera and
another by the noise of the computer), leading one participant
to stop the assessment. Details of administration or hardware
malfunctions across the study are presented in Table 5.
DISCUSSION
To date, this is the first French study of neuropsychological
assessments administered by videoconference. Our data indi-
cate highly similar results across both administration condi-
tions (F-F and TeleNP). These results are consistent with
most of the findings from previous studies in TeleNP
(Cullum & Grosch, 2012; Munro Cullum et al., 2014;
Wadsworth et al., 2018). In addition, we also investigated
the feasibility and reliability in routine clinical practice of
using neuropsychological tests commonly employed in
dementia evaluations and including complex tasks evaluating
processing speed or motor and visual abilities. In our study,
the TeleNP assessment procedure was well accepted by all
subjects. Most participants (87.1%) appeared to be very sat-
isfied with the videoconference testing, and, respectively,
82.9% and 74.2% of participants assessed the F-F testing

Downloaded from https://www.cambridge.org/core. University of Glasgow Library, on 16 Aug 2021 at 17:10:32, subject to the Cambridge Core terms of use, available at
https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355617721000679
 Teleneuropsychology and face-to-face testing
as effective and tiring as TeleNP testing, which is more than
in the study of Parikh et al. (2013) who noted that 57% of
subjects with no cognitive impairment and 63% with cogni-
tive impairments showed no preference for either testing
mode. However, our study did not involve videoconference
testing from the get-go but only during the second evaluation.
Therefore, evaluating satisfaction in this second evaluation
may have resulted in a positive bias due to the fact that the
patients have already started their follow-up and care. In addi-
tion, it would have been interesting if the survey was also pro-
vided to participants who received the F-F component in
order to allow a comparison of the two modalities on the ques-
tion of satisfaction.
All participants in the TeleNP arm were very satisfied with
the assistant’s support. Yet, the influence that an assistant
may have and the need for his/her presence during the testing
have been debated in the literature. The meta-analysis of
Brearly et al. (2017) did not include studies involving an
assistant because of considerable heterogeneity in the
results of “motor-dependent” tasks between the two ad-
ministration methods. On the other hand, other studies
have administered motor-dependent tasks with staff to
assist with the presentation of stimuli and test versions,
while a qualified examiner provided verbal instructions
from a distance and did not find any significant differences
(Temple et al., 2010; Turkstra et al., 2012). In our study,
all the subjects gave positive feedback regarding the
assistant’s support whose presence minimized active inter-
actions between the participant and the computer hard-
ware; and provided help with possible communication
problems related to computer malfunction and with the
presentation of certain stimuli and understanding of test
phases. In summary, the assistant made test administration
proceed more smoothly (facilitating the complex order of
the rules for test administration). In our experience, the
assistant’s contribution seems even more important if the
participants are likely to present cognitive disorders as
in the case of apraxia or executive disorders.
Loss of information due to audio or video stuttering or
poor signal quality may, depending on the task, have a greater
or lesser impact on subject performance. However, one of the
limitations of TeleNP is related to current imperfections in
signal transmission techniques. Although in our study, there
were no significant audio or visual transmission problems, or
inability to understand and follow the test instructions for all
participants in the TeleNP arm, it is important to consider
(especially at a time of expansion of this technique) that eight
incidents occurred in our study (Table 5). Otherwise, some
tasks such as the MMSE copy task or the Rey complex figure
test required the use of additional equipment (mobile tripod
camera manipulated by the assistant when requested by the
test). On two occasions, this caused a delay between the
sound and the image on timed tests (TMT and Rey complex
figure). This disturbance could have been avoided if the
examiner had been able to manipulate the camera remotely.
Further investigations with tests more complex to administer,
and development of more ergonomic equipment (designed
Downloaded from https://www.cambridge.org/core. University of Glasgow Library, on 16 Aug 2021 at 17:10:32, subject to the Cambridge Core terms of use, available at
https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355617721000679
9
and specific to this type of assessment), as well as standard-
ized methods for presenting visual stimuli, are therefore
needed to improve the sensitivity, homogeneity, and fluidity
in test administration via TeleNP.
Tests used in routine clinical practice have been standard-
ized and validated through an F-F testing mode, which raises
the question of their clinical interpretation and the use of
norms during TeleNP testing. Validation tests specifically
designed for TeleNP will be an absolute necessity in the
future. In our study, changes in the testing procedure, even
minor ones (e.g., administering RL/RI-16 boards over the
computer instead of in-person, although boards can also be
shown on the screen by the examiner), must also be consid-
ered in our protocol.
Generally speaking, the conclusions of our study cannot
be applied to all population groups. Indeed, our participants
do not have major cognitive impairments, have a fairly good
education level and they shared the same language and cul-
ture than the examiner.
Increased access to this clinical resource would offer
multiple opportunities for underserved areas, where these
diagnostic tools are not currently available to help isolated
individuals living in medical deserts (Adjorlolo, 2015),
institutionalized dependent or disabled peoples, and even
emigrants. TeleNP can become an instrument of choice
in this case. On the other hand, our procedure does not
allow for social distancing between the assistant and the
patients, and therefore cannot be recommended to identify
cognitive disorders in patients who are on lockdown, like
in the current coronavirus pandemic (COVID-19). After a
short video training, a member of the nursing staff could
become the TeleNP assistant. But this model is not appli-
cable for “Directly Into the Home” testing, as it is imper-
ative that the assistants do not personally know the patient
being tested, in order to allow them to be as neutral as pos-
sible, and to prevent them from helping or distracting the
subject. Moreover, the presence of an assistant also makes
it possible not to leave the tests available to the patients
after their evaluation, in particular to prevent the dissemi-
nation of tests in the public domain and the possibility for
patients to practice their administration.
Incorporated as part of an interdisciplinary plan for the
care and management of dementia, TeleNP would facilitate
early diagnosis (delaying the loss of autonomy, promoting
home care), allow better care for patients and their
caregivers (better targeting of pharmacological and non-
pharmacological interventions), and of course, lead to a
reduction in treatment costs (savings for patients and for
public funds).
In conclusion, the results of our study support the use of
TeleNP in medical deserts, but this method needs to be tested
in subjects with more severe cognitive impairment and with
subjects with major cognitive disorders and significant sen-
sory limitations to ensure generalization. With technological
developments, TeleNP may in the future be enriched by new
digital communication tools and thus become a precious aid
in case of geographical isolation.
 10
ACKNOWLEDGMENTS
The authors wish to thank Professor Bruno Dubois, Director
of the “Institut de la Mémoire et de la Maladie d’Alzheimer”
(IM2A), neurologist at IM2A, for sending participants who
were evaluated for the study in the institute’s premises.
CONFLICT OF INTEREST
The author(s) have nothing to disclose.
FINANCIAL SUPPORT
This work was supported by KLESIA pension funds N°
20150227.
ETHICAL STANDARDS
This study was conducted in accordance with the Helsinki
Declaration and the rules governing telemedicine (telemedi-
cine decree 2010–1229 of 19 October 2010) and was
approved by an Ethics Committee (CPP Ile-de-France X –
Hôpital Robert Ballanger – 93602 Aulnay-sous-Bois cedex
– N° ID-RCB : 2014-A01971-45).
REFERENCES
Adjorlolo, S. (2015). Can teleneuropsychology help meet the neuro-
psychological needs of Western Africans? The case of Ghana.
Applied Neuropsychology: Adult, 22, 388–398.
Benjamini, Y. & Hochberg, Y. (2000). On the Adaptive Control of
the False Discovery Rate in MultipleTesting with Independent
Statistics. Journal of Educational and Behavioral Statistics, 25,
60–83.
Brearly, T.W., Shura, R.D., Martindale, S.L., Lazowski, R.A.,
Luxton, D.D., Shenal, B.V., & Rowland, J.A. (2017).
Neuropsychological Test Administration by Videoconference:
A Systematic Review and Meta-Analysis. Neuropsychology
Review, 27, 174–186.
Castanho, T.C., Amorim, L., Moreira, P.S., Mariz, J., Palha, J.A.,
Sousa, N., & Santos, N.C. (2016). Assessing Cognitive
Function in Older Adults Using a Videoconference Approach.
EBioMedicine, 11, 278–284.
Chodosh, J., Petitti, D.B., Elliott, M., Hays, R.D., Crooks, V.C.,
Reuben, D.B., & Wenger, N. (2004). Physician recognition of
cognitive impairment: Evaluating the need for improvement:
Recognizing cognitive impairment. Journal of the American
Geriatrics Society, 52, 1051–1059.
Cullum, C.M., Weiner, M.F., Gehrmann, H.R., & Hynan, L.S.
(2006). Feasibility of Telecognitive Assessment in Dementia.
Assessment, 13, 385–390.
Cullum, C.M. & Grosch, M.G. (2012).Teleneuropsychology. In
K.. Myers and C. Turvey (Eds.), Telemental Health: Clinical,
Technical, and Administrative Foundations for Evidence-Based
Practice (pp. 275–294). Amsterdam: Elsevier.
Cullum, C.M., Hynan, L.S., Grosch, M., Parikh, M., &Weiner, M.F.
(2014). Teleneuropsychology: evidence for video teleconference-
based neuropsychological assessment. Journal of the Interna-
tional Neuropsychological Society, 20, 1028–1033.
Downloaded from https://www.cambridge.org/core. University of Glasgow Library, on 16 Aug 2021 at 17:10:32, subject to the Cambridge Core terms of use, available at
https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355617721000679
R. Gnassounou et al.
Deloche, G., & Hannequin, D. (1997). Test de dénomination orale
d’images : DO 80. Paris: Les Éditions du Centre de Psychologie
Appliquée (ECPA).
Dubois, B., Slachevsky, A., Litvan, I., & Pillon, B. (2000). A frontal
assessment battery at bedside. Neurology, 55, 1621–1626.
Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). “Mini-
mental state”. A practical method for grading the cognitive state
of patients for the clinician. Journal of Psychiatric Research, 12,
189–198.
Galusha-Glasscock, J.M., Horton, D.K., Weiner, M.F., &Cullum,
C.M. (2016). Video Teleconference Administration of the
Repeatable Battery for the Assessment of Neuropsychological
Status. Archive of Clinical Neuropsychology, 31, 8–11.
Godefroy, O., Azouvi, P., Robert, P., Roussel, M., Le Gall, D.,
Meulemans T., & the Groupe de Réflexion sur l’Evaluation des
Fonctions Exécutives Study Group. (2010). Dysexecutive syn-
drome: Diagnostic criteria and validation study. Annals of
Neurology, 68, 855–864.
Grosch, M.C., Weiner, M.F., Hynan, L.S., Shore, J., & Cullum,
C.M. (2015). Video Teleconference-based neurocognitive screen-
ing in geropsychiatry. Psychiatry Research, 225, 734–735.
Harrell, K.M., Wilkins, S.S., Connor, M.K., & Chodosh, J.
(2014). Telemedicine and the evaluation of cognitive impair-
ment: The additive value of neuropsychological assessment.
Journal of the American Medical Directors Association, 15,
600–606.
Hildebrand, R., Chow, H., Williams, C., Nelson, M., & Wass, P.
(2004). Feasibility of neuropsychological testing of older adults
via videoconference: Implications for assessing the capacity for
independent living. Journal of Telemedicine and Telecare, 10,
130–134.
INSEE. (2014). Statistiques de l’état civil et estimations de popula-
tion. Retrieved October 16, 2016 from www.insee.fr
Jacobsen, S.E., Sprenger, T., Andersson, S., & Krogstad, J.M.
(2003). Neuropsychological Assessment and Telemedicine:
A Preliminary Study Examining the Reliability of
Neuropsychology Services Performed via Telecommunication.
Journal of the International Neuropsychological Society, 9,
472–478.
Joint Task Force for the Development of Telepsychology Guidelines
for Psychologists (2013). Guidelines for the practice of telepsy-
chology. American Psychologist, 68, 791–800.
Jollivet, M., Fortier, J., Besnard, J., Le Gall, D., & Allain, P. (2018).
Neuropsychologie et technologies numériques. Revue de
Neuropsychologie, 10, 69–81.
Landis, J.R. & Koch, G.G. (1977). The measurement of observer
agreement for categorical data. Biometrics, 33, 159–174.
Mahieux-Laurent, F., Fabre, C., Galbrun, E., Dubrulle, A., Moroni,
C., & the members of the Groupe de Réflexion sur les Praxies du
CMRR Ile-de-France Sud (2009). Validation of a brief screening
scale evaluating praxic abilities for use in memory clinics.
Evaluation in 419 controls, 127 mild cognitive impairment and
320 demented patients. Revue de Neurologie, 165, 560–567.
Meyers, J.E. & Meyers, K.R. (1995). Rey Complex Figure Test and
Recognition Trial: Professional Manual. Odessa: Psychological
Assessment Resource.
Munro Cullum, C., Hynan, L.S., Grosch, M., Parikh, M., & Weiner,
M.F. (2014). Teleneuropsychology: Evidence for Video
Teleconference-Based Neuropsychological Assessment. Journal
of the International Neuropsychological Society, 20, 1028–1033.
Parikh, M., Grosch, M.C., Graham, L.L., Hynan, L.S., Weiner, M.,
Shore, J.H., & Cullum, C.M. (2013). Consumer acceptability of
 Teleneuropsychology and face-to-face testing
brief videoconference-based neuropsychological assessment in
older individuals with and without cognitive impairment. The
Clinical Neuropsychologist, 27, 808–817.
Spielberger, C.D., Gorsuch, R.L., Lushene, R., Vagg, P.R., &
Jacobs, G.A. (1983). Manual for the State-Trait Anxiety
Inventory. Palo Alto: Consulting Psychologists Press.
Temple, V., Drummond, C., Valiquette, S., & Jozsvai, E. (2010). A
comparison of intellectual assessments over video conferencing
and in-person for individuals with ID: Preliminary data:
Intellectual assessment over video conferencing. Journal of
Intellectual Disability Research, 54, 573–577.
Turkstra, L.S., Quinn-Padron, M., Johnson, J.E., Workinger, M.S., &
Antoniotti, N. (2012). In-Person Versus Telehealth Assessment of
Discourse Ability in Adults With Traumatic Brain Injury. Journal
of Head Trauma Rehabilitation, 27, 424–432.
Van der Linden, M., Coyette, F., Poitrenaud, J., Kalafat, M., Calacis,
F., Wyns, C., & the members of the GREMEM (2004). L’épreuve
de rappel libre/rappel indicé à 16 items (RL/RI-16). In M. Van der
Linden, S. Adam Agniel, & the members of the GREMEM
Downloaded from https://www.cambridge.org/core. University of Glasgow Library, on 16 Aug 2021 at 17:10:32, subject to the Cambridge Core terms of use, available at
https://www.cambridge.org/core/terms. https://doi.org/10.1017/S1355617721000679
11
(Eds.), L’évaluation des troubles de la mémoire: présentation
de quatre tests de mémoire épisodique (avec leur étalonnage)
(pp. 25–47). Marseille: Solal.
Vestal, L., Smith-Olinde, L., Hicks, G., Hutton, T., & Hart, J.
(2006). Efficacy of language assessment in Alzheimer’s disease:
comparing in-person examination and telemedicine. Clinical
Interventions in Aging, 1, 467–471.
Wadsworth, H.E., Galusha-Glasscock, J.M., Womack, K.B.,
Quiceno, M., Weiner, M.F., Hynan, L.S., & Cullum, C.M.
(2016). Remote neuropsychological assessment in rural
American Indians with and without cognitive impairment.
Archives of Clinical Neuropsychology, 31, 420–425.
Wadsworth, H.E., Dhima, K., Womack, K.B, Hart, J., Weiner, M.F.,
Hynan, L.S., & Cullum, C.M. (2018). Validity of teleneuropsy-
chological assessment in older patients with cognitive disorders.
Archives of Clinical Neuropsychology, 33, 1040–1045.
Wechsler, D. (1997). Wechsler Adult Intelligence Scale III: Scoring
and Administration Manual. San Antonio: The Psychological
Corporation.