Goal Management Training: A Web-Based Approach

by

Halla Fahmi

A thesis submitted in conformity with the requirements

for the degree of Master of Science
Graduate Department of Rehabilitation Science
University of Toronto

© Copyright by Halla Fahmi (2013)

 Goal Management Training- A Web-Based Approach

Halla Fahmi

Master of Science

Graduate Department of Rehabilitation Science

University of Toronto

2013

Abstract
This study was undertaken to introduce an innovative approach to cognitive rehabilitation

intervention (Goal Management Training- GMT) delivery, through a web-based platform

administered to adults with cerebrovascular disease (CVD) or suffering from CVD risk factors

who presented with executive function impairments. The feasibility of this approach was

investigated by developing a semi-quantitative-qualitative tool to measure therapist competence

and group engagement adapted from the Cognitive Therapy adherence-to-protocol scale. Results

from two raters analyzing random web-based GMT session recordings showed no compromise in

any aspect measured. In addition, the efficacy of the intervention was established using

neuropsychological and functional outcome measures, with significant results observed on the

Goal Attainment Scale functional measure. To our knowledge, this study is the first of its kind to

a) employ videoconferencing technology to overcome accessibility barrier to cognitive

rehabilitation and b) develop an adherence to protocol tool to measure various aspects of GMT.

ii
 Acknowledgments
Firstly, I would like to sincerely thank Drs. Sandra Black and Gary Turner for giving me
the opportunity to work on this unique project under their supervision. This thesis would not
have been possible without their continued support and guidance in every step of the way. I am
especially thankful to Dr. Black for referring me to Dr. Prathiba Shammi who has extensively
helped me gain professional experience as a Psychometrist and been a wonderful teacher. I am
indebted to Dr. Gary Turner for his direction, encouragement, and above all patience in helping
me formulate and attain my goals for the past two years. I would also like to thank my committee
members Drs. Brian Levine, Deirdre Dawson and Elsa Marziali for their helpful advice, expert
feedback and continued guidance in steering this project, as well as Dr. Laura Middleton who has
helped me with her valuable expertise. This research is only possible with the support of various
funding bodies that have supported me and this project: The Heart and Stroke Foundation Centre
for Stroke Recovery, the North and East GTA Regional Stroke Network and the Department of
Rehabilitation Science, University of Toronto.

I would like to acknowledge and express my gratitude to the members of the research
team at Baycrest- Rotman Research Institute who have helped me with all aspects of the study:
Priya Kumar, Wayne Khuu and Marjorie Green. I am also grateful for the help of Dr. Marziali’s
research team: Illia Tchernikov, Martin Yeung and Malcolm Chan, for their aid and training with
the technology. I owe my deepest appreciation to the generous commitment and involvement of
the eight participants who dedicated their time to this study providing us with valuable data and
findings.

Thank you to all my friends especially Sidrah Arshad, for referring me to the Graduate
Department of Rehabilitation Science and for her mentorship all throughout. Finally, I would like
to express my sincerest words of thanks and gratitude to my family; my father Dr. Khalid Fahmi
for being the role model who has inspired me to dive into the world of scientific research, my
mother for her unconditional love and for always believing in me and my sisters for their
continued support. Not forgetting my nephew Omar Estwani for never failing to make any day a
good day. Last but not least, I thank God (Alhamdulillah) for all the blessings above.

iii
 Table of Contents
Abstract ........................................................................................................................................... ii
Acknowledgments.......................................................................................................................... iii
Table of Contents ........................................................................................................................... iv
List of Tables ................................................................................................................................. vi
List of Figures ............................................................................................................................... vii
List of Appendices ....................................................................................................................... viii
Chapter 1 Introduction .................................................................................................................... 1
Chapter 2 Literature Review ........................................................................................................... 3
Introduction ................................................................................................................................ 3
2.1 Frontal executive functions: neural systems ....................................................................... 4
2.2 Models of Frontal Executive Functions .............................................................................. 6
2.2.1 The Supervisory Attentional System SAS (Shallice & Norman, 1980) ................. 6
2.2.2 Working Memory: Central Executive System ........................................................ 7
2.2.3 Goal Selection and Goal Neglect (Duncan, 1986) .................................................. 8
2.3 Goal Management Training .............................................................................................. 10
2.4 Executive Dysfunction, Stroke and Cerebrovascular Disease .......................................... 12
2.5 Web-Based Delivery Platforms ........................................................................................ 12
2.5.1 E-Education........................................................................................................... 12
2.5.2 Remote delivery of Cognitive Behavioural Therapy and cognitive
rehabilitation ......................................................................................................... 13
2.5.3 Caregiver Support Groups .................................................................................... 14
2.6 International Classification of Functioning ...................................................................... 15
2.7 Conclusion ........................................................................................................................ 18
Chapter 3 Research Plan and Protocol .......................................................................................... 19
3.1 Background ....................................................................................................................... 19
3.2 Research tools ................................................................................................................... 20
3.2.1 Goal Management Training Program ................................................................... 20
3.2.2 Web-based platform .............................................................................................. 21
Adobe connect .................................................................................................................. 21
Caring for Me Website...................................................................................................... 22
3.3 Outcome Measures............................................................................................................ 22
3.3.1 Adherence to Protocol........................................................................................... 23
3.3.2 Survey ................................................................................................................... 24
3.3.3 Neuropsychological Measures .............................................................................. 24
3.3.4 Functional Outcome Measures ............................................................................. 25
3.4 Conclusion ........................................................................................................................ 27
Chapter 4 Web Goal Management Training: Online Cognitive Training .................................... 28
Chapter 5 Discussion .................................................................................................................... 45
5.1 Executive Function ........................................................................................................... 46
5.2 Efficacy of Web-Based Goal Management Training (WebGMT) ................................... 48
5.2.1 Limitations and Future Directions ........................................................................ 50
5.3 The importance of WebGMT............................................................................................ 50
5.4 Feasibility Web-Based Goal Management Training......................................................... 51
5.4.1 Limitations and Future Directions ........................................................................ 52
5.5 Summary and Conclusions ............................................................................................... 53
iv
Appendix A: Cognitive Rehabilitation Therapy Scale for Goal Management Training –
(CRTS-GMT) ........................................................................................................................... 64
Appendix B: Participant Feedback Survey ................................................................................... 74
Appendix C: WebGMT Groups 1 and 2 compared to In-Class GMT on CRTS-GMT scale ....... 76

v
 List of Tables
Table 3.1 Goal Management Training Module Descriptions ....................................................... 20
Table 3.2 Battery Quantitative Outcome Measures ...................................................................... 26
Table 4.1 Participant Characteristics ............................................................................................ 32
Table 4.2 WebGMT Group 1 CRTS-GMT scores of two raters across randomly selected session
recordings ...................................................................................................................................... 38
Table 4.3 WebGMT Group 2 CRTS-GMT scores of two raters across randomly selected session
recordings ...................................................................................................................................... 38
Table 4.4 Participant Feedback Survey ........................................................................................ 39
Table 4.5 DKEFS raw score significance using the reliable change index .................................. 40
Table 4.6 Neuropsychological tests of executive function raw score significance using students t-
test ................................................................................................................................................. 40
Table 4.7 Dysexecutive Questionnaire raw scores using students t-test ...................................... 40
Table 4.8 Goal Attainment Scale measure .................................................................................... 41

vi
 List of Figures
Figure 2.1 Flow chart to illustrate the five steps in goal management training ............................ 11
Figure 2.2 International Classification of Functioning model ...................................................... 17
Figure 2.3 Barriers cycle impeding treatment ............................................................................... 17
Figure 3.1 Screen Shot of Web-based Platform............................................................................ 22

vii
 List of Appendices

Appendix A: Cognitive Rehabilitation Therapy Scale for Goal Management Training –

(CRTS-GMT) ........................................................................................................................... 64
Appendix B: Participant Feedback Survey ................................................................................... 74
Appendix C: WebGMT Groups 1 and 2 compared to In-Class GMT on CRTS-GMT scale ....... 76

viii
 1

Chapter 1
Introduction
The main goal of this thesis was to investigate the feasibility and efficacy of delivering an
evidence-based cognitive rehabilitation intervention, namely Goal Management Training (GMT),
using a web-based platform administered to participants with executive dysfunction. GMT has
proven successful in improving cognitive abilities in patients with executive function
impairments (Levine, et al., 2007; Fish, et al., 2007; van Hooren, et al., 2007; Levine, et al.,
2011). It typically requires significant in-class, small-group attendance, placing time and travel
burdens on participants. While telephone and web-based delivery is common in Cognitive
Behavioural Therapy (CBT) and e-learning, to date, there are no published reports investigating
the use of real-time video conferencing platforms to deliver group-based cognitive rehabilitation
(Bergquist, Gehl, Holzworth, & Beaulieu, 2008; Machtmes & Asher, 2000). The web-based
GMT approach (WebGMT) presented in this thesis aims to increase accessibility to those who
are otherwise unable to receive treatment, and for the first time to deliver a group-based
cognitive rehabilitation intervention, establishing a novel foundation for the delivery of
therapeutic interventions via online platforms.

Accordingly, the feasibility of this innovative approach was established by developing a semi-
quantitative-qualitative tool to measure the adherence to GMT protocol and various aspects of
group engagement. This scale was devised from a CBT adherence-to-protocol scale (Blackburn,
et al., 2001) and was created to ensure no aspects of GMT protocol and therapist competence
were compromised in the web-based delivery. Moreover, the efficacy of the online intervention
was demonstrated using a battery of neuropsychological and functional outcome measures. Both
of these pieces of evidence supporting the WebGMT approach have the potential for significant
impact on the delivery of therapy, such as cognitive rehabilitation intervention, to a growing
population of patients in need. Our approach is the first of its kind to be delivered remotely for
group-based cognitive rehabilitation, with promising results that can greatly benefit patients
living in isolated communities, inaccessible to healthcare facilities and community services.

This thesis is organized in a manuscript format. Chapter 2 provides the background literature to
understand the theories forming the basis for GMT. Neural structures and cognitive processes
underlying executive function are also explained. In addition, the chapter provides a review on
 2

the web-based delivery platform in the fields of CBT, cognitive rehabilitation, e-learning and
social work. Finally, Chapter 2 ties all aspects of this research to the International Classification
of Functioning model for rehabilitation research, providing the reader with a holistic picture,
highlighting the importance of this line of research. Chapter 3 then details the research protocol
and participant characteristics. Chapter 4 is written in a manuscript format to be submitted for
publication to Neuropsychological Rehabilitation. This chapter explicitly portrays all aspects of
the research and presents the findings and future directions. Finally, Chapter 5 serves as a wrap-
up chapter, summarizing the research undertaken and tying in previous findings and future
implications discussed in Chapter 2 and Chapter 4.
 3

Chapter 2
Literature Review
Introduction
In this literature review, I provide an overview of frontal executive functions, including a brief
review of their neural basis and a survey of the most influential theoretical models characterizing
the cognitive processes and operations subsumed under the umbrella of executive functioning.
Frontal executive functions are a set of complex cognitive processes that allow an individual to
set and pursue goals and flexibly adapt to shifting contingencies (Levine & Turner, 2004). It is
an inclusive term that describes a collection of higher cognitive abilities, typically associated
with the frontal lobes and frontally-mediated systems (Levine, Turner, & Stuss, 2008). Cognitive
processes that have been associated with executive functions include planning, self-monitoring,
behavioural control, adapting to novel situations, complex problem solving, decision making and
the ability to sustain attention, as well as other skills such as maintaining continuous self-
awareness over time (Chan, Shum, Toulopoulou, & Chen, 2008; Levine, Turner, & Stuss, 2008;
Hart & Evans, 2006). Executive functions allow one to formulate goals, generate strategies,
arrange complicated actions and monitor behaviour all whilst flexibly dealing with new
information and inhibiting unwanted actions.

In this thesis project, frontal executive functions are conceptualized as those higher cognitive
functions necessary for goal-directed action. According to one of the earliest and most influential
models of executive functioning proposed by Norman and Shallice (1980) and later discussed in
this chapter, actions are guided either automatically, through established stimulus-response
behaviours, or in a controlled manner by engaging a supervisory attention system (SAS). The
SAS refers to mental operations called forward in novel situations or where contingencies have
shifted and automatic responses yield behavioural errors (Gilbert & Burgess, 2008). Supervisory
attention is critical for goal direction as it allows behaviour to progress towards a future goal
state by overcoming routinized behaviours (e.g. habits, environmentally dependent responses)
that may distract from or disrupt actions that would otherwise lead to goal attainment. Thus,
supervisory attention, or executive functioning, permits humans to transcend the here and now,
to overcome automatic and stimulus-driven responses and ultimately guides their actions
according to planned goals (Gilbert & Burgess, 2008).
 4

As reviewed below, frontal executive functions are reliant on the frontal lobes and frontally-
mediated brain systems. As such, they are particularly vulnerable to insults to the frontal lobes
directly (e.g. traumatic brain injury, frontal strokes) or to white matter connections linking
frontal brain regions to posterior and subcortical brain structures (e.g. multiple sclerosis, healthy
and pathological aging including ischemic white matter disease). Given the prevalence of these
etiologies, executive function deficits are among the most common neuropsychological
disorders. Their devastating impact, specifically on real-world functional abilities, has made
them an increasingly important target for rehabilitation interventions.

Here, I describe how models of executive function (and dysfunction), particularly the model of
goal-neglect (Duncan, 1986), has informed the development of a rehabilitation intervention, Goal
Management Training – GMT, that is showing considerable promise in treating executive
dysfunction. In the following sections, I examine executive dysfunction as seen through the lens
of stroke and small vessel disease and review how GMT has been used to remediate executive
function deficits in these and other neurological populations. Finally, I discuss the practical
challenges and barriers to access for delivering an intensive cognitive rehabilitation treatment
such as GMT to an outpatient neurological population and examine how advances in internet-
based, remote delivery technologies could reduce these barriers and increase patient access.

In sum, this literature review chapter provides an overview of frontal executive function and
dysfunction and how a theory-driven intervention (GMT) is showing considerable promise in
treating executive dysfunction. From this review, I describe how internet-based technologies are
providing new opportunities to enhance access to GMT. In addition, I relate this to the
International Classification of Functioning framework, providing a large picture in describing the
necessity of this research. In the chapters to follow, I describe the results of a novel study
examining both the feasibility and efficacy of an internet-based approach to delivering GMT
using a web-based, remote delivery protocol.

2.1 Frontal executive functions: neural systems

Mapping of executive functions to specific brain regions, namely the frontal and prefrontal lobes,
has been extensively investigated and has proven to be challenging (Stuss & Alexander, 2000).
However, damage to the frontal lobes and/or its connections to other brain regions, which can
result from acquired or progressive brain disease impacting cortical, subcortical and white matter
 5

brain structures, is strongly associated with diminished executive function. Specific impairments
include deficits in initiation, planning, sequencing, inhibition, mental flexibility and shifting
(Mattson & Levin, 1990; Alexander & Stuss, 2003). Efforts to map these brain changes have
allowed researchers to identify associations between behaviour and functional brain anatomy
(Stuss, 2006).

There is increasing evidence that frontal executive functions may be fractionated into four
distinct categories, each associated with functioning in different regions of the prefrontal cortex:
1) energization and regulation, mediated by the superior medial frontal cortex, 2) executive
cognitive functions, mediated by the lateral prefrontal cortex, 3) social, behaviour and emotional
self-regulation, mediated by the ventral medial prefrontal cortex and 4) metacognitive processes,
mediated by the frontal poles (Stuss, 2006). Much of this brain-behaviour mapping has emerged
from experimental neuropsychological research with brain-injured patients, and I briefly review
each of these in turn below.

Energization and activation refer to the processes associated with initializing and maintaining a
desired action (Levine, Turner, & Stuss, 2008; Stuss, 2006). Patients with damage to the superior
medial cortex demonstrate increased reaction times in timed tasks; that is, these patients take
longer in generating their response to a stimulus, and this can translate into apathy (Stuss &
Levine, 2002). The lateral prefrontal cortex, on the other hand, mediates higher executive and
cognitive functions, such as planning, monitoring, checking and inhibition (Stuss & Alexander,
2007; Stuss & Levine, 2002). These skills allow individuals to control lower-level cognitive
processes (Stuss & Levine, 2002). Behavioural self-regulatory functions localized in the ventral
medial prefrontal cortex are responsible for decision making underlined by reward processing
(Stuss, 2007). Patients with deficits in this area tend to perform poorly in unstructured situations;
their ability to regulate behaviour is hindered and unresponsive to internal goals and constraints
(Levine, 1999; Levine, Freedman, Dawson, Black, & Stuss, 1999). Finally, patients suffering
from damage to their frontal poles score poorly on experimental neuropsychological tests
focusing on humour appreciation and episodic memory (Stuss, 2007). This brain region linked to
metacognitive processes is responsible for self-reflection and awareness, as well as one’s
capability in making social judgments and showing empathy towards others (Stuss, 2007).
 6

While it is important to understand that executive functions are comprised of multiple cognitive
processes, each localized to specific frontal brain regions – and each a potential target for
rehabilitation – one of the central focuses of this thesis project is on ‘executive cognition and its
role in goal-directed action’. Executive or ‘goal-directed’ control of behaviour has been the
subject of extensive study in the neuropsychological and cognitive science literature. In the next
section, we review three of the most influential theoretical models of goal-directed cognition and
examine how these models have informed the development of the rehabilitation protocol used in
the thesis research.

2.2 Models of Frontal Executive Functions

Several models of executive functions have been proposed over the past few decades, all of
which provide this field with a better understanding of symptom patterns occurring after frontal
lobe damage. These theories serve as potential frameworks with general principles that guide
rehabilitation through experimental research evidence. Here, I present three theories to provide
further understanding of the cognitive skills involved in executive function and how these may
inform approaches to remediating executive deficits. A critical distinction amongst the theories
presented here is in their conceptualization of executive functions as a unitary versus a
fractionatable, multi-process approach. Multiple-process theories are presented to demonstrate
how the different components of executive function are thought to be independently impaired in
brain disease and thus present potential specific rehabilitation targets. In contrast, this thesis
focuses mainly on Duncan’s single-process theory, whereby damage to a single cognitive
process (i.e., goal-directed action) is considered to be the cause of dysexecutive symptoms and
thus should be the central intervention target (Burgess & Robertson, 2002). Duncan’s theory of
goal neglect is emphasized here due to its direct link to the cognitive rehabilitation intervention
under study (Goal Management Training, see below).

2.2.1 The Supervisory Attentional System SAS (Shallice & Norman,

1980)
As described briefly in the introduction, Norman and Shallice (1980) modeled executive
functioning in terms of the variable attention demands necessary to control action. They
hypothesized two types of situations that trigger action: routine situations in which a response is
automatic and hence no attention is required to establish or maintain it, and novel situations
 7

which require the formation of a new schema, monitoring/supervising, and rejection or

acceptance of the novel schema to achieve adaptive action. The former generates actions through
what is referred to as the contention scheduling mechanism, while the latter uses the supervisory
attentional system (SAS) (Norman & Shallice, 1980). Contention scheduling is a mechanism by
which a specific action schema is selected to produce behaviour from a number of competing
schemas. The selected schema guides action, whereas alternate competing schemas are inhibited
because of their lower activation value (Turner & Levine, 2008).

On the other hand, in novel situations the SAS is engaged. This mechanism is brought about and
activated under one of the following circumstances: if the situation is novel, if the action requires
additional planning or correction of action, and, finally, if the automatic action needs to be
suppressed (Norman & Shallice, 1980). In such situations, there are no precedent schemas that
can be used by way of contention scheduling; therefore, the SAS activates the process of creating
a new schema to result in an appropriate action. Patients with damage to areas of the frontal lobe
provide evidence supporting this model. Their behaviour shows an increase in either
distractibility or perseverations, which seem to be opposing tendencies within this model of
executive function; however, both of these deficits could result from a non-functional SAS.
Where the activation of a certain schema is much greater than that of other schemas and
therefore overrides them and causes perseveration. On the other hand, when a number of
schemas have similar activation strengths, they cause distractibility (Norman & Shallice, 1980).

Overall, this multiple-process model proposes that damage to discrete regions of the frontal lobe
will have variable impact on the operation of component processes within the SAS – e.g. poor
inhibition of maladaptive schemas, poor activation of relevant schemas, or poor action
monitoring (Shallice & Burgess, 1996). Reduced capacity of the SAS increases the likelihood of
maladaptive, routine, stimulus-bound, or off-task behaviours. Treatments to enhance executive
control functioning according to this model would therefore emphasize strengthening the SAS by
targeting specific processes (inhibition, monitoring) where deficits are most pronounced– which
ties in nicely with fractionation of processes discussed above (Stuss, 2006).

2.2.2 Working Memory: Central Executive System

In 1986, following up on his work with Hitch in 1974, Baddeley introduced a new concept of
working memory, which comprised initially three and later four components: the central
 8

executive, the phonological loop, the visuospatial sketchpad and the episodic buffer. Based on
animal studies, this model demonstrates a helpful representation of executive function processes
in the human brain (Turner & Levine, 2008). The idea is that two “slave” systems, namely the
phonological loop and the visuospatial sketchpad, located in the posterior cortices, support the
central executive, located in the prefrontal cortex (Baddeley, 1986). These two slave systems act
to manipulate and store linguistic, visuospatial and visuoperceptual information. This
information is then organized and brought to bear on current actions through the operation of the
central executive (Baddeley & Hitch, 2000). The construct of the central executive draws heavily
on the earlier conceptualization of a SAS (Norman & Shallice, 1980). In this respect, the central
executive is an attentional control system, protecting the operation and contents of the slave
systems from distraction and coordinating the flexible entry and exit of information from these
systems based on current goal states. The central executive system (CES) was later fractionated
to include the episodic buffer as a bridge between the slave systems supporting online mnemonic
operation and long term memory representations (Baddeley & Hitch, 2000).

While similar to the Norman and Shallice (1980) conceptualization of the SAS, the CES in
Baddeley’s model represents a somewhat more unitary account of executive functioning. In this
model, the integrity of the central executive of the working memory system is a critical
determinant of executive function capacity. Thus, improving working memory and the operation
of the central executive more specifically (e.g. preserving the integrity of online information in
the context of distractions) would be a critical intervention strategy according to Baddeley’s
model of executive functioning.

2.2.3 Goal Selection and Goal Neglect (Duncan, 1986)

This model follows from Duncan’s observation that patients with frontal lobe damage show a
discrepancy between goals and actions, which he refers to as the theory of disorganized
behaviour. When presenting his model, Duncan (1986) begins by examining problem-solving
processes in both humans and machines to determine key features of the series of complex
processes undertaken by the healthy brain in executing higher cognitive skills. Firstly, Duncan
proposes that goal lists are an important prerequisite (e.g. the car must be parked), then he
requires action structures, which he describes as information-processing sub-components
composed of actions and mental operations in order to monitor and maintain goals to be achieved
 9

(e.g. pushing the brake hard to avoid hitting an object). Once the set goal is achieved, a means-
end analysis must follow to determine to what degree the achieved goal matches the initial
desired goal (e.g. is the car parked within the parking lines?) (Duncan, 1986).

This theory overlaps somewhat with Baddeley’s working memory model as he characterizes
patients with executive dysfunction as being incapable of maintaining goals and goal lists on-line
using working memory (Burgess & Robertson, 2002). However, unlike Baddeley’s model where
working memory is the central feature of executive functioning, Duncan emphasizes the link
between intention and action as being a central feature of executive control. Indeed, he defines
executive dysfunction as “disregard of a task requirement even though it has been understood
and remembered” (Duncan & Stuss, 2000). For any behaviour to occur, relevant information
from the environment, required action plans and rules must be organized and mapped out in a
structured program that Duncan and his colleagues (2008) refer to as a task model. In their study
investigating the competing parts of a complex task, they show that the more complex a task
model is (i.e., the more facts, rules and requirements), the more likely it is that goal neglect will
be exhibited (Duncan, et al., 2008). Interestingly, neglect is sensitive to the mere presence of
additional knowledge, even when there is a direct instruction to disregard its content (Duncan, et
al., 2008). However, this is the case only when the task model is relatively new; the more
familiar and practiced the task is the less effect its complexity has on neglect (Duncan, Emslie,
Williams, Johnson, & Freer, 1996; Duncan, et al., 2008).

Duncan and his colleagues (2008) attribute this relationship between goal neglect and the
increase in number of task components to attentional mechanisms. Unlike Baddeley’s central
executive, attentional control in this model is not affected by real-time stimulus processing or by
increasing working memory load. Rather, it is directly affected by the complexity and novelty of
task instructions (Duncan, et al., 2008). Thus, goal-directed action is highly vulnerable to
increases in task requirements and in the knowledge that must be organized into a coherent task
model to guide behaviour (Duncan, Emslie, Williams, Johnson, & Freer, 1996; Duncan, et al.,
2008). This explains why goal neglect (i.e., executive dysfunction) is so obviously manifested in
complex real-world settings where constraints are few, contingencies are high and novelty and
demand for flexibility are constantly present.
 10

Duncan’s theory of goal-directed action and goal neglect defines executive function as the
guidance of action by its intended outcome. Thus, goal setting and goal attainment are key
rehabilitation targets in this model. This conceptualization of executive function as linking
intention and action provides the theoretical foundation for Goal Management Training, the
executive function rehabilitation program utilized in this thesis project.

2.3 Goal Management Training

While the number of cognitive rehabilitation interventions specifically targeting executive
dysfunction have been increasing, the quality of evidence supporting the efficacy of these
interventions remains low (Levine & Turner, 2004; Rees, Marshall, Hartridge, Mackie, Weiser,
& Erabi Group, 2007; Cicerone, et al., 2011). In examining cognitive rehabilitation research
between 1971 and 2008, Cicerone and his colleagues identified metacognitive strategy training,
including self-monitoring and self-regulation, as an effective intervention technique for people
suffering from executive dysfunction following traumatic brain injury (TBI; Cicerone, et al.,
2011). However, they pointed out that the efficacy and underlying mechanisms of cognitive
rehabilitation interventions need to be further investigated (Cicerone, et al., 2011). In a
systematic review of executive function interventions following TBI, Kennedy and her
colleagues (2008) examined 15 studies and suggested that areas of future research should focus
on efficacy of specific intervention techniques, participant co-morbid characteristics and training
requirements for delivering interventions and setting goals (Kennedy, et al., 2008). Goal
Management Training (GMT) is one intervention that has demonstrated efficacy in improving
executive functions in acquired brain injury (TBI & stroke) and aging populations (Levine, et al.,
2000; Fish, et al., 2007; Levine, et al., 2011; Levine, et al., 2007).

Goal Management Training is a cognitive rehabilitation protocol developed by Robertson et al.

(1996), which introduces goal lists and divides tasks into small subtasks in an attempt to
remediate executive function in people with frontal lobe damage. Basing their intervention on
Robertson’s research (1996) and Duncan’s model of goal neglect (1986), as well as Shallice and
Norman’s SAS model (1986), Levine and his colleagues (2011) conducted studies with stroke
patients with executive function deficits. They implemented Robertson’s GMT protocol, where
nine modules are covered across a five week period. In this protocol, participants are expected to
attend a total of 20 hours of in-class instruction and commit to doing 20 hours of assigned
 11

homework based on the material taught. The nine modules consist of material that teaches formal
problem-solving techniques, which are employed to define a goal, make a plan for its attainment,
and monitor progress. Drawing from Norman and Shallice’s early work (1980), GMT teaches
patients to use a stop-state-split strategy, whereby they are trained to recognize and leave the
automatic mode governing their behaviour and to re-establish executive control by stopping their
ongoing action, stating their goals, creating sub-goals and monitoring their progress towards
those goals. They are also taught to check their goals vis-à-vis their actual outcomes in a means-
ends analysis to determine whether corrective action is necessary to maintain progress towards
the desired goal state. Figure 2.1 displays the five stages of GMT taught across the nine modules
(Levine, et al., 2000; Levine, et al., 2007).

Figure 2.1 Flow chart to illustrate the five steps in goal management training (Levine, et al.,
2000)
 12

2.4 Executive Dysfunction, Stroke and Cerebrovascular Disease

Stroke and other types of cerebrovascular disease (CVD) are common sequelae of vascular risk
factors such as hypertension, sedentary lifestyle, genetic factors, etc. Overt stroke occurs in
50,000 Canadians every year (Heart and Stroke Foundation, 2012). Canada has a population of
300,000 survivors living with the aftermath of overt stroke, 64% of whom suffer from cognitive
impairment, compared to 21% of stroke-free elderly (Heart and Stroke Foundation, 2012; Jin,
Legge, Ostbye, Feightner, & Hachinski, 2006). CVD and its risk factors, one of the most
reported of which is hypertension, are highly associated with cognitive decline, referred to as
vascular cognitive impairment (VCI) (Verghese, Wang, Katz, Sanders, & Lipton, 2009; Black S.
E., 2007). VCI is often characterized by executive dysfunction (Verghese, Wang, Katz, Sanders,
& Lipton, 2009; Hachinski, et al., 2006; Gorelick, et al., 2011). Executive dysfunction can also
be associated with signs of white matter small vessel disease, manifested as focal or confluent
silent lacunar infarcts on computerized tomography or magnetic resonance imaging (Black, Gao,
& Bilbao, 2009). Critically, small vessel or ischemic white matter disease is estimated to be 10-
20 times as prevalent as large-vessel stroke (Vermeer, Prins, den Heijer, Hofman, Koudstaal, &
Breteler, 2003; Black S. E., 2007), suggesting VCI may be an emerging public health challenge.
Given the prevalence of executive dysfunction in this population, access to evidence-based
rehabilitation treatments such as GMT is extremely important. However, as described in the
GMT protocol section above, the intervention places significant burdens on participants in terms
of in-class time along with concomitant travel burdens and related costs. Indeed, for populations
suffering from brain disease and cognitive impairment, such travel coordination and costs may
be prohibitive. To address these issues, the goal of this thesis project was to improve access to
this treatment option through the use of innovative remote delivery technology.

2.5 Web-Based Delivery Platforms

2.5.1 E-Education
The advantages of remote delivery technologies have not yet been fully realized in the delivery
of group-based cognitive rehabilitation, even though technical progress has made such distance
learning and rehabilitation opportunities possible. Using video-conferencing as an alternative to
traditional classroom instruction has become more widespread in corporations and universities
(Moore & Kearsley, 1996; Webster & Hackley, 1997; Sugrue, 2003). A meta-analysis of distant
 13

learning techniques identified three instructional features that impact distant learning levels by
students (Machtmes & Asher, 2000).These three features were interaction available during e-
learning delivery, type of course delivered and the type of remote site. Two-way communication
provided by video-conferencing platforms allows the highest amount of interaction between
instructor and student, resulting in greater student achievement. In addition, the type of course
and type of remote site greatly impact student motivation. For instance, a course required by an
employer and delivered in a workplace environment showed the highest effect size; learners
were motivated to do well because their company was covering the expense, which in turn
affected their learning outcomes (Machtmes & Asher, 2000). Brown and his colleagues (2005)
also identified two effects on training outcomes: class size and instructor immediacy. Instructor
immediacy refers to the communication style of the instructor; the more engaging the interaction
between the instructor and students, the higher the learning (Brown, Rietz, & Sugrue, 2005). The
recommendations made to improve learning and motivation were to boost interaction and for the
instructor and students to get to know one another outside of class time (Brown, Rietz, & Sugrue,
2005). It has also been demonstrated in prior research that when the number of attendees
increases in video-conferencing classes , the students feel disconnected, therefore negatively
impacting the learning experience (Gowan & Downs, 1994).

2.5.2 Remote delivery of Cognitive Behavioural Therapy and cognitive

rehabilitation
Cognitive behavioural therapy (CBT) as well as cognitive rehabilitation have used several
platforms of remote delivery such as telepractice, off-line web-based and, in recent years, real-
time videoconferencing delivery modes (Bouchard, et al., 2004; Ruwaard, Broeksteeg,
Schrieken, Emmelkamp, & Lange, 2010; Ruwaard, et al., 2009; Bergquist, Gehl, Holzworth, &
Beaulieu, 2008; Frueh, Monnier, Yim, Grubaugh, Hamner, & Knapp, 2007; Germain, Marchand,
Bouchard, Drouin, & Guay, 2009; Morland, Pierce, & Wong, 2004; Man, Soong, Hui Chan, &
Hui Chan, 2006; Arundine, Bradbury, Dupuis, Dawson, Ruttan, & Green, 2012). CBT is a
psychotherapeutic approach to solving maladaptive behaviour in which the person’s thoughts or
cognitive patterns are targeted in order to change behaviour (Alford & Beck, 1997).The therapist
works with the client as a pair to overcome the client’s problems. Studies comparing the
administration of CBT face-to-face and via a web-based videoconferencing platform with
various patients suffering from post-traumatic stress disorder (Frueh, Monnier, Yim, Grubaugh,
 14

Hamner, & Knapp, 2007; Germain, Marchand, Bouchard, Drouin, & Guay, 2009; Morland,
Pierce, & Wong, 2004) and panic attacks (Bouchard, et al., 2004) demonstrated no differences in
therapeutic outcomes between these two delivery modes. For instance, Bouchard et al., 2004
compared a group of patients receiving CBT for panic attacks caused by agoraphobia in either a
face-to-face traditional method of delivery or in a videoconferencing delivery mode. The results
showed significant improvements in panic symptoms, panic-related characteristics, agoraphobic
avoidance, general anxiety, depressive affect, and general functioning (Bouchard, et al., 2004).
In their study, Germain et al., (2009) administered telepsychotherapy and face-to-face CBT to
patients with post-traumatic stress disorder and found improvements in both groups. They also
found via administered questionnaires that technical problems, the initial perception of therapy
by videoconference and the sense of presence felt during videoconferencing had no impact on
treatment outcome (Germain, Marchand, Bouchard, Drouin, & Guay, 2009). Another form of
therapy that has also utilized this novel technology is cognitive rehabilitation, where various
modes of delivery (telepractice, traditional in-class and web-based modes) of problem-solving
skill training were used on acquired brain injury patients. Results showed significance in efficacy
of treatment regardless of the mode of delivery across all groups (Man, Soong, Hui Chan, & Hui
Chan, 2006). All of these studies dealt with single therapist-patient interactions, whether face-to-
face or via on-line psychotherapy. In other words, none of them investigated the efficacy of
therapy in group settings, which is the goal of this study.

2.5.3 Caregiver Support Groups

Marziali and colleagues (Marziali, Damianakis, & Donahue, 2006) have demonstrated the
efficacy of therapeutic group interventions in caregivers using an online video-conferencing
delivery method. These studies have compared web-based caregiver groups and traditional face-
to-face psychosocial groups using various methods of qualitative analysis (Marziali & Donahue,
2006; Marziali, 2006; Marziali, Damianakis, & Donahue, 2006; Marziali, 2008; Damianakis,
Climans, & Marziali, 2008). Results show that older adults can be trained to effectively use a
computer and access websites leading to videoconferencing platforms (Marziali, 2009). In
follow-up interviews, participants reported high levels of involvement in the web-based groups
(Marziali, 2008), strong bonding between group members and group members and facilitators
(Marziali, 2006), as well as comfort and preference for using videoconferencing platforms
(Marziali, 2009). Given that in some studies up to 50% of the participants had never used a
 15

computer, participants in one study were asked about the difficulty of using a computer after
sufficient training, 78% reported that the website was easy and 95% responded with positive
feedback to online meetings; after a six month follow-up, 90% of caregiver group members
reported extremely or very positive benefits from their online support groups (Marziali &
Donahue, 2006). In addition, one study collected therapist feedback in response to using
videoconferencing platforms for meeting with clients. The results showed that therapists needed
to reflect on their subjective responses to transitioning intervention skills from a face-to-face
environment to online video conferencing with a group of clients. For some, several iterations of
working online were needed to gain maximum comfort levels for using technology to
communicate with clients (Damianakis, Climans, & Marziali, 2008; Marziali, Damianakis, &
Donahue, 2006). With such robust findings, this thesis draws on the caregiver support group
success, as well as the CBT individual delivery successes, and uses online platforms to
investigate the efficacy and effectiveness of delivering GMT by remote real-time video-
conference.

2.6 International Classification of Functioning

The following section uses the International Classification of Functioning (ICF) framework to
support the project’s approach to using technology to deliver a cognitive rehabilitation program
(WHO, 2001). Rehabilitation is a continuous process that entails classification of a disease and
related factors in the environment, followed by successful administration of rehabilitation
interventions (Stucki, Ewert, & Cieza, 2002). This thesis project is aimed at focusing on the
selection of successful methods of delivery, appropriate to the given disease and environment. It
has potential to significantly improve access to cognitive rehabilitation for clients facing travel,
time, financial, mobility or other barriers that restrict in-person participation in cognitive
rehabilitation therapy, making it an easily translated research program.

Within the rehabilitation field, it is essential to have an organized framework that can be utilized
in research and clinical practices. For this reason, the World Health Organization created a
standard framework called the International Classification of Functioning, Disability and Health
(ICF) (Stucki, Ewert, & Cieza, 2002), which provides a standard language for all health-related
research, from both individual and societal perspectives (Vrankrijker, 2003). It incorporates the
health condition, which reflects body functions and structures, in relation to daily activities and
 16

participation in society (Vrankrijker, 2003). It also considers the environmental and personal
factors that may play an important role in disability. The research I am conducting fits within the
ICF framework (shown in Figure 2.2) on several levels.

The approach (i.e., web-based delivery) takes into consideration health, environmental and
personal factors and as such attempts to bridge gaps among these three factors to decrease
barriers that typically reduce access to people requiring therapy. For example, a stroke patient
suffering from executive dysfunction, within the ICF, has a cognitive impairment arising from
brain network dysfunction, which in turn interferes with goal-directed activities and contributes
to restriction of successful participation in the society. The result of the impairment and
restrictions posed through contextual factors affect the ability to comply with treatment to
compensate or restore function arising from the brain pathway, creating a cycle that is difficult to
break (see Figure 2.3). In other words, decreased access to the treatment of the cognitive
impairment in this case further exacerbates the impairment. By providing alternative means in
accessing treatment, in this case cognitive rehabilitation, we are overcoming barriers arising from
the patient’s impairment and pre-existing environmental obstacles (such as geographical barriers)
to work around the patient’s personal circumstance in order to provide rehabilitation.

This model can also be beneficial in guiding the selection of outcome measures for analyzing
efficacy of treatment. For instance, functional outcome measures (discussed in Chapter 3), which
assess everyday functions and examine the participation aspect of a disability, shed light on the
direct benefits an intervention has on the individual’s life. On the other hand, neuropsychological
measures examine the effects of an impairment of body functions and structure on activity,
providing us with a method of assessing limitations placed on the individual (Bilbao, et al.,
2003).

Moreover, specific GMT material incorporates all branches of this model. Each module
(described in Chapter 3) introduces and teaches concepts explaining the health condition (i.e.,
executive dysfunction) and its effects on activities and participation (e.g. failure to plan a
vacation), and also draws on examples from the participants’ lives, incorporating personal and
environmental factors that may benefit in the training. In conclusion, the ICF model serves as a
point of reference in which the interaction of various aspects of a disability are analyzed and
whereby rehabilitation is tailored accordingly.
 17

Figure 2.2 International Classification of Functioning model (WHO, 2001)

Figure 2.3 Barriers cycle impeding treatment

 18

2.7 Conclusion
This chapter reviewed the literature on executive function, executive dysfunction in the stroke
population, and Goal Management Training, as well as provided the background on the research
done in delivering various therapeutic and educational materials through the World Wide Web.
The rationale behind using the ICF, an international model of rehabilitation, for this project has
been discussed. The next several chapters describe the specific aspects of my Master’s research
(Chapter 3- Methods, Chapter 4- Manuscript format of study results and Chapter 5- Discussion)
which investigates remote delivery of GMT to remediate executive function deficits in a stroke
and small vessel disease population.
 19

Chapter 3
Research Plan and Protocol

3.1 Background
GMT is an evidence-based cognitive rehabilitation program aimed at training cognitive
processes involved in executive functions (Levine, et al., 2007). As mentioned in Chapter 2,
GMT participation involves a significant commitment, both in time and travel demands. This
level of commitment creates potential barriers to participation, particularly for those who live in
more remote settings where transit options are limited and travel times might be prohibitive for
attendance at twice weekly group meetings. Moreover, if participants are confronting any
physical disability that might limit independent movement, time and travel burdens would be
exponentially greater.

In order to address the accessibility issue linked to patient participation in this specific cognitive
rehabilitation intervention, we have adopted the on-line novel delivery platform used in “Caring
for Me” psychosocial groups (Marziali, 2006; Marziali & Donahue, 2006; Marziali, Damianakis,
& Donahue, 2006). “Caring for Me” groups have proven successful in connecting caregivers
regardless of location. The experience was deemed highly engaging for individuals who are
otherwise unable to meet other caregivers and therapists (Marziali, 2008). While
psychotherapeutic interventions (e.g. cognitive behavioral therapy- CBT) are increasingly taking
advantage of remote delivery methods, for the most part, these have consisted of one-on-one
therapy sessions or web-facilitated interactions not occurring in real time (Bouchard, et al.,
2004). To date, with the exception of Marziali’s (2006-2008), there are no reports in the
published literature demonstrating the use of web-based methods to facilitate real-time, group
therapy interventions. Here, I describe a research project to investigate whether GMT can be
delivered via the web without compromising therapeutic efficacy. Specifically the study asks two
core questions: (i) Can GMT be delivered effectively over the web – i.e., is it feasible? (ii) Does
web-delivery impact the therapeutic efficacy of the intervention?
 20

3.2 Research tools

3.2.1 Goal Management Training Program
GMT is facilitated by a psychologist or an occupational therapist specifically trained to
administer the intervention using a structured manual (Levine, Manly, & Robertson, 2008). It is
typically carried out through a five week time period in which a maximum of four participants
meet with the facilitator twice a week for two hours to complete a total of 20 hours of training
material. In addition to the in-class therapy, the program requires 20 hours of at-home assigned
work. Every two hour session covers various concepts, strategies, and real life examples to help
participants in recognizing and developing solutions to their day-to-day problems of attention,
problem solving and goal achievement (Table 3.1). Packages consisting of GMT materials that
are required throughout the training program were provided to all participants prior to start of
intervention.

Table 3.1 Goal Management Training Module Descriptions

Module Concepts & Strategies

1: The absent mind, the present mind Introduction of goal hierarchies, the mental
laboratory, absentmindedness and presentmindedness

2: Absentminded slip-ups Relation of absentmindedness to other abilities,

consequences of slips, conditions for slips how Goal
Management Training will reduce slips.
3: The automatic pilot The automatic pilot and how it leads to errors in
every day functioning

4: The automatic pilot Training to stop the automatic pilot

5: The Mental Blackboard Mental blackboard (i.e., working memory)

6: State your goal Getting side tracked from your goal (i.e., distractions)

7: Making Decisions Goal conflict and decision making

8: Splitting Tasks into Subtasks Dealing with overwhelming tasks by splitting them
into smaller tasks
9: Checking (“STOP!”) Checking (reducing slip-ups)

10: Wrap-up and feedback Review of material and taking up ongoing goals

Throughout the GMT program, participants are presented with module content explaining the
concepts and strategies mentioned above, encouraging them to share real life experiences and
 21

perform interactive tasks. During a single session, various homework assignments are also
assigned. For instance, session two covers concepts such as absentmindedness in relation to other
abilities, consequences of slips, conditions for slips, and how GMT will reduce slips. In covering
this module, a task is performed where participants are presented with fruit names on the screen
appearing one at a time. After the participants see the fruit name, they are requested to clap
unless they see the word ‘pear’. This is done to demonstrate how the automatic pilot works and
how slips occur. In addition to this activity, participants are provided with worksheets (included
within their package) to record everyday slips that happen during their week before the next
session. This is done to encourage participants to recognize when they are absentminded and to
self-assess the causes of their behaviour. Therefore, each module presents concepts and covers
strategies, involves homework assignments as well as engages all participants in interactive tasks
or group discussions (see Appendix B for module outlines).

3.2.2 Web-based platform

Adobe connect
Adobe connect is an online meeting platform used to connect multiple users through a real-time
video conference. It has numerous features, including video sharing, audio sharing, screen
sharing, slideshow presentations, single or multiple presenter options, chatting and notepad note
taking options as well as recording capabilities. As many as six users may join a meeting with no
lagging problems. Administrators purchase the license to use this platform and may then create
password protected user accounts for members to join online meetings. All recorded sessions are
then uploaded onto a secure server where only administrators can access video files. Below is a
meeting screen shot labeled with all features used in our study (Fig. 3.1).
 22

Figure 3.1 Screen Shot of Web-based Platform

Caring for Me Website

In order to schedule and connect users to Adobe Connect, the Caring for Me website, adapted
from Dr. Marzialis’s work on caregiver support group studies, was used. This website acts as a
gateway between users and the meeting. Once the users are logged on, they have access to a
plethora of resources, from logging their food intake to accessing specific links provided to them
by their therapist. In this study, however, the website was strictly used to schedule and attend
meetings. To insure security of the web site users and content, the web site is accessible only
through the use of unique passwords for each user. Consequently the web site is not available in
the public Internet domain. Within the web site there are three levels of security; the site
administrator has access to all links and user groups, the professional facilitators have access to
other facilitators but only to the members within the groups that they facilitate, and the
participants have access only to the members within their group. All e-mail messages and video
conferencing sessions are encrypted and stored on a secure server for subsequent analysis.

3.3 Outcome Measures

The objectives of this research were a) to determine whether GMT can be delivered using the
web-based platform described above (WebGMT) and b) whether WebGMT is as effective on-
line as it has been in class (Levine, et al., 2011). To address each of these questions, two
approaches to collecting data were carried out; Approach 1- focusing on the feasibility of the
 23

WebGMT, determined using qualitative (semi-quantitative) measures, and Approach 2- focusing

on the efficacy of the web-based intervention, determined through quantitative
neuropsychological outcome measures, self-report measures and functional outcome measures.

Qualitative Measures
Two main tools were used to qualitatively analyze the web-based delivery mode: an adherence-
to-protocol scale and a survey collected from all participants reflecting their learning experience
using the technology.

3.3.1 Adherence to Protocol

The adherence-to-protocol scale was adapted from an existing revised cognitive behavioral
therapy scale (CTS-R) originally devised to assess competence in cognitive therapy (Blackburn,
et al., 2001). The CTS-R consists of 12 items, some of which are labeled general items as they
are not CBT-specific, such as feedback, collaboration, efficient use of time and interpersonal
effectiveness. Other items are labeled cognitive therapy-specific items, such as agenda setting
and adherence, eliciting appropriate emotional expression, eliciting key cognitions and more. For
the purposes of our study, and in order to assess GMT specific techniques, the CBT specific
items were either eliminated or revised and GMT theoretic bases were incorporated in the items
added. The scale was transformed from a 12-item CBT scale to a 9-item GMT scale. As with the
CTS-R, each item is rated across a six-level model of competence adapted from the Dreyfus
level of competence model using a seven-point Likert scale ranging from 0 - where the therapist
did not adhere to that aspect of therapy to 6- where there is adherence at very high skill (Dreyfus,
1989). For scoring, the maximum score on the CTS-R is 72 (that is, 12 items receiving a score of
6). To determine a minimum level of competence, the Newcastle Cognitive Therapy Centre set
the standard of 36, making it an average of three marks per item (James , Blackburn, & Reichelt,
2001). Following the same logic, for the devised 9-item scale of GMT, the maximum score is 54
and the minimum level of competence is 27, for an average of three marks per item.

The CTS-R scale with the above mentioned revisions was then named The Cognitive
Rehabilitation Therapy Scale for Goal Management Training (CRTS-GMT) and was completed
by two raters in a sample of the sessions, specifically three to five out of the 10 recorded
 24

sessions, of the web-based delivery program. Attached in Appendix C is the complete CRTS-
GMT with specific features for each item.

3.3.2 Survey
The short survey created acts as a self-report tool devised to tap into participant experience using
the online technology. The survey consists of four closed-ended questions asking about the
participants level of comfort with the technology, their benefit from the technical support
provided, their opinion on how helpful the web-based platform was in the learning experience, as
well as whether given the choice in the future they would prefer an online to an in-class
participation. In addition, we included an open-ended feedback section where participants
provided any other comments they had on the overall experience. This qualitative tool allows us
to gather data reflecting how useful the participants found the web-based delivery mode as well
as suggested changes that can be implemented in the future. Attached in Appendix B is the short
survey described above.

Quantitative Measures

3.3.3 Neuropsychological Measures

The efficacy of treatment was assessed via the administration of these tests immediately before
intervention and immediately after for pre and post cognitive changes. All assessments were
administered by a trained psychometrist.

The neuropsychological battery included standardized tests such as trail making A and B to
assess visual attention and task switching, digit span for attention and verbal working memory
(Wechsler, 1985), auditory consonant trigrams for verbal working memory and divided attention
(Brown, 1958) as well as self-ordered pointing task for planning and working memory (West,
Ergis, Winocur, & Saint-Cyr, 1998). To assess frontal cognitive processes such as planning,
problem solving, learning and maintenance of instructional sets, two versions of the tower task
were used in this study to eliminate practice effects by alternating items from the original task
(Delis, Kaplan, & Kramer, 2001; Levine, et al., 2011). Moreover, some tests from the Delis-
Kaplan Executive Function System (DKEFS) were used: tests to assess verbal fluency,
examining frontal and temporal regions; design fluency tests to assess fluency in the spatial
domain; and an additional measure of attention, inhibition and switching using the Stroop test
 25

(Delis, Kaplan, & Kramer, 2001). All the neuropsychological tests were analyzed using a paired
t-test of raw scores. In addition, the DKEFS tests were analyzed using an extension to the
reliable change index (RCI). The RCI is defined as the change in a client’s score divided by the
standard error of the difference for the tests, yielding a standardized score representing the
change observed (Jacobson & Truax, 1991). Additionally, the extension to the RCI takes into
account the practice effects (i.e., improvement in a test solely due previous exposure and not the
intervention) by incorporating a constant based on the group-level average change (Parsons,
Notebaert, Shields, & Guskiewicz, 2009), yielding an RCI “corrected” for practice effects
(Heaton, et al., 2001). The DKEFS tests are the only measures in which the reliability values
were available and thus were the only ones analyzed using the RCI.

3.3.4 Functional Outcome Measures

Self-reported changes were determined using questionnaires collected from patients and their
significant others. The Dysexecutive Questionnaire (DEX) was included as a 20-item
questionnaire designed to assess changes in inhibition, intention, personality, cognition and
behavioural changes (Burgess, Alderman, Evans, & Emslie, 1996). A patient-specific outcome
measure known as goal attainment scaling (GAS) was used as a measure of clinically important
change in cognitive rehabilitation (Rockwood, Joyce, & Stolee, 1997). For the GAS measure,
each of the four participants of the second WebGMT group met with the psychometrist in the
pre-assessment stage to formulate a minimum of one goal that was individually identified to suit
their level of performance. These goals were set such that the participants worked towards them
throughout the five-week training session independent of the GMT content. For each goal, the
psychometrist along with the participant determined level of achievement across a 5-point scale
ranging from -2 to +2. At the end of this session, the goals were reviewed with the participant
again and they were asked to work on them for the next five weeks. At the post-assessment
session the achievement level was determined, if the participant reported expected level of
outcome, they were give a score of 0, if they reported a somewhat less level of outcome, they
were given a score of -1 and if they reported much less level of outcome, they received a score of
-2. Consequently, if the participant reported a somewhat more than expected level of outcome,
they received a score of +1 and if they reported a much more than expected level of outcome,
they received a score of +2. Table 3.2 lists all outcome measures and provides a brief description
of each.
 26

Table 3.2 Battery Quantitative Outcome Measures

Measure Assessment Domain Description
Neuropsychological Trails A Visual attention Test has 25 scattered dots on a piece of
Assessments paper. The dots have numbers and subjects
are asked to connect the numbers in
sequence as accurately and quickly as they
can.
Trails B Visual attention and The second part to Trails A. Trails B has 25
task switching dots with numbers and letters scattered. The
subject is instructed to alternate between a
number and a letter connecting them in
sequence as quickly and accurately as they
can.
Consonant visual search speed, Subjects listen to a string of three
Trigrams scanning, speed of consonants immediately followed by a
processing, mental number. The subject is asked to count
flexibility, and task backwards in threes until instructed to recall
switching the consonant trigram.
Verbal Fluency Verbal fluency (frontal Phonemic Fluency: subjects are instructed to
and temporal regions) say as many words as they can that begin
with specific letters within a given time (i.e.,
as many words as you can beginning with
the letter ‘c’).
Semantic Fluency: subjects are instructed to
say as many words from a category as they
can within a given time (i.e., as many
animals as you can name within 60 seconds)
Design Fluency Verbal fluency in Subjects are provided with a paper with
spatial domain boxes. Each box contains dots. Subjects are
instructed to connect four dots in each box
making each line touch at least one other
line at a dot. Subjects are instructed to make
those designs working as quickly as they can
and by not making the same design more
than once. In other parts subjects are also
provided with distractor dots and asked to
ignore them or alternate between filled and
empty dots.
Stroop Inhibition and attention Subjects are presented with names of
colours printed in a different ink colour.
Subjects are asked to name the colour of the
ink the word is printed in and not the name
denoted by the word.
Tests of Everyday Selective attention, Subjects are asked to perform a task that is
Attention sustained attention, based on an everyday scenario. They are
and attentional required to count floor numbers in an
switching elevator using auditory and visual stimuli.
They are required to count low, medium and
high pitch tones to determine what floor the
elevator stops at.
Tower Task Planning, problem Subjects are presented with three rods, and
solving, learning, and disks that can be stacked on each rod
maintenance of creating a tower. Subjects are also shown a
instructional sets picture of the a tower that they must create
following three rules, by moving only one
disk at a time, by not placing a small piece
 27

on a larger piece, and by using only one

hand. They are instructed to work as quickly
as they can with the fewest moves possible.
Self-Ordered Planning and working A set of stimuli are presented on a number
Pointing Task memory of sheets in a booklet. The subjects are
instructed to point to one stimulus at a time,
without ever pointing at the same stimulus
twice in each set.
Functional Dysexecutive Self-report of every Subjects are asked to answer questions
Outcome Measures Questionnaire day executive related to everyday executive function
(DEX)- Self problems problems rating each item from 0-never to 4-
very often.
DEX- Other Other report of Subjects significant others are requested to
everyday executive complete the other form of the Self-DEX
problems questionnaire.
Goal Attainment Goal achievement A functional outcome measure in which
Scale (GAS) goals are set and revisited post-intervention
to determine achievement level.

3.4 Conclusion
To the best of our knowledge, this is the first time that a qualitative tool has been devised to
assess levels of competence in administering GMT. The adherence-to-protocol measure (CRTS-
GMT), in combination with efficacy outcome measures assessing the novel delivery of GMT
(WebGMT), makes this both a comprehensive and innovative approach to cognitive
rehabilitation.
 28

Chapter 4
Web Goal Management Training: Online Cognitive Training
(In preparation for submission to Neuropsychological Rehabilitation)

Authors: Fahmi, H., Middleton, L., Marziali, E., Dawson, D., Levine, B., Turner, G., & Black.,
S.E.

Abstract

The aim of this study was to evaluate the feasibility and efficacy of an innovative, web-based,
cognitive rehabilitation intervention (Goal Management Training - GMT) for adults with
cerebrovascular disease (CVD) or suffering from risk factors of CVD and reporting executive
function complaints (N=8). GMT is one of the few evidence-based interventions for remediating
executive control dysfunction and is typically delivered using a small-group, in-class format.
Here we report the results of a pilot study to investigate the feasibility and efficacy of a remote,
web-based GMT delivery protocol to reduce barriers of accessibility. To assess feasibility of web
delivery, a rating scale was adapted and further developed to analyze various aspects of
intervention delivery such as alignment with the training protocol, therapist competence and
participant engagement. Qualitative analyses of recorded training sessions revealed high protocol
adherence, competence and engagement. Self-reported feedback questionnaires showed positive
opinions of technology used within the learning experience. To assess intervention efficacy, a
battery of neuropsychological measures was administered at baseline and post-intervention. No
significant results were observed on standardized neuropsychological measures; however,
improvements were observed on the Goal Attainment Scale, a functional outcome measure of
goal attainment in real world settings and a critical measure of GMT intervention success. This
pilot intervention study is the first to demonstrate both feasibility and efficacy of a remote, web-
based platform for delivering small-group cognitive rehabilitation. These results suggest that
web-based cognitive rehabilitation offers considerable promise as a tool for improving access to
therapy for persons facing significant physical, geographic or economic barriers to accessing
cognitive rehabilitation.
 29

Introduction
Rehabilitation of executive control dysfunction: Goal Management Training
Cognitive dysfunction secondary to frontal lobe damage or damage to frontally-mediated
systems typically presents in the clinic as executive control problems with deficits in areas such
as planning, problem solving, behavioural control, mental flexibility, decision making and the
ability to sustain attention (Levine, Turner, & Stuss, 2008). According to Duncan and Miller
(2002), these executive control functions are primarily associated with flexible deployment of
attentional resources necessary to complete a given task – i.e., guidance of goal-directed
behaviour. Sustained attention is especially crucial to goal-direction in non-routine situations
requiring the persistent maintenance of higher order goals in working memory with competing
goals; environmental distractors may lead to off-task behaviours (Levine, et al., 2011). Levine
and colleagues (2011) describe this phenomenon of goal neglect as the displacement of a novel
goal in working memory due to the passive activation of a habitual response. In remediating
these functions, an evidence-based cognitive rehabilitation intervention, Goal Management
Training (GMT) (Levine, Manly and Robertson, 2008), has been developed.

GMT is a strategy-based intervention that promotes the re-engagement of endogenous sustained

attention in patients (or healthy older adults – see van Hooren et al., 2007; Levine et al., 2007)
who have been diagnosed with executive function deficits or who express difficulties in goal
direction in their everyday lives. The program emerges from Duncan’s model of goal neglect
(1986) which conceptualizes executive control dysfunction as the separation of ongoing action
from its original intent. The intervention promotes meta-cognitive strategies to aid participants in
‘stopping’ off-task behaviours, refocusing attention towards relevant goals, creating an explicit
goal hierarchy, dividing complex tasks into small subtasks, and actively monitoring progress
towards goal attainment (Duncan, Emslie, Williams, Johnson, & Freer, 1996). Levine and
colleagues (2000) originally tested this approach in a cohort of traumatic brain injury (TBI)
patients with executive control deficits. Their results showed a significant improvement in
executive control on simulated real-life tasks. A follow-up case study involving a post-
encephalitis patient with severe executive function deficits showed similarly promising results
with evidence that the benefits of GMT transferred beyond the clinic to improvement in
everyday life and application of learned strategies to untrained situations.
 30

Building upon these preliminary results, Levine et al. (2007) further investigated the
effectiveness of GMT by using an elaborated multi-session form of the intervention on normal
aging individuals who reported problems with executive functions. Following GMT, older adults
demonstrated significant improvement on simulated real life tasks as well as examiner-rated
performance on complex tasks (Levine et al., 2007). However, in this first large randomized
control trial, GMT was embedded in a larger multi-model intervention protocol, which limited
the ability of the researchers to attribute training gains to GMT alone. In a recent randomized
trial of GMT in an acquired brain injury cohort, Levine and colleagues (2011) reported training-
related gains in both standardized measures of executive control and in real-world outcomes. In
sum, evidence for the efficacy of GMT as an executive function rehabilitation intervention has
now been demonstrated in several populations including traumatic brain injury and stroke (Fish,
et al., 2007; Levine, et al., 2011; Chen , et al., 2011), normal aging (Levine, et al., 2007; van
Hooren, et al., 2007) and case studies of patients with focal cerebellar damage (Schweizer, et al.,
2008).

Web-based interventions: a novel approach to delivering GMT in the clinic

There is increasing evidence that GMT can be used to remediate executive control deficits in
populations where such deficits are clinically important. Based on the success of these efficacy
studies, efforts are now underway to expand the availability of GMT as a rehabilitation tool in
the clinic. A critical challenge going forward is how to most effectively deliver this intervention
to participants who often confront significant access barriers. In its typical implementation
format, GMT is delivered in a small group consisting of three to four participants and a trained
facilitator. Sessions are typically two hours in duration, occurring twice or once per week for five
or ten weeks (20 hours of total in-class time). Participants are also asked to complete an
additional 20 hours of at-home assignments. This schedule places significant time and travel
burdens on participants. In an urban setting, where all efficacy trials published to date have been
undertaken, these burdens include parking costs, transit access as well as safety issues. Outside
of urban centers, lack of trained facilitators or critical mass of participants in a single location,
effectively preclude GMT as a rehabilitation option. To eliminate these barriers to access, we
describe here a novel approach for delivering GMT using a web-based cognitive intervention
platform.
 31

There are no published reports investigating the feasibility or efficacy of web-based delivery for
small-group cognitive therapy. However, there is considerable support for such an approach
emerging from studies of cognitive behavioural therapy (CBT). Early evidence suggests that
clinical results from web-based delivery methods closely replicate those from more standard in-
person delivery methods in panic and post-traumatic stress disorder (Bouchard, et al., 2004;
Germain, Marchand, Bouchard, Drouin, & Guay, 2009; Frueh, Monnier, Yim, Grubaugh,
Hamner, & Knapp, 2007; Morland, Pierce, & Wong, 2004). However, in these reports one-on-
one therapeutic trials were employed and group-based therapeutic methods were not assessed.
Web-based group interventions have recently been investigated in psychosocial support groups
for dementia patient caregivers (Marziali & Donahue, 2006). Through qualitative methods of
analysis, web-based psychosocial groups were evaluated and showed positive therapeutic
outcomes in areas such as group bonding, shared-experiences and extended use of technology for
building bonds amongst group members (Marziali, 2006). Critically, social workers facilitating
on-line group treatments reported comparable group bonding between clients as well as clients
and therapists (Damianakis, Climans, & Marziali, 2008).

While no studies have directly examined the efficacy and effectiveness of delivering small group
cognitive therapy using a web-based platform, evidence emerging from CBT and caregiver
groups suggest that such an approach may be feasibly undertaken without compromising
therapeutic efficacy. In this preliminary rehabilitation probe, we report the first feasibility and
efficacy data of web-based GMT (WebGMT). There are two primary research objectives. First,
we will examine the clinical feasibility of WebGMT, including assessment of protocol
adherence, using a semi-quantitative-qualitative rating scale developed for this study as well as
self-report client satisfaction surveys. Second, we will investigate the efficacy of WebGMT,
contrasting pre- versus post-training performance on standard clinical assessments as well as
functional outcome measures of real-world goal attainment.

Materials and Methods

Participants
For this pilot study, we implemented convenience sampling wherein participants were selected
based on their clinical availability. This method is appropriate for exploratory research and for
populations where recruitment is difficult due to low prevalence and/or difficulty with patient
 32

access (Stevens, 1996). Our target population for this preliminary study of WebGMT was
persons diagnosed with or at high risk of cerebrovascular disease. The typical pattern of
cognitive dysfunction in this population is comprised of executive control deficits in the context
of relatively preserved memory functioning, making this an ideal cohort for a cognitively
demanding, strategy-based intervention such as GMT. Two WebGMT groups were conducted
each consisting of four participants (N=8; see Table 4.1).

The eight participants were recruited from Sunnybrook Health Sciences Centre and Baycrest
Centre for Geriatric Care in Toronto, Canada, as well as through the local community.
Participants were selected based on the following inclusion criteria: age ranging between 50-80,
diagnosis or physician report of at least one of the following: transient ischemic attack (TIA),
white matter disease (WMD), CVD, stroke risk factors (i.e., hypertension). Participants also had
to have evidence of executive function difficulty as noted in patient records or endorsed during
intake interviews, including two elders with a past history suggestive of attention deficit
hyperactivity disorder (ADHD). All participants were required to have a home computer, high
speed internet service and at least some experience with using a computer and accessing the
internet.

Table 4.1 Participant Characteristics

Subject ID Gender Age Education Recruitment Site Medical Conditions
WB-GMT-01 F 72 18 Sunnybrook TIA/WMD/hypertension
WB-GMT-02 M 70 18 Sunnybrook TIA/WMD/hypertension
WB-GMT-03 M 69 20 Community TIA/hypertension
WB-GMT-04 F 74 13 Sunnybrook TIA/WMD/hypertension
WBCVD2001 M 61 19 Baycrest Hypertension
WBCVD2002 F 77 17 Baycrest TIA/WMD/hypertension
WBCVD2003 M 75 23 Community ADHD/hypertension
WBCVD2004 F 65 16.5 Baycrest ADHD/hypertension

Patients were not excluded based on performance on traditional neuropsychological tests of

frontal lobe function, as research shows that individuals may do well on these standardized
assessments and still exhibit executive function impairments in everyday life (Burgess, et al.,
2006; Chan, Shum, Toulopoulou, & Chen, 2008; Stuss & Levine, 2002). Exclusion criteria
 33

included: inability to fluently communicate in English, primary memory deficit and any other
cognitive or functional impairment limiting ability to participate in the training sessions.

All participants reported having hypertension, which is a known risk factor for CVD. All
participants reported functional impairments such as problems with attention, word finding,
absentmindedness, problems with organization, and problems concentrating. Five of the total
number of participants had a stroke or displayed evidence of white matter disease on brain scans
obtained from medical records. The three remaining participants had self-reported executive
control dysfunction and two of these had been diagnosed ADHD, in which executive control
deficits are a hallmark (Proal, et al., 2011).

The study was approved by the Baycrest Research Ethics Board as well as the Sunnybrook
Research Ethics Board. All participants provided written informed consent prior to participation.

Goal Management Training

All participants participated in ten two-hour sessions of GMT (see table below for highlights of
each training module) over five weeks. GMT was delivered using both didactic methods,
involving scripted sessions, as well as more experiential learning opportunities, including group
discussions and in-class exercises following a GMT manual (Levine, Manly, & Robertson,
2008). The primary objective of the intervention was to equip participants with a step by step
strategy that could readily be adopted in their everyday lives. All participants were trained to
‘stop-state-split’ wherein they recognized off-task behaviours, stated their relevant goals and
conducted a means-end analysis to ensure progress towards the goal is maintained (Levine et al.,
2008). In addition to in-class sessions, they were required to complete approximately 20 hours of
at-home assignments, where they were taught to identify and incorporate learned strategies into
their everyday life.

Web Platform
The Adobe Connect videoconferencing platform for meetings and e-learning was the web-
platform selected for the WebGMT protocol. It is an on-line, real-time meeting platform
consisting of features such as screen sharing, high-quality video conferencing, recording
capabilities, notepad sharing, and group chat options. All users received private accounts on this
platform and accessed scheduled meetings through a host web-site created by Dr. Marziali’s
 34

research team. The host web-site served as a simple gateway allowing the facilitator to schedule
and post meeting links for participants. In addition, all ten sessions were recorded and saved on a
password secured server for further qualitative analyses.

Procedures
Upon participant recruitment, and prior to start of study, the GMT facilitator met with each GMT
group member in person at the Baycrest site. The objective of this meeting was to a) provide all
participants with a face to face meeting with the facilitator and one another before virtual
sessions, and b) debrief participants and gauge their comfort with technology through open
discussion. Once all participants had met each other and were informed of future steps, they were
provided with GMT packages consisting of intervention specific materials as well as a web-
camera with built-in microphones. The next step was to create accounts and train both therapist
and participants in accessing and using the videoconferencing software. Training was provided
by an experienced research assistant (RA) from Dr. Marziali’s team, who had worked with an
elderly population in the Caring for Others Internet Group intervention (Marziali & Donahue,
2006). The individual meetings between the RA and participants consisted of phone calls and
one-on-one training for web-camera, microphone and headphone installations the procedure for
logging onto the Adobe Connect software via the Caring for Me host website with the help of a
user’s manual. The user manuals were developed by Dr. Marziali and were used in this study as
a technical handbook to guide participants and as a reference during the training sessions. The
initial training meetings (two for each participant) were completed and additional on-line training
sessions were booked based on participant request and comfort with technology. The therapist
received similar training and was provided with an administrator’s manual to host on-line
meetings. Once the GMT facilitator was confident and comfortable with the technology, the
WebGMT group sessions were scheduled.

For participants, attending the GMT sessions via the described web-based platform entailed
logging into Adobe Connect with the web-camera and microphone and “tuning-in” to the shared
screen, where the facilitator shared the GMT modules referencing the provided workbook and
package materials whenever necessary. The participants were able to answer questions via voice
or chat options and to follow notes provided by therapist using the notepad option. The three
sessions were also attended by the RA in order to address any technology related problems as
 35

they occurred. Once several sessions were completed with the RA present, the participants were
able to troubleshoot common problems such as a brief disconnection or inability to hear other
participants on their own, and the RA’s input was no longer necessary. In addition, for some
interactive tasks, the GMT instructions were altered to accommodate the delivery mode (for
example, if the task required that participants work in pairs, the instructions were tweaked such
that the task was performed by participants individually). The homework assignments were also
adjusted from requiring participants to remember to come to class with their workbooks to
remembering to send the facilitator an email one day before scheduled session. It is also worth
noting that the concepts behind the interactive tasks and homework assignments were not altered
(e.g. absentmindedness was demonstrated in both tasks whether with a partner or individually).

For qualitative comparison purposes (objective 1), the WebGMT groups were rated by two
independent raters using a devised outcome tool assessing for competence. The first rater was an
occupational therapist who was trained in the GMT protocol and she rated both WebGMT
groups. The second rater was different across both groups, for the first WebGMT group, the rater
was a Master’s student and for the second group, the rater was a PhD student. Both second raters
were in the cognitive rehabilitation field of study and were familiar with the GMT intervention.
In addition, to assess impact of delivery mode on effectiveness of cognitive rehabilitation, a
within-subject comparison method was carried out using neuropsychological tests, self-reported
measures and functional outcome measures.

The WebGMT approach was administered in two separate trials on groups of four participants.
Both groups received identical GMT administration, but the battery of tests administered pre-
and post-intervention was slightly different. Specifically, WebGMT group 1 did not perform the
Tests of Everyday Attention as it was not initially incorporated within the study protocol. As the
objective was to focus on feasibility of administration in the first trial, the quantitative outcome
measures were not emphasized; the participants in the first group did not return the self-report
questionnaires and opted out of an optional GAS follow-up. Running the first WebGMT group
provided insights on logistics for the second WebGMT group as well as experience for the
therapist in using the technology.

Data Analysis
Feasibility
 36

All web-based GMT sessions were archived on a secured server where they were accessible via
username and password. Web-based recordings were analyzed using the devised adherence-to-
protocol scale for GMT adapted from the Cognitive Therapy Scale-Revised (CTS-R), a measure
of therapist competence in Cognitive Therapy (CT). The CTS-R contains 12 items covering
various features of therapy; some are general items such as adherence to set agenda, feedback,
collaboration, pacing and efficient use of time as well as interpersonal effectiveness, while others
are CT specific features that were irrelevant to GMT such as eliciting cognitions, guided
discovery and eliciting appropriate emotional expression (Blackburn, et al., 2001). The general
items were unchanged as they could apply to any therapy with a pre-set protocol. The CT
specific items were taken out and replaced with items important to GMT and derived from the
intervention. For instance, a participant engagement item was added to capture group interaction.

All nine items on the newly devised scale, The Cognitive Rehabilitation Therapy Scale for Goal
Management Training (CRTS-GMT), were rated along a six-level model of competence adapted
from the Dreyfus level of competence model (Dreyfus, 1989), across a seven-point Likert scale
ranging from 0- where the therapist did not adhere to that aspect of therapy, to 6- where there is
adherence at very high skill (Likert, 1932). The two raters described above, with good
knowledge of GMT, were selected to watch and rate randomly selected sample sessions. As
described earlier, two WebGMT groups were run. Three sample sessions of the first, and five
sample sessions of the second group were rated by two different raters. In order to establish
competence, a minimum score of 27 on the CRTS-GMT was set as the criterion; this averages to
a score of three on each item (Blackburn, et al., 2001). Data was analyzed by establishing inter-
rater reliability and calculating percent agreement. The objective was to ensure that critical
therapeutic aspects of the GMT intervention were addressed in a similar manner in the online
environment as they were in the classroom.

Efficacy

The efficacy of WebGMT was assessed via a battery of standardized and experimental
neuropsychological tests of executive functioning administered immediately before the
intervention and immediately after to detect post-training cognitive changes. Standardized
laboratory tests may not pick up on cognitive changes manifested in the everyday lives of
individuals, hence functional outcome measures are crucial in determining various changes that
 37

are otherwise undetected (Dawson, Anderson, Burgess, Cooper, Krpan, & Stuss, 2009). The
battery comprised of these functional outcome measures; a self-report questionnaire to assess
changes in daily life as well as the Goal Attainment Scale (GAS) measure. All assessments were
administered by the graduate student, who was also a trained psychometrist.

The neuropsychological battery comprised the standardized tests mentioned previously such as
trail making A and B to assess visual attention and task switching, digit span for attention and
verbal working memory (Wechsler, 1985), auditory consonant trigrams for verbal working
memory and divided attention (Brown, 1958) as well as self-ordered pointing task for planning
and working memory (West, Ergis, Winocur, & Saint-Cyr, 1998). To assess frontal cognitive
processes such as planning, problem solving, learning and maintenance of instructional sets, two
versions of the tower task were used in this study to eliminate practice effects by alternating
items from the original task (Delis, Kaplan, & Kramer, 2001; Levine, et al., 2011). Further tests
from the Delis-Kaplan Executive Function System were used: tests to assess verbal fluency,
examining frontal and temporal regions; design fluency tests to assess fluency in spatial domain;
and an additional measure of attention, inhibition and switching using the Stroop test (Delis,
Kaplan, & Kramer, 2001).

Self-reported changes were determined using the Dysexecutive Questionnaire (DEX), a 20-item
questionnaire designed to assess changes in motivation, personality, cognition and behavioural
changes (Burgess, Alderman, Evans, & Emslie, 1996). To measure clinically important change
in cognitive rehabilitation, we administered the GAS which is a patient-centered assessment tool
that enables the quantification of progress towards attainment of goals relevant and meaningful
in participants’ everyday lives (Rockwood, Joyce, & Stolee, 1997). It is a standardized method
which has proven more robust than various other functional outcome measures of cognitive
rehabilitation (Rockwood, Joyce, & Stolee, 1997).

Results

Feasibility- Qualitative Measures

CRTS-GMT. The two sets of ratings for group 1, sessions 2, 5 and 9, and for group 2, sessions
1, 3, 5, 6, and 9 exceeded the minimum score of 27, indicating competence in WebGMT delivery
 38

(see Table 4.2 and Table 4.3). The average total across all sessions for both raters was 47 (SD=
2.8) for group 1 and 48 (SD= 3.3) for group 2. Inter-rater reliability was calculated using the
intra-class correlation statistic for quantitative measurements and yielded an ICC of 0.49,
indicating a fair agreement, between raters of group 1, and an ICC of 0.59, indicating a moderate
agreement between raters of Group 2 recordings. Percent agreement between raters was
calculated to be 56% for WebGMT group 1 and 64% for WebGMT group 2 with an overlap of
plus and minus 0.5 points across the ratings.

Table 4.2 WebGMT Group 1 CRTS-GMT scores of two raters across randomly selected session
recordings

Session 2 5 9
Rater 1 2 1 2 1 2
1. Adherence to module outline 6 5 4 5 4.5 4
2. Feedback 5.5 5 6 6 6 6
3. Collaboration (verbal and non-verbal) 5.5 2 3 2 5 5
4. Pacing and efficient use of time 5 4 5 5 5 4
5. Interpersonal effectiveness 5.5 5 6 6 6 6
6. Conceptual integration 5.5 4 6 6 6 6
7. Therapist Skills 5 6 5 6 6 6
8. Homework Setting 5 6 5 6 6 5
9. Participant Engagement 4.5 5 5 5 5.5 6
Total (max. 54) 47.5 42 45 47 50 48
Percent Agreement between raters 56%
Intraclass correlation 0.49 (fair agreement)

Table 4.3 WebGMT Group 2 CRTS-GMT scores of two raters across randomly selected session
recordings
Session 1 3 5 6 9
Rater 1 2 1 2 1 2 1 2 1 2
1. Adherence to module outline 3.5 6 3.5 6 6 6 6 6 6 6
2. Feedback 6 5 6 6 6 6 6 6 5.5 5
3. Collaboration (verbal and non-verbal) 5.5 4 5.5 5 6 6 4.5 5 5 5
4. Pacing and efficient use of time 3.5 5 3.5 3 6 3 6 4 6 5
5. Interpersonal effectiveness 6 6 6 6 6 6 6 6 6 6
6. Conceptual integration 5.5 6 6 5 6 6 6 6 6 6
 39

7. Therapist Skills 5.5 6 6 6 6 5 6 6 6 5

8. Homework Setting 5 6 3 6 6 6 6 6 ** 6
9. Participant Engagement 5 4 5 4 6 5 5.5 6 5 5
Total (max. 54) 45.5 45.5 44.5 44.5 54 49 51 51 45.5 49
Percent agreement between raters 64%
Intraclass correlation 0.59 (moderate agreement)

Feedback Survey. The feedback survey showed that seven out of the eight participants had
very or somewhat positive feedback on each item of the survey. In addition, five of the total
number reported that they would prefer a WebGMT group over a traditional in-class GMT
intervention (see Table 4.4).

Table 4.4 Participant Feedback Survey

Questions V S N NV NA
Overall how easy or difficult did you find the use of Adobe XXX XXX X
Connect? X
Overall how helpful was the technical support offered? XXXXX X X
X
How helpful did you find Adobe Connect specifically in the XXXXX XX X
learning experience?
If you were to participate in this brain training program, would you rather sign up to a web-based or
in-class group?
5 x Web-based
3 x In-class

NOTE: V=very, S=somewhat, N=neutral, NV=not very, NA= not at all (NA for Question 2 =Not
applicable)

Efficacy- Pre-Post Intervention Measures

Neuropsychological Tests of Executive Functions. Reliable changes indices (RCI) were
calculated for the Delis-Kaplan Executive Function System (DKEFS) measures (see Table 4.5)
as these had published test-retest reliability information. As can be seen in Table 6, few
individual measures showed significant RCI values from pre-to post-training. Similarly, no
significant results (p > .05 all tests) were observed on t-tests of for pre- versus post-GMT
neuropsychological data (Table 4.6).
 40

Table 4.5 DKEFS raw score significance using the reliable change index
Reliable Change Index 1 2 3 4 5 6 7 8 Mean 1 (SD) Mean 2 (SD)
Phonemic Fluency NS NS NS NS NS NS NS NS 40.8 (18.0) 45.4 (16.6)
Semantic Fluency NS NS NS NS NS NS NS NS 41.4 (9.59) 38.8 (9.3)
Design Fluency 1 NS S NS NS NS NS NS S 10.0 (2.3) 9.8 (4.2)
Design Fluency 2 NS NS NS NS NS NS NS NS 11.3 (2.4) 11.3 (2.3)
Design Fluency 3 NS NS NS S NS NS NS NS 7.8 (4.3) 8.4 (1.8)
Stroop NS NS NS NS NS NS NS NS 77.6 (23.9) 73.1 (25.8)
Stroop Switching NS NS NS NS NS NS NS NS 86.1 (29.0) 80.4 (18.4)

Table 4.6 Neuropsychological tests of executive function raw score significance using Student’s
t-test
Executive Function Assessment N t-test Mean 1 (SD) Mean 2 (SD)
Domain significance
Working memory SOPT 8 NS 10.6 (4.1) 11.6 (7.5)
Consonant 9 seconds 4 NS 9.3 (5.1) 8.8 (3.4)
Trigrams 18 seconds 4 NS 9.8 (5.5) 5.3 (3.8)
Processing Speed Trails A 8 NS 33.6 (8.4) 36.2 (8.4)

Task Switching Trails B 8 NS 114.9 (47.8) 111.7 (47.3)

Attention Digit Span 8 NS 17.1 (4.1) 16.8 (4.3)
Tests of Everyday Attention 4 NS 6.3 (1.9) 5.5 (1.3)
Planning Tower Task- Achievement 8 NS 9.8 (1.8) 9.9 (2.3)
score

Questionnaires. There were no significant effects from within-subject t-test analyses (P > .05,
all tests) on the five DEX scales or overall scores. Table 4.7 shows group means across self and
other scores of specific DEX scales.

Table 4.7 Dysexecutive Questionnaire raw scores using Student’s t-test

Questionnaire Inhibition Mean 1 (SD) Mean 2 (SD) Intention Mean 1 Mean 2
Component (SD) (SD)
Self t-test NS 11.0 (4.4) 12.333 (5.0) NS 7.7 (3.2) 8.3 (1.2)
Other t-test NS 8.0 (7.0) 7.000 (6.9) NS 6.0 (4.6) 4.0 (3.6)
Questionnaire Executive Mean 1 (SD) Mean 2 (SD) Positive Mean 1 Mean 2
Component Memory Affect (SD) (SD)
Self t-test NS 2.0 (2.6) 3.000 (3.0) NS 5.0 (1.0) 5.333 (2.1)
 41

Other t-test NS 1.7 (1.2) 1.333 (0.5) NS 3.7 (2.1) 4.000 (2.0)
Questionnaire Negative Mean 1 (SD) Mean 2 (SD) Overall Mean 1 Mean 2
Component Affect (SD) (SD)
Self t-test NS 3.0 (1.0) 3.667 (2.3) NS 28.7 (7.5) 32.7 (9.0)
Other t-test NS 1.3 (0.6) 1.333 (1.5) NS 20.7 (12.1) 21.7 (9.5)

Goal Attainment Scale (GAS). For the four participants in the WebGMT group 2, GAS scores
(see table 4.8) showed that all participants achieved expected level (N=1) or higher (N=3) on at
least one of their established goals.

Table 4.8 Goal Attainment Scale measure

Subject Summary of Goal Score Achievement Level
WBCVD2001 Goal 1 0 Expected level
Reduce incidence of word-finding difficulty in conversations by 40%-
50%. Be able to find the right word in conversations about half the time
WBCVD2002 Goal 1 2 Much better than
Use the same book to track tasks/lists at least 50% of the time expected level
Goal 2 1 Somewhat better than
Complete at least half of required tasks without being distracted or expected
starting a new task
WBCVD2003 Goal 1 1 Somewhat better than
Begin at least 50% of all projects on time expected
Complete at least 40% of all projects on time
WBCVD2004 Goal 1 -1 Somewhat less than
Get into bed by midnight at least 4 out of 7 nights in the week expected
Goal 2 2 Much better than
Prepare lunch/dinner in advance and take it along at least 3 out of 5 expected
work-days in the week

Discussion
This pilot study is, to our knowledge, the first investigation of remote, web-based and real-time
delivery of a small group cognitive training intervention (GMT). There were two primary
research objectives: i) to evaluate the feasibility of this innovative delivery approach and ii) to
assess the efficacy of WebGMT for remediating executive control deficits.
 42

Objective 1: Feasibility
The semi-quantitative-qualitative adherence-to-protocol rating scale adapted from the CTS-R
(Blackburn, et al., 2001) demonstrated that GMT can be delivered effectively via a web-interface
without compromising learning objectives while maintaining a high level of therapeutic integrity
(therapist competence, participant engagement and adherence to protocol). The analyses of
sample sessions demonstrated that across two raters the CRTS-GMT totals far exceeded the
minimum level of competence set at the midpoint level for the rating scale. The inter-rater
reliability across all sessions from both web-based intervention groups demonstrated fair to
moderate agreement, deeming the results reliable. Percent agreement on specific items between
raters ranged from 55%-65% for both groups. When total scores were closely examined, the
difference between the totals of two raters was minimal (0-10, with a total of 52); therefore,
although raters do not agree exactly on the score of specific items, the disagreement is not drastic
enough to make large changes in total scores. Moreover, as a supplementary analysis (see
appendix C), we contrasted these therapist ratings with ratings of previously recorded in-class
GMT sessions from an earlier study of Multiple Sclerosis patients. While this was purely a
speculative analysis, given the non-matched patient samples and different therapists, it is
nonetheless encouraging that a large majority (72%) of ratings were equal to or higher for the
WebGMT delivery versus the traditional in-class format. These preliminary data provide strong
support for WebGMT as a feasible strategy for increasing access to GMT and potentially other
cognitive rehabilitation programs where time, cost, geographic or physical barriers may impede
participation.

To further support the feasibility of this approach, surveys completed by research participants
contained positive feedback, suggesting that this novel approach did not impede participation in
a much needed cognitive rehabilitation intervention. When subjects were asked whether they
would choose a web-based intervention versus a traditional in-class intervention, five out of the
eight participants chose the web-based platform. This level of positive affirmation from subjects
who had no previous experience with this type of web-interface supports the notion that the
remote delivery approach was not a significant disincentive for subject participation; indeed, for
over half of the participants, the loss of in-person contact was more than offset by the reduction
in travel time afforded by WebGMT.
 43

Previous research examining the feasibility of web-based platforms in psychosocial support

groups addressed this objective through extensive qualitative methods (Marziali & Donahue,
2006; Marziali, 2006; Marziali, Damianakis, & Donahue, 2006). This study followed a different
approach in devising a brief 9-item scale assessing therapist competence of following the GMT
outline and participants’ level of engagement with the therapist and as a group. As a future
direction, this tool can also be used in training therapists to administer GMT, whether through a
web-based platform or in traditional in-class methods, ensuring GMT is delivered consistently
irrespective of location, therapists or delivery modality.

Objective 2- Efficacy
We used a similar battery of neuropsychological tests and functional outcome measures as had
been reported in previous GMT studies (Levine, et al., 2007; van Hooren, et al., 2007;
Schweizer, et al., 2008; Levine, et al., 2011) to assess executive functioning pre- and post-
WebGMT. Results were consistent with these previous studies wherein few significant changes
were identified on standardized neuropsychological measures. This is likely due to the small
sample size of this pilot study, high baseline performance in our samples, and insensitivity of
standardized executive function measures (Levine & Downey-Lamb, 2002).

The insignificant DEX findings observed here reflect those reported in the most recent GMT
study (Levine, et al., 2011). As reported by Levine and colleagues, we suggest that the increase
in symptoms (albeit insignificant) observed here may reflect increased insight or awareness of
cognitive impairments by the participant and their significant other causing them to increase item
endorsement.

Here we also used the GAS, a functional outcome measure of goal attainment in the daily life of
participants. This measure, which has not previously been reported in conjunction with GMT
studies, is a semi-quantitative tool that enables researchers to obtain a standardized measure of
goal attainment across a wide range of goals and individuals. It is a standardized method of
quantifying achievement and has proven to be a sensitive measure of cognitive rehabilitation
efficacy (Rockwood, Joyce, & Stolee, 1997). Given that the emphasis of GMT is on re-
establishing goal-directed behaviour for persons with executive dysfunction, the positive results
obtained here are particularly promising for WebGMT as an efficacious training protocol that
can effect change in the lives of participants.
 44

Conclusion
This pilot study is the first of its kind to present the administration and delivery of group-based
therapy using current advances in technology. Accordingly, this study examines the feasibility
and effectiveness of WebGMT, an approach to delivering GMT using web-based real-time
videoconferencing software that has not been previously undertaken. To the best of our
knowledge, this is also the first study to employ an adherence-to-protocol scale, a newly
developed semi-quantitative-qualitative tool devised to analyze various aspects of therapy. We
administered WebGMT to two groups of participants with CVD or with CVD risk factors.
Preliminary data from the devised semi-quantitative-qualitative scale (CRTS-GMT) showed that
this approach is feasible and that factors such as group engagement and adherence to GMT
protocol are not compromised. In addition, the efficacy of this approach was analyzed through a
battery of neuropsychological and functional outcome measures. We found that, consistent with
previous research, participants showed significant improvement through functional outcome
measures and no significant changes through neuropsychological measures. These results in
combination can be used as the stepping stone for further research investigating this innovative
delivery mode. This data can also provide us with guidance to help determine effect sizes for
future randomized controlled trials. Moreover, future studies can further develop the CRTS-
GMT tool to use as a training tool for therapists administering GMT in traditional and web-based
modes.
 45

Chapter 5
Discussion
The main purpose of this study was to investigate the feasibility and efficacy of delivering group
cognitive rehabilitation therapy using a web-based platform for people with executive function
impairments associated to cerebrovascular risk factors and/or cerebral small vessel disease. This
study was important to undertake in order to introduce a novel approach to improve the
accessibility of cognitive rehabilitation interventions. As mentioned previously in Chapter 2,
Goal Management Training (GMT) is an evidence-based cognitive rehabilitation intervention
(Levine, et al., 2000; Levine, et al., 2007; Fish, et al., 2007; Levine, et al., 2011) that requires
participants to attend ten 2-hour sessions over five weeks, placing time and travel burdens on
patients that may limit accessibility and decrease patient engagement and participation. To
address these access barriers, this study investigated the feasibility and efficacy of web-based
administration of GMT (WebGMT). Cerebrovascular Disease (CVD) patients were selected as
the target population for this pilot intervention study. Executive dysfunction in the presence of
relatively preserved memory functioning is a common cognitive profile of CVD, and these
patients are well placed to benefit from a strategy-based, executive training program.

Although increasingly common in individualized cognitive behavioural therapy and in

psychosocial support groups, to the best of our knowledge, no one has examined the delivery of a
group-based cognitive intervention in real-time using a web-based platform (Bouchard, et al.,
2004; Marziali, 2006; Marziali & Donahue, 2006). In this study, two WebGMT sessions were
implemented to evaluate the feasibility and efficacy of delivering GMT. Feasibility was
primarily assessed using a semi-quantitative-qualitative scale of adherence to protocol,
developed from a cognitive behavioural therapist rating instrument (Blackburn, et al., 2001).
Various therapeutic aspects of the GMT training protocol, including group engagement,
therapist-participant interactions, etc. were assessed (See Appendix A). Furthermore, I created a
self-report feedback survey through which participants were able to report the usefulness of
technology as well as their overall preference for intervention delivery (in-class versus web-
based - see Appendix B). Based on the results of these preliminary feasibility analyses, I
concluded that this innovative virtual delivery approach for delivering GMT could be
implemented successfully in a patient population, closely adhering to the established therapeutic
protocol and without compromising participant engagement.
 46

To address the efficacy question, I adopted a within-subject, longitudinal design in which I

administered a battery of neuropsychological and functional outcome measures at baseline (pre-
intervention) and post-intervention, using cognitive assessment protocols adopted for previous
GMT studies with acquired brain injury and normal aging populations (Levine, et al., 2011; van
Hooren, et al., 2007). No significant changes were observed on standardized neuropsychological
measures, as has been reported in previous GMT studies, but in this preliminary proof-of-
concept study, likely reflecting the small sample size and limited statistical power to detect
changes. Nevertheless, preliminary results from a real world measure of goal attainment (Goal
Attainment Scale- GAS, see Chapter 3), indicated that patients participating in WebGMT were
able to achieve real world goals at a level that exceeded expectation – a critical measure of GMT
success.

This final chapter provides a summary of my thesis project, bringing together the literature
review in Chapter 2 and the findings from the study. The goal of this summary is to provide the
reader with conclusions from this proof-of-concept intervention project as well as lay out
directions for future research. First, I briefly review the relevant background information on
executive control functions and cognitive rehabilitation for the population at hand. Specifically, I
relate the findings from the efficacy study presented here to the results of previous research with
GMT. Second, I provide an overview of existing research using similar web-delivery platforms
in a therapeutic context, highlighting the novelty and promise of this delivery platform for GMT
and other cognitive therapy interventions based on the results obtained in this pilot project.
Finally, I address limitations of the research and provide recommendations for future studies.

5.1 Executive Function

My thesis project focused on GMT, one of the few cognitive rehabilitation interventions with
demonstrated efficacy for remediating executive function impairments associated with normal
aging, as well as acquired brain injury. In reviewing the results of the study, it is important to
have a clear understanding of the cognitive domain involved as well as the theories that the
intervention is based upon. In Chapter 2, I defined executive function as a set of higher cognitive
processes that allow an individual to problem solve, maintain attention, demonstrate behavioural
control, plan and monitor on-going actions as well as adapt to novel situations (Levine, Turner,
& Stuss, 2008). These skills, amongst others, are dependent on the integrity of the frontal lobes
 47

and frontally mediated systems and, as such, are highly susceptible to focal brain changes that
affect this area directly (e.g. stroke) or distributed neuropathological changes impairing multiple
brain regions or the connections between them (e.g. small vessel disease) (Stuss & Alexander,
2000). Not surprisingly, diffuse ischemic white matter changes secondary to small vessel disease
are often associated with executive dysfunction, including reduced goal-directed behaviours
(Verghese, Wang, Katz, Sanders, & Lipton, 2009; Black, 2007). While many schemas have been
proposed to characterize executive control functions, in this study I have characterized executive
control specifically in terms of goal-directed cognitive processing. This characterization of
executive control emerges from three theoretical models of executive function that converge
around an understanding of executive dysfunction as a reduction in goal-directed cognition
leading to impairment of a patient’s ability to maintain focus and avoid off-task behaviours. This
theoretical framework serves as a foundation of the current study, wherein the intervention
protocol explicitly targets goal-directed behaviour in a patient population with known diffuse
brain changes (small vessel disease), and concomitant executive dysfunction.

The three distinct models of executive function reviewed in Chapter 2 and those that inform the
development of GMT are: the supervisory attentional system (SAS) by Shallice and Norman
(1980), Baddeley's model of working memory (1986), and Duncan's model of goal neglect
(1986). Shallice and Norman (1980) define executive functioning as an attentional system which
mediates the activation of existing schemas or the formation of new schemas depending on
whether the cognitive context is routine or novel, with the supervisory attention system (SAS,
i.e., executive control) engaged by the latter but not the former. Baddeley's theory of working
memory expanded on this model of the SAS by introducing the concept of a central executive
system (CES), which is necessary for flexible goal maintenance in the context of distractions that
may promote maladaptive or off-task behaviours. These models were seminal in advancing our
understanding of executive control functioning as an attentional control system, necessary for
maintaining goal-directed behaviours in non-routine, novel and distracting environments.
However, a third model proposed by Duncan, based on studies of patients with frontal lobe
injury, more directly conceptualized executive control functions in terms of goal-directed
behaviour. This theory characterized executive dysfunction as a separation of knowledge from
intended action, resulting in a failure of goal selection and maintenance ultimately leading to
goal neglect (Duncan, 1986). This work directly informed the development of GMT, a cognitive
 48

rehabilitation protocol explicitly targeting goal neglect and failures to maintain goal direction
following frontal system dysfunction.

The review of the cognitive intervention used provides a clearer explanation of why the models
reviewed above are important to the study undertaken. In addition, the definition of executive
function helps the reader to understand specific outcome measures used and their relevance in
measuring cognitive domain.

5.2 Efficacy of Web-Based Goal Management Training

(WebGMT)
The cognitive rehabilitation intervention used in this study is GMT. GMT was originally
developed by Robertson (1996), and has been further elaborated as a multi-session, small group,
cognitive therapy protocol by Levine, Manly and Robertson (2008). GMT is an executive control
function training protocol that emphasizes the re-engagement of endogenous attentional control
processes. It promotes the formulation of explicit goal lists and the use of means-ends analysis to
ensure behaviour is constantly guided by its intended outcome (i.e., the goal). Since the earliest
formulation of this intervention approach, many studies have been conducted on various patient
populations such as acquired brain injury (stroke and TBI) (Levine, et al., 2011; Levine, et al.,
2000; Fish, et al., 2007), focal cerebellar damage (Schweizer, et al., 2008) and normal aging
populations with higher cognitive complaints (Levine, et al., 2007; van Hooren, et al., 2007).
These studies have provided convincing evidence that GMT is effective for remediating
executive control. Indeed. in a recent review of cognitive rehabilitation interventions, GMT was
one of the only executive function intervention strategies recommended as a practice standard
(Cicerone, et al., 2011). All of these studies utilized similar forms of the GMT protocol, adopting
a strategy-based approach to remediate executive control dysfunction and increase goal-directed
behaviour in everyday life. While the efficacy of GMT has been amply demonstrated in a
research context, translation of the intervention protocol from the laboratory to clinical settings
remains a considerable challenge owing to the significant time, travel and potential economic
burdens associated with participation.

My project investigated the feasibility of reducing these barriers by delivering GMT remotely via
a web-based platform. One of the objectives of this study was to use neuropsychological and
functional outcome measures in order to demonstrate the efficacy of this novel WebGMT
 49

approach. To address this question, I used a combination of standardized and experimental

neuropsychological measures, the results of which failed to show statistical significant changes
after pre-post intervention. While the small number of subjects recruited for this pilot study
limited the statistical power for the group analysis, the failure to observe significant changes in
standardized neuropsychological measures is generally consistent with previous research on
GMT. For example, in a normal aging population the Stroop test, a measure of inhibition, did not
show any significant results after GMT and only a few significant improvements were shown on
other standardized measures (van Hooren, et al., 2007). In a recent study of GMT in a healthy
aging cohort, van Hooren and colleagues (2007) attributed their null findings on
neuropsychological testing to the lack of significant impairment on the measures prior to
training, thus limiting the capacity for improvement. The same conclusion can be made for the
results observed in this study. The participants in both intervention sessions displayed high levels
of cognitive performance at baseline and therefore significant changes after intervention were
less likely. Furthermore, standardized measures of frontal lobe function are notoriously
insensitive to executive function deficits in real world settings (Shallice & Burgess, 1991) and
thus would not be expected to show training-related changes (Levine, et al., 2011).

For this reason, the majority of studies investigating the efficacy of executive function
interventions include functional outcome or ‘real-world’ measures, including simulated real life
tasks (Levine, et al., 2000), complex tasks (e.g. Multiple Errands Test) (Dawson, Anderson,
Burgess, Cooper, Krpan, & Stuss, 2009), and self-reported questionnaires (e.g. the dysexecutive
questionnaire) (Burgess, Alderman, Evans, & Emslie, 1996). All of these functional outcome
measures showed improvements following GMT (Levine, et al., 2011; Fish, et al., 2007;
Schweizer, et al., 2008). Given the sensitivity of these measures to GMT intervention, I included
two functional outcome measures, the DEX as well as a novel functional assessment not
heretofore used in GMT intervention research: the Goal Attainment Scale (Rockwood, Joyce, &
Stolee, 1997). The DEX showed no immediate improvement after intervention. This is consistent
with the recent report from Schweizer and colleagues (2008), who suggested that treatment
effects require time for consolidation into the lives of individuals. As this was a pilot
intervention, no long-term follow-up assessments were conducted. Hence, it is unclear whether
this explanation might apply in the current study. In contrast to the DEX outcomes, participants
receiving the GAS reported beyond-expected levels of goal attainment post-GMT.
 50

In summary, this study adapted a proven intervention approach for remediating executive control
dysfunction. The overall goal of the research is to reduce barriers to access imposed by the
standard multi-session in-class delivery protocol through the development of a web-based remote
delivery protocol that (i) is feasible to deliver to individuals with executive dysfunction and (ii)
does not compromise therapeutic efficacy. The results of this pilot study provide encouraging, if
preliminary, support for this approach.

5.2.1 Limitations and Future Directions

While the preliminary data presented here supports the feasibility and efficacy of WebGMT,
future research will need to address several limitations of the current study to confirm the results
presented. First, the participants involved in both WebGMT sessions were a convenience sample
and did not demonstrate clinical impairments on standardized assessment. Therefore, as
mentioned previously, room for improvements on tests of executive functions was limited. In
order to demonstrate improvement in the future, a larger, prospective cohort with greater
variability in baseline performance as well as variability in level of education should be
recruited. Given that this study was a pilot study focusing on feasibility, the lack of a randomized
control trial (RCT) was acceptable; the foundation has successfully been laid for future studies in
which an RCT is used to directly contrast WebGMT with traditional in-class delivery methods.
In addition, the ultimate goal of an intervention is to demonstrate changes in daily behaviour;
that is, functional outcome measures should be focused on, in order to capture changes that are
otherwise undetected using traditional neuropsychological assessments of higher cognitive skills.
Furthermore, in a CBT study, Arundine and her colleagues (2012) observed effects of the
therapy 6-months after treatment, providing evidence that therapy impact may be observed after
a certain time duration which may apply to GMT studies. While preliminary functional outcome
data was presented here, future studies will need to collect multiple functional outcomes with
longer-term follow-up assessments to measure the durability of the training outcomes.

5.3 The importance of WebGMT

In Chapter 2, I highlighted the obstacles that may impede participation in GMT. I also presented
the International Classification of Functioning (ICF), providing a framework to further explain
various factors interacting to affect one’s daily life (Vrankrijker, 2003). In this study, the health
condition is executive dysfunction primarily due to vascular risk factors and small vessel disease
 51

including two elders with ADHD in addition to vascular risk factors. This impairment interacts
with body function and activity to hinder the individual from leading a normal lifestyle in which
they are able to plan and carry out activities as successful as they want or need to. Other
contextual factors, such as distance from urban centers as well as limited access to transportation
can hinder access to treatment that may improve the health condition and in turn increase
participation in society. In the standard cognitive intervention that is the basis of the one
implemented here, such barriers are present. GMT poses time and travel burdens on participants,
making it difficult for patients in need to access and fully engage in the treatment protocol. In
order to overcome these barriers, the WebGMT approach was introduced as a novel way of
delivering cognitive rehabilitation to people otherwise unable to access treatment. This method
required patients to have access to high-speed internet and be comfortable with using the video-
conferencing technology. Two WebGMT groups met online through the Adobe Connect
platform and attended a training program over five weeks of treatment. This new method proved
feasible and initially successful as determined via tools to assess platform delivery. In the
following section, I provide an overview of the qualitative data collected and specific features
that were assessed as fundamental aspects of GMT to gauge the feasibility of delivering
WebGMT.

5.4 Feasibility Web-Based Goal Management Training

One of the objectives of this study was to assess the feasibility of delivering WebGMT, a group-
based cognitive rehabilitation intervention using a web-based platform. Although online real-
time caregiver support groups have been organized (Marziali, 2006), as well as one-on-one
videoconference cognitive behavioural therapy (Bouchard, et al., 2004), to date no group-based
cognitive therapeutic intervention has been delivered using the web. This study is the first to
implement WebGMT using the Adobe Connect videoconferencing platform, thereby allowing
participants to access this rehabilitation intervention without the accessibility barriers described
earlier. To assess feasibility, two semi-quantitative-qualitative tools were developed: the
adherence-to-protocol scale for GMT and participant feedback surveys.

In Chapter 3, I provided an in-depth explanation of the adherence-to-protocol scale and the

changes that were made in order to transform the instrument from a cognitive behavioral therapy
scale to a cognitive rehabilitation therapy scale for GMT (CRTS-GMT). In analyzing data using
 52

this tool, it is crucial to highlight each feature examined and the importance it carries in assessing
various aspects of GMT. Important features of this cognitive rehabilitation intervention include
therapist adherence to GMT module outlines and structure, as well as participant feedback and
effective time utilization. Each of these were assessed using the CRTS-GMT across two raters,
who scored higher than the average score of three. In addition, other features such as group
collaboration, therapist skills, participant engagement and homework setting were looked at and
across sample sessions also scored higher than the average score. The threshold total score for
the CRTS-GMT for each session was placed at 27, above which competence was established.
For both raters familiar with the GMT protocol, the total scores exceeded 27, providing sufficient
preliminary data to indicate no significant compromises in the assessed features of the group
intervention.

Taking this analysis a step further, the WebGMT session recordings were also compared to in-
class GMT sessions using a very similar protocol (see Appendix C). The differences between the
items and total CRTS-GMT scale scores across both platforms of delivery did not differ
significantly, further establishing that no crucial aspects of therapy were compromised. The
percent at which WebGMT scores were higher than traditional in-class GMT ranged between
55% and 83% across two sample sessions, demonstrating that the WebGMT excelled at specific
aspects of therapy. This difference may have been due to variability in therapist and populations
of the WebBased groups versus the in-class GMT groups.

The feedback questionnaire, used to establish participant satisfaction with the therapy and
technology, showed positive responses to questions specific to the learning experience using the
Adobe Connect platform. In addition, five of the 8 participants indicated that they would prefer
using a web-based platform in receiving the intervention.

5.4.1 Limitations and Future Directions

This qualitative data shows that the novel web-based approach to the delivery of GMT is
promising and requires future research focusing on qualitative methods of analyzing this
approach. However, it is important to recognize limitations of this study that may help in
directing future projects using similar approaches. For example, though raters were given
specific instructions that the score of six on CRTS-GMT items should be a rare occurrence (i.e.,
only when there is no chance for improvement) many rating were still at ceiling. This Adherence
 53

to Protocol instrument, specifically developed for this study, requires further validation to
determine its utility as a tool for assessing the quality and consistency of intervention delivery,
whether across delivery platforms (as described here), training sites or therapists. More detailed
instructions to raters including a rating manual and pre-rating training sessions would improve
the quality and consistency of ratings. As described above, an RCT design, contrasting adherence
ratings across in-class and WebGMT sessions with similar patient populations and therapist
teams, would be necessary to provide a full validation of the consistency and utility of the rating
tool.

5.5 Summary and Conclusions

This research study explored a novel approach for delivering GMT using a real-time
videoconferencing platform. There were two primary study objectives: to investigate the
feasibility of delivering GMT using this web-based approach and to examine the efficacy of the
WebGMT approach vis-à-vis past research examining GMT efficacy. To address these two
research aims, semi-quantitative-qualitative assessment tools were developed and
neuropsychological as well as functional outcome measures were administered following a
within-subject, longitudinal experimental design. Using an adherence-to-protocol tool developed
for this study, the preliminary qualitative data confirmed that WebGMT was feasible in a
population of cerebrovascular risk patients with reported executive control deficits. Moreover,
preliminary results suggest that GMT can be delivered remotely without compromising
rehabilitation outcomes. This preliminary, proof-of-concept, study has established the empirical
foundation and research tools necessary for larger, randomized controlled trials. These
prospective studies are necessary to provide concrete evidence supporting feasibility and efficacy
delivering group-based cognitive rehabilitation therapy remotely using a web-based platform.
 54

References

Alexander, M. D., & Stuss, D. T. (2003). Executive disorders. In E. J. Noseworthy, Neurological

Therapeutics, Principles and Practice (Vol. 2, pp. 2808-2809). London: Martin Dunitz.

Alford, B., & Beck, A. T. (1997). The integrative power of cognitive therapy. New York:
Guilford Press.

Arundine, A., Bradbury, C. L., Dupuis, K., Dawson, D. R., Ruttan, L. A., & Green, R. E. (2012).
Cognitive behavior therapy after acquired brain injury: maintenance of therapeutic benefits at 6
months posttreatment. The Journal of head trauma rehabilitation , 27 (2), 104-12.

Baddeley, A. D. (1986). Working Memory. Oxford: Clarendon Press.

Baddeley, A. D., & Hitch, G. (2000). Development of working memory: Should the Pascual-
Leone and Baddeley and Hitch models be merged? ournal of Experimental Child Psychology ,
77, 128-137.

Bergquist, T., Gehl, C., Holzworth, N., & Beaulieu, W. (2008). Internet-based cognitive
rehabilitation in individuals with acquired brain injury: A pilot feasibility study. Brain Injury ,
22(11), 891-897.

Bilbao, A., Kennedy, C., Chatterij, S., Ustun, B., Luis, J., Barquero, V., et al. (2003). The ICF:
Applications of the WHO model of functioning, disability and health to brain injury
rehabilitation. NeuroRehabilitation , 18, 239-250.

Black, S. E. (2007). Therapeutic Issues in Vascular Dementia: Studies, Designs and Approaches.
The Canadian Journal of Neurological Sciences , 34(1), S125-30.

Black, S., Gao, F., & Bilbao, J. (2009). Understanding White Matter Disease Imaging-
Pathological Correlations in Vascular Cognitive Impairment. Stroke , 40(1), S48-S52.

Blackburn, I. M., James , I. A., Milne, D. L., Baker, C., Standart, S., Garland, A., et al. (2001).
The Revised Cognitive Therapy Scale (CTS-R): Psychometric Properties. Behavioural and
Cognitive Therapy , 29, 431-446.
 55

Bouchard, S., Paquin, B., Payeur, R., Allard, M., Pivard, V., Fournier, T., et al. (2004).
Delivering Cognitive-Behavior Therapy for Panic Disorder with Agoraphobia in
Videoconference. Telemedicine Journal and e-Health , 10, 13-25.

Brown, J. (1958). Some tests of the decay theory of immediate memory. Quarterly Journal of
Experimental Psychology , 10, 12-21.

Brown, K. G., Rietz, T. A., & Sugrue, B. (2005). The Effects of Videoconferencing, Class Size,
and Learner Characteristics on Training Outcomes. Performance Improvement Quarterly , 18(1),
59-82.

Burgess, P. W., & Robertson, I. H. (2002). Principles of the Rehabilitation of Frontal Lobe
Function. In D. T. Stuss, & R. T. Knight, Principles of Frontal Lobe Function (pp. 557-572).
New York: Oxford University Press.

Burgess, P. W., Alderman, N., Evans, J. J., & Emslie, H. (1996). "The dysexecutive
questionnaire". In Behavioral Assessment of the Dysexecutive Syndrome. Bury St. Edmunds:
Thames Valley Test Company.

Burgess, P. W., Alderman, N., Forbes, C., Costello, A., Coates, L. M., Dawson, D. R., et al.
(2006). The case for the development and use of "ecologically valid" measures of executive
function in experimental and clinical neuropsychology. Journal of the International
Neuropsychological Society , 12, 194-209.

Chan, R. C., Shum, D., Toulopoulou, T., & Chen, E. Y. (2008). Assessment of executive
function: Review of instruments and identificaiton of critical issues. Archives of Clinical
Neuropsychology , 201-216.

Chen , A. J., Novakovic-Agopian, T., Nycum, T. J., Song, S., Turner, G. R., Hills, N. K., et al.
(2011). Training of goal-directed attention regulation enhances control over neural processing for
individuals with brain injury. Brain , 134(Pt 5), 1541-54.

Cicerone, K. D., Langenbahn, D. M., Braden, C., Malec, J. F., Kalmar, K., Fraas, M., et al.
(2011). Evidence-based cognitive rehabilitation: updated review of the literature from 2003
through 2008. Archives of physical medicine and rehabilitation , 92 (4), 519-530.
 56

Cicerone, K. D., Langenbahn, D. M., Braden, C., Malec, J. F., Kalmar, K., Fraas, M., et al.
(2011). Evidence-Based Cognitive Rehabilitation: Updated Review of the Literature From 2003
Through 2008. Archives of Psychical Medicine and Rehabilitation , 92, 519-530.

Damianakis, T., Climans, R., & Marziali, E. (2008). Social Workers’ Experiences of Virtual
Psychotherapeutic Caregivers Groups for Alzheimer’s, Parkinson’s, Stroke, Frontotemporal
Dementia, and Traumatic Brain Injury. Social Work with Groups , 31(2), 99-116.

Dawson, D. R., Anderson, N., Burgess, P., Cooper, E., Krpan, K. M., & Stuss, D. T. (2009).
Further development of the Multiple Errands Test: Standardized scoring, reliability and
ecological validity for the Baycrest Version. Archives of Physical Medicine & Rehabilitation , 90
(11), S41-S51.

Delis, D. C., Kaplan, E., & Kramer, J. H. (2001). Delis-Kaplan Executive Functioning System
(D-KEFS).

Dreyfus, H. L. (1989). The Dreyfus model of skull acquisition. In J. Burke, Competency based
education and training. London: Falmer Press.

Duncan, J. (1986). Disorganisation of behaviour after frontal lobe damage. Cognitive

Neuropsychology , 3(3), 271-290.

Duncan, J. (1986). Disorganization of behaviour after frontal lobe damage. Cognitive

Neuropsychology , 2, 271-290.

Duncan, J., & Stuss, D. T. (2000). Rehabilitation of executive functioning: an experimental-

clinical validation of goal management training. Journal of the International Neuropsychological
Society , 6(3), 299-312.

Duncan, J., Emslie, H., Williams, P., Johnson, R., & Freer, C. (1996). Intelligence and the frontal
lobe: the organization of goal-directed behavior. Cognitive Psychology , 30(3), 257-303.

Duncan, J., Parr, A., Woolgar, A., Thompson, R., Bright, P., Bishop, S., et al. (2008). Goal
neglect and Spearman's g: competing parts of a complex task. Journal of Experimental
Psychology , 137(1), 131-48.
 57

Fish, J., Evans, J. J., Nimmo, M., Martin, E., Kersel, D., Bateman, A., et al. (2007).
Rehabilitation of executive dysfunction following brain injury: "content-free" cueing improves
everyday prospective memory performance. Neuropsychologia , 45(6), 1318-30.

Frueh, B. C., Monnier, J., Yim, E., Grubaugh, A. L., Hamner, M. B., & Knapp, R. G. (2007). A
randomized trial of telepsychiatry for post-traumatic stress disorder. Journal of Telemedicine and
Telecare , 13, 142-147.

Germain, V., Marchand, A., Bouchard, S., Drouin, M., & Guay, S. (2009). Effectiveness of
Cognitive Behavioural Therapy Administered by Videoconference for Posttraumatic Stress
Disorder. Cognitive Behaviour Therapy , 38(1), 42-53.

Gilbert, S. J., & Burgess, P. W. (2008). Executive Function. Current Biology , 18, R110-114.

Gorelick, P. B., Scuteri, A., Black, S. E., DeCarli, C., Greenberg, S. M., Iadecola, C., et al.
(2011). Vascular Contributions to Cognitive Impairment and Dementia : A Statement for
Healthcare Professionals From the American Heart Association/American Stroke Association.
Stroke , 42, 2672-2713.

Gowan, J. A., & Downs, J. M. (1994). Video Conferencing Human-Machine Interface.

Information and Management , 27, 341-356.

Hachinski, V., Iadecola, C., Petersen, R. C., Breteler, M. M., Nyenhuis, D. L., Black, S. E., et al.
(2006). National Institute of Neurological Disorders and Stroke–Canadian Stroke Network
Vascular Cognitive Impairment Harmonization Standards. Stroke , 37, 2220-41.

Hart, T., & Evans, J. (2006). Self-regulation and goal theories in brain injury rehabilitation. The
Journal of Head Trauma Rehabilitation , 142-155.

Heart and Stroke Foundation. (2012). Retrieved November 15, 2012, from Heart and Stroke
Foundation Website: http://www.heartandstroke.com

Heaton, R. K., Temkin, N., Dikmen, S., Avitable, N., Taylor, M. J., Marcotte, T. D., et al.
(2001). Detecting change: A comparison of three neuropsychological methods, using normal and
clinical samples. Archives of Clinical Neuropsychology , 16, 75-91.
 58

Jacobson, N. S., & Truax, P. (1991). Clinical significance: a statistical approach to defining
meaningful change in psychotherapy-research. Journal of Consulting and Clinical Psychology ,
59, 12-19.

James , I. A., Blackburn, I. M., & Reichelt, F. K. (2001). Manual of the Revised Cognitive
Therapy Scale (CTS-R).

Jin, Y., Legge, S. D., Ostbye, T., Feightner, J. W., & Hachinski, V. (2006). The reciprocal risks
of stroke and cognitive impairment in an elderly population. Alzheimer's and Dementia , 171-
178.

Kennedy, M. R., Coelho, C., Turkstra, L., Ylvisaker, M., Moore Sohlberg, M., Yorkston, K., et
al. (2008). Intervention for executive functions after traumatic brain injury: a systematic review,
meta-analysis and clinical recommendations. Neuropsychological Rehabilitation , 18(3), 257-
299.

Levine, B. (1999). Methodology of frontal and executive function. Trends In Cognitive Sciences
, 3(1), 42.

Levine, B., & Downey-Lamb, M. (2002). Design and evaluation of intervention experiments. In
E. P.J. (Ed.), Neuropsychological Interventions: Clinical Research and Practice (pp. 80-104).
New York: Guilford Publications.

Levine, B., & Turner, G. R. (2004). Disorders of executive functioning and self-awareness. In
Cognitive and Behavioral Rehabilitation: From Neurobiology to Clinical Practice. Guilford
Press.

Levine, B., Freedman, M., Dawson, D., Black, S., & Stuss, D. T. (1999). Ventral frontal
contribution to self-regulation: Convergence of episodic memory and inhibition. Neurocase , 5,
263-275.

Levine, B., Manly, T., & Robertson, I. H. (2008). Goal Management Training: Administrator's
Manual. Toronto, Ontario: Baycrest Centre.
 59

Levine, B., Robertson, I. H., Clare, L., Carter, G., Hong, J., Wilson, B. A., et al. (2000).
Rehabilitation of executive functioning: an experimental-clinical validation of goal management
training. Journal of International Neuropsychological Society , 6(3), 299-312.

Levine, B., Schweizer, T. A., O'Connor, C., Turner, G., Gillingham, S., Stuss, D. T., et al.
(2011). Rehabilitation of Executive Functioning in Patients with Frontal Lobe Brain Damage
with Goal Management Training. Frontiers in Human Neuroscience , 5, 9.

Levine, B., Stuss, D. T., Winocur, G., Binns, M. A., Fahy, L., Mandic, M., et al. (2007).
Cognitive rehabilitation in the elderly: effects on strategic behavior in relation to goal
management. Journal of the International Neuropsychological Society , 13(1), 143-52.

Levine, B., Turner, G. R., & Stuss, D. T. (2008). Rehabilitation of frontal lobe functions. In W.
G. Stuss D.T., Cognitive Neurorehabilitation: Evidence and Application (pp. 464-486).
Cambridge: Cambridge University Press.

Likert, R. (1932). A Technique for the Measurement of Attitudes. Archives of Psychology , 140,
1-55.

Machtmes, K., & Asher, J. W. (2000). A meta-analysis of the effectiveness of telecourses in

distance education. American Journal of Distance Education , 14(1), 27-46.

Man, D., Soong, W., Hui Chan, T., & Hui Chan, C. (2006). A randomized clinical trial study on
the effectiveness of a tele-analogy-based problem-solving programme for people withacquired
brain injury (ABI). NeuroRehabilitation , 21, 205-217.

Marziali, E. (2006). Developing Evidence for an Internet-Based Psychotherapeutic Group

Intervention. Journal of Evidence-Based Social Work , 3, 149-165.

Marziali, E. (2009). E-Health Program for Patients with Chronic Disease. Telemedicine and e-
Health , 15(2), 1-6.

Marziali, E. (2008). The Design and Evaluation of e-Health Intervention Programs for Older
Adults. eHEALTH INTERNATIONAL JOURNAL , 6-13.
 60

Marziali, E., & Donahue, P. (2006). Caring for Others: Internet Video-Conferencing Group
Intervention for Family Caregivers of Older Adults With Neurodegenerative Disease. The
Gerontologist , 3, 398-403.

Marziali, E., Damianakis, T., & Donahue, P. (2006). Internet-Based Clinical Services: Virtual
Support Groups for Family Caregivers. Journal of Technology in Human Services , 24(2), 39-54.

Mattson, A. J., & Levin, H. S. (1990). Frontal lobe dysfunction following closed head injury. A
review of the literature. The Journal of Nervous and Mental Disease , 178(5), 282-291.

Moore, M., & Kearsley, G. (1996). Distance Education: A Systems View. Belmont, California:
Wadsworth.

Morland, L. A., Pierce, K., & Wong, M. Y. (2004). Telemedicine and coping skills groups for
Pacific Island veterans with post-traumatic stress disorder: A pilot study. Journal of
Telemedicine and Telecare , 10, 286-289.

Norman, D. A., & Shallice, T. (1980). Attention to Action: Willed and automatice control of
behaviour. In M. S. Gazzaniga (Ed.), Cognitive Neuroscience: A Reader (pp. 376-390). Malden,
Massachusetts: Blackwell Publishers Ltd.

Parsons, T. D., Notebaert, A. J., Shields, E. W., & Guskiewicz, K. M. (2009). Application of
Reliable Change Indices to Computerized Neuropsychological Measures of Concussion.
International Journal of Neuroscience , 119, 492-507.

Proal, E., Reiss, P. T., Klein, R. G., Mannuzza, S., Gotimer, K., Ramos-Olazagasti, M. A., et al.
(2011). Brain Gray Matter Deficits at 33-Year Follow-up in Adults With Attention-
Deficit/Hyperactivity Disorder Established in Childhood. Archives of General Psychiatry , 68
(11), 1122-1134.

Rees, L., Marshall, S., Hartridge, C., Mackie, D., Weiser, M., & Erabi Group. (2007). Cognitive
Interventions Post Acquired Brain Injury. Brain Injury , 21(2), 161-200.

Rockwood, K., Joyce, B., & Stolee, P. (1997). Use of Goal Attainment Scaling in Measuring
Clinically Important Change in Cognitive Rehbilitation Patients. Journal of Clinical
Epidemiology , 50 (5), 581-588.
 61

Ruwaard, J., Broeksteeg, J., Schrieken, B., Emmelkamp, P., & Lange, A. (2010). Web-based
therapist-assisted cognitive behavioral treatment of panic symptoms: A randomized controlled
trial with a three-year follow-up. Journal of Anxiety Disorders , 24, 387-396.

Ruwaard, J., Schrieken, B., Schrijver, M., Broeksteeg, J., Kekker, J., Vermeulen, H., et al.
(2009). Standardized Web-Based Cognitive Behavioral Therapy of Mild to Moderate
Depression: A Randomized Controlled Trial with a Long-Term Follow-Up. Cognitive Behavior
Therapy , 38(4), 206-221.

Schweizer, T. A., Levine, B., Rewilak, D., O'Connor, C., Turner, G., Alexander , M. P., et al.
(2008). Rehabilitation of Executive Functioning After Focal Damage to the Cerebellum. The
American Society of Neurorehabilitation , 22, 72-77.

Schweizer, T. A., Levine, B., Rewilak, D., O'Connor, C., Turner, G., Alexander, M. P., et al.
(2008). Rehabilitation of executive functioning after focal damage to the cerebellum.
Neurorehabilitation and Neural Repair , 22, 72-77.

Shallice, T., & Burgess, P. (1996). The domain of supervisory processes and temporal
organization of behaviour. Philosophical Transactions of the Royal Society of London B , 351,
1405-1412.

Shallice, T., & Burgess, P. W. (1991). Deficits in Strategy Application Following Frontal Lobe
Damage in Man. Brain , 114, 727-741.

Stevens, J. (1996). pplied multivariate statistics for the social sciences (3rd ed.). Mahwah, New
Jersey: Lawrence Earlbaum Associates.

Stucki, G., Ewert, T., & Cieza, A. (2002). Value and application of the ICF in rehabilitation
medicine. Disability and Rehabilitation , 24(17), 932-938.

Stuss, D. T. (2006). Frontal lobes and attention: Processes and networks, fractionation and
integration. Journal of the International Neuropsychological Society , 12, 261-271.

Stuss, D. T. (2006). Frontal lobes and attention: Processes and networks, fractionation and
integration. Journal of the International Neuropsychological Society , 12, 261-271.
 62

Stuss, D. T. (2007). Philosophical Transactions of the Royal Society B. In B. Miller, & J.

Cummings, The Human Frontal Lobes: Functions and Disorders (pp. 292-305). New York:
Guilford Press.

Stuss, D. T., & Alexander, M. P. (2000). Executive functions and the frontal lobes: a conceptual
view. Psychological Research , 63, 289-298.

Stuss, D. T., & Alexander, M. P. (2007). Is there a dysexecutive syndrome? Philosophical

Transactions of the Royal Society B , 362, 901-915.

Stuss, D. T., & Levine, B. (2002). Adult clinical neuropsychology: lessons from studies of the
frontal lobes. Annual Review of Psychology , 53, 401-433.

Sugrue, B. (2003). State of the industry: ASTD's annual review of U.S. and international trends
in workplae learning and performance. . Alexandria, VA: American Society for Training and
Development.

Turner, G. R., & Levine, B. (2008). Augmented neural activity during executive control
processing following diffuse axonal injury. Neurology , 71, 812-18.

van Hooren, S. A., Valentijn, S. A., Bosma, H., Ponds, R. W., van Boxtel, M. P., Levine, B., et
al. (2007). Effect of a structured course involving goal management training in older adults: a
randomised controlled trial. Patient Education and Counselling , 65, 205-213.

Verghese, J., Wang, C., Katz, M., Sanders, A., & Lipton, R. B. (2009). Leisure Activities and
Risk of Vascular Cognitive Impairment in Older Adults. Journal of Geriatric Psychiatry and
Neurology , 2, 110-118.

Vermeer, S. E., Prins, N. D., den Heijer, T., Hofman, A., Koudstaal, P. J., & Breteler, M. M.
(2003). Silent Brain Infarcts and the Risk of Dementia and Cognitive Decline. The New England
Journal of Medicine , 348, 1215-22.

Vrankrijker, K. (2003). The long way from the International Classification of Impairments,
Disabilities and Handicaps (ICIDH) to the International Classification of Functioning,
Disabilities and Handicaps (ICF). Disability and Rehabilitation , 25, 561-564.
 63

Webster, J., & Hackley, P. (1997). Teaching effectiveness in technology-mediated distance

learning. Academy of Management Journal , 40(6) , 1282-1309.

Wechsler, D. (1985). The Wechsler adult intelligence scale- revised. New York: Psychological
Corporation.

West, R., Ergis, A. M., Winocur, G., & Saint-Cyr, J. (1998). The Contribution of Impaired
Working Memory Monitoring to Performance of the Self-Ordered Pointing Task in Normal
Aging and Parkinson's Disease. Neuropsychology , 12, 546-554.

WHO. (2001). International Classification Functioning, Disability and Health (ICF). Geneva:
World Health Organization.
 64

Appendix A: Cognitive Rehabilitation Therapy Scale for

Goal Management Training – (CRTS-GMT)
65
66
67
68
69
70
71
72
73
74
 75

Appendix B: Participant Feedback Survey

Web-Based BTP- Participants Questionnaire

Please answer the following questions based on your experience as a participant in the Brain
Training Program Study.
1) Overall how easy or difficult did you find the use of Adobe Connect?
a. Very easy
b. Somewhat easy
c. Neutral
d. Somewhat difficult
e. Very difficult

2) Overall how helpful was the technical support offered (i.e manual provided and
Illia/Halla’s help)?
a. Very helpful
b. Somewhat helpful
c. Not very helpful
d. Not helpful at all
e. I didn’t need technical support

3) How helpful did you find adobe connect specifically in the learning experience?
a. Very helpful
b. Somewhat helpful
c. Not very helpful
d. Not helpful at all, I would rather not use it for this program

4) If you were to participate in this brain training program again, would you rather sign up
to:
a. A web-based group
b. An in-class group

What are some things that can be done to improve the online delivery experience?

Additional comments:
 76

Appendix C: WebGMT Groups 1 and 2 compared to In-

Class GMT on CRTS-GMT scale
Session 5 9
In-Class (IC) vs. Web-Based (WB) sessions IC WB1 WB2 IC WB1 WB2
1. Adherence to module outline 6 4 6 6 4.5 6
2. Feedback 5 6 6 5 6 5.5
3. Collaboration (verbal and non-verbal) 5 3 6 4.5 5 5
4. Pacing and efficient use of time 6 5 6 6 5 6
5. Interpersonal effectiveness 5 6 6 4.5 6 6
6. Conceptual integration 5 6 6 4.5 6 6
7. Therapist Skills 5 5 6 4.5 6 6
8. Homework Setting 6 5 6 n/a 6
9. Participant Engagement 5.5 5 6 5 5.5 5
Total (max. 54) 48 45 54 40 50 45.5
Comparing WebGMT to In-Class Group 1 Group 2
Percent of times Web-Based scores equal or 55% 83%
higher than In-Class session