Psychotherapy Research

ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/tpsr20

Can cognitive inflexibility reduce symptoms of

anxiety and depression? Promoting the structural
nested mean model in psychotherapy research

Henrik Børsting Jacobsen, Peter Solvoll Lyby, Thomas Johansen, Silje

Endresen Reme & Ole Klungsøyr

To cite this article: Henrik Børsting Jacobsen, Peter Solvoll Lyby, Thomas Johansen, Silje
Endresen Reme & Ole Klungsøyr (2023) Can cognitive inflexibility reduce symptoms of anxiety
and depression? Promoting the structural nested mean model in psychotherapy research,
Psychotherapy Research, 33:8, 1096-1116, DOI: 10.1080/10503307.2023.2221808

To link to this article: https://doi.org/10.1080/10503307.2023.2221808

© 2023 The Author(s). Published by Informa View supplementary material

UK Limited, trading as Taylor & Francis
Group

Published online: 11 Jul 2023. Submit your article to this journal

Article views: 1185 View related articles

View Crossmark data Citing articles: 1 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=tpsr20
Psychotherapy Research, 2023
Vol. 33, No. 8, 1096–1116, https://doi.org/10.1080/10503307.2023.2221808

RESEARCH ARTICLE

Can cognitive inflexibility reduce symptoms of anxiety and depression?

Promoting the structural nested mean model in psychotherapy research

HENRIK BØRSTING JACOBSEN1,2∗ , PETER SOLVOLL LYBY1, THOMAS JOHANSEN3,

SILJE ENDRESEN REME2, & OLE KLUNGSØYR4,5∗
1
CatoSenteret Rehabilitation Center, Son, Norway; 2The Mind-Body Lab, Department of Psychology, University of Oslo,
Oslo, Norway; 3Norwegian National Advisory Unit on Occupational Rehabilitation, Rauland, Norway; 4Institute of Clinical
Medicine, Faculty of Medicine, University of Oslo, Oslo, Norway & 5Department for Research and Innovation, Division of
Mental Health and Addiction, Oslo Centre for Biostatistics and Epidemiology, Oslo, Norway
(Received 29 June 2022; revised 30 May 2023; accepted 31 May 2023)

ABSTRACT
Objective To estimate the causal effect of executive functioning on the remission of depression and anxiety symptoms in an
observational dataset from a vocational rehabilitation program. It is also an aim to promote a method from the causal
inference literature and to illustrate its value in this setting.
Method With longitudinal (four-time points over 13 months) data from four independent sites, we compiled a dataset with
390 participants. At each time point, participants were tested on executive function and self-reported symptoms of anxiety
and depression. We used g-estimation to evaluate whether objectively tested cognitive flexibility affected depressive/anxious
symptoms and tested for moderation. Multiple imputations were used to handle missing data.
Results The g-estimation showed a strong causal effect of cognitive inflexibility reducing depression and anxiety and
modified by education level. In a counterfactual framework, a hypothetical intervention that could lower cognitive
flexibility seemed to cause improvement in mental distress at the subsequent time-point (negative sign) for low
education. The less flexibility, the larger improvement. For high education, the same but weaker effect was found, with a
change in sign, negative during the intervention and positive during follow-up.
Discussion An unexpected and strong effect was found from cognitive inflexibility on symptom improvement. This study
demonstrates how to estimate causal psychological effects with standard software in an observational dataset with
substantial missing and shows the value of such methods.

Keywords: executive functioning; anxiety; depression; cognitive flexibility

Clinical and methodological significance of this article: The results in the current study challenge the scientific consensus
of higher cognitive flexibility leading to improvement in symptoms of depression and anxiety. Estimating causal effects with a
structural nested mean model with g-estimation yields unbiased effect estimates when conventional methods are likely to fail.

Executive functions (EFs) are an umbrella term for & Friedman, 2012). They are generally thought to
top-down regulation central to our emotional, cogni- encompass inhibitory control and interference
tive, and behavioral control (Diamond, 2013; Miyake control, working memory, and cognitive flexibility

∗
HBJ and OK contributed equally to the manuscript.
Correspondence concerning this article should be addressed to Henrik Børsting Jacobsen, The Mind-Body Lab, Department of Psychology,
University of Oslo, Oslo, Norway. Email: henrbors@uio.no
This article has been corrected with minor changes. These changes do not impact the academic content of the article.

© 2023 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/
licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly
cited. The terms on which this article has been published allow the posting of the Accepted Manuscript in a repository by the author(s)
or with their consent.

(Miyake & Friedman, 2012). EFs can uniquely predict
fundamental aspects of adult life such as complex
reasoning, literacy, and education from as early as 2
years of age (Mulder et al., 2017), and are closely
linked to general functioning and work life (Huizinga
et al., 2018). Recent trials of multidisciplinary rehabili-
tation indicate an important role of EF capacity in
return-to-work, quality of life, and overall functioning
(Aasvik et al., 2015; Aasvik et al., 2017; Jacobsen
et al., 2016; Johansen et al., 2019), but the “why”
and the “how” of this relationship remain unclear.
A viable candidate for illuminating this relation-
ship is understanding the impact from symptoms of
anxiety and depression on EFs. A large body of
research associates mood disorders with poor EFs
(Snyder, 2013; Snyder et al., 2015), and EF capacity
in the form of indecisiveness and distractibility has
been included in the DSM-5 classification of mood
disorders (First et al., 2021). Several influential
psychological models echo the DSM by proposing
EF deficits as early signs of depression and anxiety
(Fresco et al., 2007; Hayes et al., 2006; Nolen-Hoek-
sema, 2011; Price & Duman, 2020; Teasdale et al.,
2002; Wells, 2011; Wells & Matthews, 1996). As
such, deficits in EFs are often conceptualized as
transdiagnostic risk factors for developing mood dis-
orders (Liu et al., 2021; Snyder et al., 2015).
However, while EFs have been shown to prospec-
tively predict mood disorders (Caspi et al., 2020; Let-
kiewicz et al., 2014; Mac Giollabhui et al., 2019), and
cognitive inflexibility appears particularly salient (Mac-
Pherson et al., 2021; Stange et al., 2016; Stange et al.,
2017), the relationship is not straightforward (Fu
et al., 2021). The nature and role of EF deficits vary
in psychopathology, as well as within different sub-
classifications of mood disorders (Snyder et al., 2015).
When comparing patients with depression to healthy
controls, systematic reviews and meta-analyses
observe deficits across attention, executive and
memory measures, with no evidence of specific
domains being affected more than others (Rock et al.,
2014). To further complicate the picture, EFs can
also deteriorate because of mood disorders, and
symptom remission has been documented to happen
independently of improvements in EFs (Porter et al.,
2015; Torrent et al., 2012).
Moreover, there are only a handful of studies
looking at the relationship between treatment
effects, symptom remission, and EF deficits, and
these show conflicting results (Porter et al., 2015).
In bipolar patients, poor performance on verbal
recall has been associated with symptom severity
after cognitive–behavioral group therapy (Sachs
et al., 2020). On the other hand, following anti-
depressant therapy, improvements in depression
and anxiety can happen without improvements in
Psychotherapy Research 1097
EFs (Devanand et al., 2003; Levkovitz et al.,
2002). Moreover, such treatment effects are strongly
modified by age and years of education (Gudayol-
Ferré et al., 2021). In summary, it is not yet estab-
lished whether EFs deficits drive, predict, or are
caused by psychopathology, or if improvements in
EFs from therapies depend on age or level of edu-
cation (Gudayol-Ferré et al., 2021). Although our
knowledge of EFs in mood disorders has grown, a
major drawback is that the prospective role of EF def-
icits in mood disorders is poorly understood (Fu
et al., 2021).
A formal framework that provides technical nota-
tion to conceptualize causation can give insight in
this relationship. Such a framework is that of “poten-
tial outcomes” or “counterfactuals”, employed in a
wide variety of disciplines to assess causality. If an
outcome would have differed had some exposure or
action been other than it was, then the exposure or
action can be said to cause the outcome. In statistics,
a formal notation for the counterfactual approach was
described by Neyman (1923), and later developed and
extended to observational studies by Rubin (1974,
1978). Robins extended it further to include multiple
and time-varying exposures (1986), and it was related
to a graphical representation of causality by Spirtes
et al. (1993) and Pearl (1995, 2009).
An Introduction to Robins’ g-methods
Even in the perfectly designed randomized con-
trolled trial (RCT), the treatment comparison can
be a biased estimate of the “true” treatment effect
due to non-adherence or differential loss to follow-
up. Moreover, conducting an RCT is often not feas-
ible, leaving the causal question unanswered. The
overwhelming amount of research questions of a
causal nature in the health sciences has inspired the
development of methods for causal inference for
any study design to come as close as possible to the
“true” causal effect, with as few and mild assump-
tions as possible. In epidemiology, causal questions
have been assessed from observational studies as
RCTs are hard to come by, and progress in method-
ology has benefitted other fields. Identification and
estimation of causal effects including underlying
assumptions, e.g., effect of a time-varying exposure,
treatment, or action on an outcome has been
studied in detail with a variety of applications
(Herńan & Robins, 2020), but might still need justi-
fication in psychological research. From here on, the
variable of which the causal effect is of interest is
called the “action” variable, to represent the specific
action itself or a hypothetical intervention that could
change the action variable.
1098 H. B. Jacobsen et al.
Figure 1. Causal diagram (DAG) representing the relation
between a time-varying exposure (A0 , A1 ), a confounder (L1 ) for
the relation between A1 and Y and an unmeasured confounder
for the relation between L1 and Y .
In observational studies of a time-varying action,
time-varying confounding is always a potential chal-
lenge. This is commonly illustrated in epidemiology
with a causal diagram, a so-called causal-directed
acyclic graph (causal DAG) (Pearl, 1995). In a
causal DAG, with arrows representing direct causal
effects, a statistical association between an action
variable and an outcome can only be produced by
the following three causal structures: (i) a true
cause and effect, (ii) common cause (confounding):
if the action and the outcome share a common
cause that is not adjusted for, and (iii) common
effects (selection/collider bias): if the action and the
outcome have a common effect, or a descendant of
a common effect, that is adjusted for.
If the time-varying confounder is itself affected by
earlier actions, conventional methods for confounder
control (by stratification/adjustment in a regression)
are flawed (Figure 1). With interest in the joint
effect of A0 and A1 on Y (Daniel et al., 2013), adjust-
ing for L1 in a standard regression model would block
part of the effect of A0 (through A1 ) and potentially
lead to bias of the effect of A0 through the unmea-
sured confounder U for the L1 − Y relationship (L1
becomes a collider).
Robins’ generalized methods (g-methods) were
developed for identification and estimation of such
effects. It’s a family of methods that include the g-
formula / g-computation, g-estimation for Structural
Nested Models (SNMs) (Robins et al., 1992;
Robins, 1994, 1997; Vansteelandt & Joffe, 2014;
Vansteelandt & Sjolander, 2016), and inverse prob-
ability weighting (IPW) for Marginal Structural
Models (MSM) (Robins et al., 1999). In the counter-
factual framework, a marginal causal effect is defined
by the theoretical contrast between the mean if every-
body in the population was exposed to the action at
both timepoints versus the mean if everybody was
not exposed to the action at both timepoints.
Assumptions on which the g-methods rely form the
link between this hypothetical effect and the
observations:
1. “Counterfactual consistency” allows us to
equate the observed outcome to the counter-
factual outcome that would be observed
under the observed action level. This equality
should hold, regardless of the nature of the
assignment mechanism (e.g. whether it was
randomized or observational). If, for
example, there are multiple versions of the
action (let’s say, therapy with different
content) which might give rise to different
outcomes depending on which version is
administered, this renders the counterfactual
outcome not well defined, and consistency is
violated (VanderWeele & Hernán, 2013).
2. “Sequential conditional exchangeability” is
the same as “no unmeasured confounding”,
in the time-varying case. It implies that the
counterfactual outcomes are independent of
the observed action levels, conditional on
past covariates and action levels (in Figure 1
it corresponds to no unmeasured variables
that affect A0 and Y , or A1 and Y ). From
here on, the term “no unmeasured confound-
ing” is used and is meant for every time point.
3. “Positivity” is met when there are individuals
in all levels of the action variable, within all
confounder and prior action levels (not
needed for g-estimation). This implies well-
defined and positive probabilities for each
action level.
Under these assumptions, the hypothetical obser-
vational study in Figure 1 equals a sequentially ran-
domized trial in which the action level was
randomized at baseline and randomized again at
time 1 with probability depending on L1 (Naimi
et al., 2017). Of the three g-methods, IPW esti-
mation for MSMs is by far the most popular
method (and newest). It appears much simpler
than the two others, and is easy to implement in stan-
dard software. However, the less known method of g-
estimation outperforms IPW in several ways, it’s
almost always more efficient, more robust for unmea-
sured confounding, particularly well-suited for a con-
tinuous action variable, can accommodate
moderation by time-varying covariates and does not
require the positivity-assumption (Vansteelandt &
Sjolander, 2016). G-computation is even more effi-
cient, but highly computer-intensive with many para-
metric assumptions (distributional assumptions for
confounders etcetera) and can also not handle mod-
eration by time-varying covariates.
Structural nested mean models (SNMMs) solve
the problems in the conventional confounder-
control (stratification/adjustment in regression), by
avoiding the adjustment on post-action variables
(adjustment for L1 in Figure 1). This is achieved by
modeling the outcome at each time conditional on
the action variable and covariate history up to that

time; after having removed the effects of subsequent
action variables to disentangle the unique contri-
butions of each action at each time (Vansteelandt &
Joffe, 2014). The parameters in the SNMMs are esti-
mated with g-estimation and “nested” refers to a
conceptualization of longitudinal data as resulting
from a nested series of trials (Picciotto & Neophytou,
2016). Starting with the last time-point (with no
post-action variables), the action–outcome relation
is estimated and adjusted for past actions and covari-
ates. Iteration for each prior time-point after having
“removed” the effect of the later action variable on
the outcome ends with a counterfactual outcome
for everyone being predicted under an “action-free”
regime. Combined with the no unmeasured con-
founding assumption (for every time-point), the g-
estimation finds the counterfactual outcome (with
the action removed) that satisfies the independence
assumption between this counterfactual and the
observed action variable. This counterfactual is a
function of the causal action effect through par-
ameters in the SNMM, and inversion of this function
yields the g-estimated parameters.
The limited popularity of g-estimation has largely
been related to a lack of implementation in standard
software. In 2016, Vansteelandt and Sjolander
described how to obtain the g-estimator for a linear
SNMM by combining separate regression models
for the outcome and the action variable (Vanstee-
landt & Sjolander, 2016). If either of these models
(but not both) are mis-specified, the g-estimator for
the causal parameters in the SNMM is still consistent
(provided the SNMM is also correct), a property
which is called “double-robust.” This implies
indirectly some protection against unmeasured con-
founding in the outcome model (one form of misspe-
cification), also shared by the instrumental variable
approach (Ertefaie et al., 2017; Joffe & Brensinger,
2003). It can be shown that the g-estimator is equiv-
alent to an instrumental variable (two-stage least
squares) estimator (Joffe & Brensinger, 2003;
Naimi et al., 2017). Double-robustness means that
consistent causal parameters in the SNMM are
obtained even if the model for the action variable,
or for the outcome (but not both) has some form of
misspecification. This property can be exploited in
the model fitting, with one model held constant
while varying the other. Large variation in the
causal parameter estimates indicates that the model
which was held constant is wrong. In this way,
double-robustness can guide model selection
(Wallace et al., 2016).
More applications of g-estimation have been called
for (Vansteelandt & Joffe, 2014; Vansteelandt & Sjo-
lander, 2016). In two subfields of epidemiology there
has been an increase in applications in recent years,
Psychotherapy Research 1099
(Picciotto & Neophytou, 2016) which should easily
translate to psychotherapy research and other
mental health therapies. In pharmacoepidemiology,
g-estimation has been applied to adjust for non-
adherence and to determine optimal personalized
strategies, which has increased interest in the area
of dynamic treatment regimens (Tsiatis et al.,
2020). A dynamic treatment regime is a sequence
of interventions and of interest in all personalized
treatment of chronic diseases, including many
mental disorders. In occupational epidemiology, g-
estimation has been used to adjust for the so-called
“healthy worker survivor bias,” which creates the
illusion that an unhealthy exposure is protective,
because the workers of poorer health are more
likely to reduce their exposure. This is also seen in
psychotherapy research and the mental health field,
where patients with more severe symptoms tend to
illicit more frequent and comprehensive treatment,
thereby generating an inverse association between
treatment and outcome. G-estimation is appropriate
to adjust for such feedback.
Even though the g-estimation can be implemented
in standard software, and the regression models for
the action variable and for the outcome are flexible
(within standard limitations for regression models),
there is a lack of flexibility for link functions other
than identity or log link (Vansteelandt & Sjolander,
2016). The implementation of log-linear SNMMs
for counts and similar models for survival endpoints
can be informed by the development of the linear
SNMM (Vansteelandt & Sjolander, 2016). Based
on semi-parametric theory (Tsiatis, 2020), Robins
formulated the g-estimator as the solution of an
unbiased estimating equation, without distributional
assumptions (consistency under mild regulatory con-
ditions) (Robins, 1994).
Traditionally, causality in epidemiology and psy-
chology has been assessed in structural equation
models (SEMs). Even though some of the more
recent causal inference methods have developed out
of the SEM literature, e.g. causal graphs (DAGs),
traditional SEMs have many limitations with
respect to causal inference when, e.g. the effect of
an action on an outcome, or mediation is of interest
(Rijnhart et al., 2021 VanderWeele, 2012). SEMs
tend to make more assumptions, like linearity in
functional form among covariates and multivariate
normal distribution. In fact, in a regression model,
internal relations among covariates are not
modeled, only the relation between the action vari-
able and outcome (covariates play the role of con-
founders or moderators). This means that bias from
unmeasured confounding is more likely in a multi-
variate SEM, since any common cause of two vari-
ables in a causal graph (representing the SEM) also
1100 H. B. Jacobsen et al.
must be included (VanderWeele, 2012). The usual
assumption of uncorrelated errors in an SEM
would be violated in Figure 1, and therefore
indirectly implies an assumption of no unmeasured
confounding for both the relationship between A0 -
Y, A1 - Y and L1 – Y (De Stavola et al., 2015).
One reason for the popularity of the SEM model in
psychology is that it can be extended to account for
measurement error, within restrictions of normality
and uncorrelated errors. A realistic causal DAG is a
general and useful way of depicting different types
of measurement errors. It should simultaneously rep-
resent biases arising from confounding, selection
(collider-bias) and measurement error, and thereby
reveal how (if possible) to correct for these (Herńan
& Robins, 2020). Sensitivity analysis techniques for
direct and indirect effects in causal mediation have
been developed, for unmeasured confounding as
well as for the presence of measurement error
(VanderWeele, 2015).
The main aim of this study is to demonstrate how
the structural nested mean model with g-estimation
can be used to disentangle causal pathways in an
observational longitudinal dataset of mental health
variables with some design limitations. The causal
effect of a hypothetical intervention that could
change cognitive flexibility on remission of
depression and anxiety symptoms over a 12-month
period is assessed. As EF capacity is strongly
related to socio-demographics such as age and edu-
cation, these were, in addition to coping style, con-
sidered as candidate moderators (VanderWeele,
2015). It was also an aim to see whether symptoms
of depression and anxiety was associated with differ-
ences in baseline EFs or long-term return-to-work
for our participants.
Methods
Setting
Individuals completing either inpatient or outpatient
multidisciplinary occupational rehabilitation pro-
grams based on multimodal cognitive behavior
therapy were recruited from four clinics, alongside
healthy controls. The duration of the rehabilitation
programs varied between the clinics from 3 to 12
weeks.
Participants
A total of 390 participants were recruited for this
study. Of these 317 were patients who were either
on partial or full sick leave, volunteered to take part
in the study as well as 73 healthy volunteers who
were currently working full time. A total of 187 par-
ticipated in an inpatient occupational rehabilitation
program and 93 in an outpatient program. Seventy-
three workers in the control group who volunteered
to take part were all working full time and had no
sick leave during the testing period. They were
recruited from the wider community and employees
from three rehabilitation clinics and included a
wide selection of different blue- and white-collar
workers. The two groups were matched for age,
sex, and number of days between pre- and post-
tests. Of participants in rehabilitation, 80% had an
ICD-10 diagnosis either in the categories F, mental
and behavioral disorders or M, diseases of the mus-
culoskeletal system and connective tissue. Exclusion
criteria for the rehabilitation and control group were
a history of head injury or having applied for disabil-
ity pension. The flow of participants is illustrated in
Figure 2.
Design
This study had an explorative, non-randomized pre–
post design. All participants in the study were
assessed with cognitive and emotional tests and
work and health questionnaires on four time points
spanning a 13-month period and are included in
the estimation of effects irrespective treatment
groups.
Multidisciplinary Return-to-Work (RTW)
Rehabilitation Program
The patients were referred to occupational rehabili-
tation by general practitioners or social security
offices. The main aim of rehabilitation was RTW,
and the programs lasted between 3 and 12 weeks.
The patients were followed up by an interdisciplinary
team including at least four of the following pro-
fessionals: physician, physiotherapist, psychologist,
work consultant, coach, nurse/psychiatric nurse,
and sports pedagogue. The assessment of work
ability, physical fitness, and current work and
health situation was carried out to tailor rehabilita-
tion efforts.
Materials
Computerized Cognitive and Emotional
Tests (Exposure Variables)
Eight validated tests from the Cambridge Neurop-
sychological Test Automated Battery (CANTAB)
were used to assess cognitive and emotional func-
tioning on a touch screen. The order of the tests

Figure 2. Time course of cognitive flexibility (IEDt ) and mental distress (depression/anxiety symptoms) by HADDt , HADAt and HADTt
(with f (t) from Equation 1 included).
was fully counterbalanced across participants at
each testing session and within each group. All par-
ticipants were introduced to the touch screen by way
of a motor screening task performed before testing
both at pre- and post-tests. This screening was per-
formed to familiarize the participants with the touch
screen and reduce as much as possible any initial
apprehension before testing. A description of com-
puterized cognitive and emotional tests is given in
brief below.
Rapid Visual Information Processing, Spatial
Working Memory, Spatial Recognition Memory,
Stockings of Cambridge (a version of the Tower of
London task measuring executive planning),
Emotion Recognition Task. All tests were adminis-
tered on a touch-sensitive computer screen.
Cognitive Flexibility (Primary Action
Variable)
Intra-Extra Dimensional Shift (IED) tests the par-
ticipants’ cognitive flexibility and is a computerized
version of the Wisconsin Card Sorting Test (Grant
& Berg, 1993). It measures a person’s ability to
acquire and reverse rules and taps into lower-
order cognitive functions such as discriminating
between visual stimuli and flexibly maintaining
and changing an attentional set (Cambridge Cogni-
tion, n.d.). On the screen, four white boxes appear,
and a stimulus is shown inside two of them. The
Psychotherapy Research 1101
stimuli are composed of color-filled shapes and
white lines. The participant is asked to select a
stimulus at random. Following a selection, the par-
ticipants get feedback on their answer—correct or
incorrect. The task is to discover the rule determin-
ing which stimulus is correct. We chose IED total
errors adjusted as our action variable. IED total
error is a summation of all errors made at different
stages. For those that fail a stage and therefore are
not given a chance to attempt further stages, an
additional sum of 25 is added for each stage not
attempted. A lower score represents better perform-
ance (higher flexibility).
Depressive and Anxious Symptoms
(Response Variable)
The 14-item Hospital Anxiety and Depression Scale
(HADS) measures mental distress and is divided into
an anxiety and a depression scale, each with 7 items
(Zigmond & Snaith, 1983). Twenty-one is the
maximum score on each scale. The cut-off for
mental distress is usually set at 7 or above on either
the anxiety (HAD-A) and/or the depressive scales
(HAD-D) (Wu et al., 2021) and this is validated
for a Norwegian population (Bjelland et al., 2002).
In the current sample, the HADS showed satisfactory
internal consistency (Cronbach’s α = 0.89) at
baseline.
1102 H. B. Jacobsen et al.
Moderators, Time-Varying
Theoretically Originated Measure of the
Cognitive Activation Theory of Stress
(TOMCATS)
TOMCATS is a measure designed to measure the
concept of response outcome expectancies as
defined in the Cognitive Activation Theory of
Stress (CATS) [2, 18]. In this context, it was pro-
posed as a moderator between the unmeasured
effects of stress on executive functioning. Response
outcome expectancies are thought to drive, or limit
sustained stress activation, thus potentially moder-
ating the adverse effects of cortisol on brain areas
involved in EFs, thus acting on depressive
symptoms.
The inventory consists of three factors that rep-
resent the three response outcome expectancy
dimensions of CATS: positive expectancy (one
item), no expectancy (two items) and negative
expectancy (three items). All items are rated on a
four-point scale from “not true at all” to “completely
true.” For this study, we only used the negative
expectancy item that has been associated with
depressive symptoms, namely an expectancy of hope-
lessness (TChope) and helplessness (TChelp). In the
current sample, the TOMCATS showed satisfactory
internal consistency (Cronbach’s α = 0.81) at
baseline.
Non-linear Decreasing Magnitude Over
Time
In treatment-effect trials, effects often decrease over
time. In the present study, moderation from a non-
linear function of time f (t) with a similar shape as
the outcome proved a good fit and is given by
(similar to the Box–Cox transformation):
⎧ l
⎨ (365 − t l )/l l=0


f (t) = 365 (1)
⎩ log l=0
t+1
where l . 0 ( , 0) yields a function that decreases
less (more) rapidly than the negative log(t), l = 1
yields a linear function of time (in days).
Moderators, Time Constant
Education Levels
For this analysis, education was divided into two
levels: low = up to high school (<13 years of edu-
cation) and high = college and university degree
(>13 years of education).
Statistical Analyses
Baseline demographics for selected variables were
analyzed using means and standard deviations for
continuous variables or percent for categorical. A
comparison of average percent sick leave over 12
months and EFs on baseline was tested for education
using independent sample t-tests.
Causal Effects
The counterfactual approach is used to assess the
causality between cognitive flexibility and subsequent
depressive/anxiety symptoms. In terms of the effect of
an action on an outcome, the counterfactual outcome
Y (a) is the potentially unobserved outcome for the
hypothetical action level a. In the following, random
variables and their realizations are represented by
upper- and lower case letters, respectively.
A conditional average causal effect of a hypotheti-
cal intervention setting the action variable to a versus
0 (0 can represent action-free or any reference value)
among subjects with covariate value l, can be formu-
lated by the linear structural mean model (SMM)
(Robins, 1994; Vansteelandt & Joffe, 2014; Vanstee-
landt & Sjolander, 2016) as
E(Y (a) − Y(0)|l) = c′ za, where E(.|.) is the con-
ditional population mean, z is a covariate vector
possibly depending on l, and c is the vector of
causal parameters of interest.
For example, with l representing gender and z = 1,
the additive action effect on an outcome from action
level a relative to no action, is equal between genders.
On the other hand, z = l would describe different
action effects between men and women.
With time-varying action variable At , vector of cov-
ariates Lt and outcome Yt , observations are made at
time t = 1, 2.. with the history up until (and including)
t, denoted by A
t and L
2 = {A1 , A2 }.

t , for example A
Ys (

at , 0) is the counterfactual outcome at time
s = 1, 2 . . ., with s . t, for an action history equal to

t up until time t and zero there – after. This construct
a
facilitates assessments of a causal effect of a time-
varying action on the following outcome as well as on
all subsequent ones, formulated by the structural
nested mean model (SNMM) (Robins, 1994):
at , 0) − Ys (

E(Ys (
at−1 ,
l t ) = c′ zst at
at−1 , 0)|
(2)
The action effect posited in (2) represents the differ-
ence in outcome when the action level at time t is set
equal to at and 0 thereafter, relative to when action
level at time t and onward is set equal to 0, conditional
on action and covariate history. In other words, the
contribution to the action effect from a specific time-
point (so-called “blip”) when later contributions are

“removed.” This effect is identifiable and can be esti-
mated from observed data under the usual causal
assumptions of consistency and no unmeasured con-
founding (see Introduction).
The causal effects to be assessed in the present
application are the effects of a hypothetical interven-
tion that could change cognitive flexibility (IEDt ) at
time t, on subsequent depression and anxiety symp-
toms, by HADDs , HADAs , and HADTs , s . t, and
with potential effect-modifiers TChopet and TChelpt .
Different causal hypotheses can easily be assessed
with different choices of zst in (2) (Appendix),
although lack of power is a limitation with increased
model complexity. A simple model with moderation
from a time-fixed covariate and a non-linear func-
tion of time proved to have a good fit in the
present application (with HADDs as outcome),
and is given by
E(HADDs (ied t , 0) − HADDs (ied t−1 , 0)|ied t−1 ,
l t )
= c′ zst iedt = (c0 + c1 HighEdu + c2 f (t))iedt
(3)
for = 2, 3, 4 s . t, and with f (t) from (1).
Following Vansteelandt and Sjolander, fitting the
SNMM for the causal effect of the time-varying
IEDt on HADDs , HADAs , and HADTs , s ., can
be done in three steps, by combining ordinary
regression models for IEDt , one model for each
time-point (e.g. linear regression) and a longitudinal
model for the outcome, including the SNMM—
terms, fitted and refitted (Appendix)
I. Regressing the action variable IEDt on its
history, baseline, and preceding time-
varying covariates and outcome for each,
including non-linearities and interactions.
With data in a “wide” format, these models
could for example be fitted with the linear
model command in the statistical software R
(R Core Team, 2014): “lm(IEDt  . . .)”
(OLS regression) for t = 1, 2, 3, with the
fitted values (called “propensity scores”),
saved and denoted P1 , P2 , and P3
respectively.
II. Regressing the observed outcome, e.g.,
HADDt , on covariates and moderators as
well as the SNMM terms with previous
action (1, HighEdu, f (t − 1))iedt−1 and pro-
pensity score (1, HighEdu, f (t − 1))pt−1 in a
repeated measures regression model, by
forming a dataset in “long” format and for
example fitting the model with the GEE
command in R: “geem((HADD2 , HADD3 ,
HADD4 )  . . . )” (independence working
Psychotherapy Research 1103
correlation). The (1, HighEdu, f (t − 1))iedt−1
coefficients are the preliminary SNMM par-
ameter estimates ĉ(0) = (ĉ(0) 0 , ĉ1 , ĉ2 ) in (3).
(0) (0)
III. Predicting Ys (A
t , 0) for all t, s with s . t by
means of ĉ(0) (removing subsequent effects
from the observed outcome, see appendix),
denoted H31 , H42 , and H41 . The updated
and improved ĉ(1) is found by a second inde-
pendence GEE with these predictions as out-
comes in an extended data set (long format),
and the same covariates as above, for example
fitted by the R command: “geem
((HADD2 , HADD3 , HADD4 , H31 , H42 ,
H41 )  . . . )” and again with the SNMM par-
ameter estimates ĉ(1) = (ĉ(1) 0 , ĉ1 , ĉ2 ) as the
(1) (1)
(1, HighEdu, f (t))iedt coefficients. Standard
errors are estimated by bootstrap.
In the outcome model (steps II and III), f (t) was
included as one component in lt with
l = 0.2, 0.1, 0.4 giving good fit for HADDt ,
HADAt , and HADTt respectively as a function of
days (Figure 3).
The steps above describe how g-estimation can be
achieved by the simple combination of standard
regression models. To achieve an automatic model
selection and identify the “best” model for each boot-
strap sample (and thereby precise bootstrap standard
errors), the OLS regression in step I, can be replaced
by for example an automatic lasso-regression algor-
ithm (as in the present application where the R-
package glmnet was used) (Friedman et al., 2010).
Automatic model selection was also performed in
the calculation of the censoring weights (adjusting
for potential selection bias from loss to follow-up,
see Appendix).
Even though the procedure is flexible (e.g. to
include automatic model selection), the
implementation requires some experience in stat-
istical programming (see R-code for the present
application in the supplementary file). Recent
development in software has made g-estimation
based on this method available, in the R-package
gestools (De Stavola et al., 2015). This package
takes as input the data set in long format and
changes it to a convenient form, to ensure there
exists a data entry for each person at each time
point, generates lagged versions of the time-
varying covariates, and covariate-histories (set to
zero at the start). It performs g-estimation for
different types of action variables and for continu-
ous or dichotomous outcomes.
A causal graph (DAG) of the design is shown in
Figure 3.
1104 H. B. Jacobsen et al.

Figure 3. Causal graph (DAG) of study design, with time-varying action variable, covariates and outcome. Directed arrows represent poss-
ible direct causal effects (red arrows are short-term effects being estimated), in a longitudinal design with 390 adults in an occupational reha-
bilitation program in Norway.
Note. At: action variable (IEDt), Lt: baseline and time-varying covariates, Yt: outcome (HADDt, HADAt, HADTt), Ct: censoring-indicator
(loss-to-follow up) – surrounding box represents conditioning (the outcome is observed only for those not lost-to-follow up)

Multiple Imputation (MI) to Handle Missing
Values
Incomplete data were of concern, both for the out-
comes (HADDt , HADAt , HADTt ) for the action
variable (IEDt ), and different covariates. Missing
values in an outcome was considered as loss-to-
follow-up and the person was not allowed to re-
enter. Potential selection bias from loss-to-follow-
up was adjusted for by inverse probability of censor-
ing weights (Appendix).
To address the issue of incomplete data in covari-
ates, multiple imputation (MI) (Appendix) was per-
formed under the assumption of missing at random
(MAR), with the R-package mice (Buuren &
Groothuis-Oudshoorn, 2011). The outcomes
served as covariates both in the propensity score
and outcome, steps 1, 2, and were imputed as
missing covariate, but not as missing outcome. Out
of 69 variables in the dataset, the covariates with
missing values are shown in supplementary table 1.
Efficiency gain from MI was assessed by the frac-
tion of missing information (FMI), approximated
by FMI = r/(1 + r) (for a high number of imputa-
tions), where r is the relative increase in variance
due to the missingness (Madley-Dowd et al., 2019;
Schafer, 1999) (Appendix). The FMI is parameter
specific and quantifies the loss of information due
to missingness, while accounting for the amount of
information retained by other variables (Madley-
Dowd et al., 2019). A low number of imputations
in MI is usually sufficient, for example with an
FMI of 20%, 10 imputations correspond to a relative
efficiency above 98% (Schafer, 1999).
With respect to standard errors in the g-estimation
algorithm, the bootstrap is recommended (Vanstee-
landt & Sjolander, 2016). To account for uncertainty
from incomplete data and to achieve unbiased standard
errors from the g-estimation, the bootstrap was per-
formed for each imputed dataset. Ten complete data-
sets were generated from the imputation algorithm,
and for each complete dataset a bootstrap parameter
estimate with the standard error was generated (500
resamples), and finally combined according to
Rubin’s rules (Rubin, 1987). Model selection was
carried out for every bootstrap sample, both in the cen-
soring weights and propensity score models, to identify
the best model. A large set of covariates was entered for
each bootstrap sample, and the automatic lasso
regression algorithm returned the best cross-validated
model for that sample, in terms of minimum prediction
error, while avoiding overfitting.
Results
Selected demographics and baseline covariates in the
sample are shown in Table I and baseline scores on
executive functioning for the whole sample are
shown in supplementary table 2.
At baseline, those with higher education scored
significantly better compared to those with low edu-
cation on the Stockings of Cambridge, Emotional
recognition, and Rapid visual processing, but not
on cognitive flexibility (see Table II).
 Psychotherapy Research 1105
Table I. Selected demographics at baseline for all participants. during the intervention period (mean length = 28
days), represented by a decrease in IEDt (increased
Participants (n =
390) flexibility) and depression/anxiety symptoms
(HADDt , HADAt , and HADTt ) followed by gradu-
Variable Mean SD ally flatter mean levels during follow-up.
Also, a constant difference between the groups, in
Age 45.2 9.8
HADS
favor of the high-education group, was evident.
Anxiety (0–21; 0 = no anxiety) 7.7 4.5 Included in the plot is the function f (t) from
Depression (0–21; 0 = no depression) 5.7 4.0 Equation (1), used in the outcome model (steps 2
Total 13.2 7.9 and 3) as one component of lt , representing a good
Variable n % fit of the observed outcomes.
Gender
Female 267 69
Different causal models (SNMMs) were fitted to
Male 123 34 assess long-term effects, group differences in short-
Education term effects (interactions with the action variable or
High (13 years or more) 189 50 other covariates) and whether TChopet or TChelpt
Low (up to or including 12 years) 189 50 could play a role as moderators for IEDt ’s influence
Work status
Not in work 172 44
on mental distress (Appendix). No such significant
Part-time work 126 32 effects were found, but few repeated measures with
Full-time work 74 19 limited observation time, and model complexity
resulted in a lack of power for one or more of these
Note: SD, standard deviation; HADS, Hospital Anxiety and
tests.
Depression Scale.
The simpler SNMM from Equation (3), modeling
a constant and short-term low- versus high education
Results from g-Estimation effect as well as a time-varying part, showed signifi-
cant causal parameters for all outcomes. Final com-
Time-course of the mean level of the action variable bination estimates from the 10 imputed datasets are
(IEDt ) and outcomes (HADDt , HADAt , and shown in Table III. The effect of IEDt on HADDt
HADTt ), stratified by low- versus high education had a non-significant intercept (c0 ), a significant
(HighEdut ), is shown in Figure 2. The overall difference between low- and high education
feature in both groups was clear improvement (ĉ1 = 0.043, p = 0.016) and a significant time-

Table II. Baseline scores on executive functioning measures divided by high and low education levels.

Education low (n = 189) Education high (n = 189) Statistics

Mean SD Mean SD T (df)# p-value

Attention
Simple reaction time
Reaction time (ms) 256.2 56.5 246.9 41.4 1.813 (374) .071
Choice reaction time
Reaction time (ms) 324.4 68.2 313.7 50.3 1.722 (373) .086
Rapid visual information processing
Latency (ms) 414.2 93.8 395.5 74.2 2.132 (367) .034
Probability of hit 0.58 0.19 0.67 0.17 −4.461 (368) <.001
Memory
Spatial working memory
Total between errors 13.2 10.3 11.5 9.2 1.658 (374) .098
Spatial recognition memory
Response time (ms) 2796.3 1144.7 2583.7 991.2 1.928 (375) .055
Total correct (%) 80.4 10.0 80.7 10.7 –.272 (375) .786
Executive function
Stockings of Cambridge
Choice duration (ms) 4315.1 2389.5 4125.3 2120.2 .815 (374) .416
Total correct 8.7 2.1 9.2 1.9 −2.537 (375) .012
Intra-extra dimensional set shift
Total errors adjusted 26.2 24.5 22.0 19.2 1.855 (374) .064
Emotion recognition
Emotion recognition task
Total correct (%) 57.3 10.6 60.7 8.8 −3.400 (374) <.001
1106 H. B. Jacobsen et al.
Table III. Estimated causal effects (SNMM coefficients) on mental distress by depressive symptoms (HADDt + 1), anxiety (HADAt + 1), and
sum (HADTt + 1) (separate models), of hypothetical interventions on cognitive flexibility (IEDt), by g-estimation.

HADDt+1 HADAt+1 HADT t+1

Estimate 95% CI p-value Estimate 95% CI p-value Estimate 95% CI p-value

Int 0.002 −0.0021, 0.006 0.33 0.0035 −0.002, 0.009 0.2 0.0038 8e-6, 0.008 0.0495
HighEdu 0.043 0.008, 0.08 0.016 0.048 0.017, 0.08 0.003 0.12 0.04, 0.2 0.0026
f (t) −0.005 −0.009, −0.001 0.016 −0.009 −0.015, −0.003 0.0036 −0.006 −0.01, −0.002 0.0022

Note. Parameter estimates refer to c0 , c1 , c2 in the SNMM = (c0 + c1 HighEdu + c2 f (t))IEDt (Equation 3), where f (t) is the function in
(1) with l = 0.2, 0.1, 0.4 respectively.

varying part (ĉ2 = −0.005, p = 0.016). The effect
of IEDt on HADAt had similar characteristics with
a non-significant intercept (c0 ), a significant differ-
ence between low- and high education
(ĉ1 = 0.048, p = 0.003) and a significant time-
varying part (ĉ2 = −0.009, p = 0.0036). The effect
of IEDt on the total score, HADTt had a significant
intercept (ĉ0 = 0.0038, p = 0.05), a significant
difference between low- and high education
(ĉ1 = 0.12, p = 0.0026) and a significant time-
varying part (ĉ2 = −0.006, p = 0.0022).
These estimates are inserted in the SNMM from
Equation (3) (c0 + c1 HighEdu + c2 f (t)) to give the
coefficient for the short-term effect for each time
point (Table IV). The causal effect had the same
characteristics for all three outcomes.
For low education, the effect was negative overall,
strongest during the intervention period and gradually
weaker. In other words, a hypothetical intervention
that could increase inflexibility seemed to cause
improvement in mental distress at the subsequent
time-point, the more inflexibility, the larger improve-
ment. For high education, a change in sign was esti-
mated, the effect was negative during the
intervention and positive during follow-up (for
HADDt and HADTt immediately following the inter-
vention, and for HADAt the sign-change happened in
the last period). In this group, a hypothetical interven-
tion that could increase inflexibility seemed to
improve mental distress during the intervention
period (although weaker than for the low-education
group). After the intervention period, the effect
changed sign, a hypothetical intervention that could
increase inflexibility had a negative influence on
mental distress.
In terms of magnitude, a hypothetical intervention
that could inflict inflexibility in the low-education
group, equal to 26 points on the IEDt scale relative
to no inflexibility (IEDt = 0) at Time1 (baseline),
would yield a change of −0.053 × 26 = −1.37
(95% CI: − 2.46, − 0.27) points on the HADDt
scale at Time2 (improvement). A similar intervention
at Time2 to change IEDt with 20 points relative to no
inflexibility in the low-education group, would yield a
change of −0.033 × 20 = −0.65 (95% CI: − 1.18,
−0.13) points on the HADDt scale at Time3 , and
finally to change IEDt with 19 points at Time3 rela-
tive to no inflexibility in the low-education group,
would yield a change of −0.02 × 19 = −0.38
(95% CI: − 0.68, − 0.08) points on the HADDt
scale at Time4 . In other words, a total change in
HADDt of approximately −2.4 points would be
expected for the low-education group. In contrast,
the change in the high-education group would
be −0.01 × 22 = −0.21 (95% CI: − 1.14, 0.71)
HADDt points at Time2 , 0.01 × 17 = 0.18
(95% CI: − 0.27, 0.62) HADDt points at
Time3 and 0.023 × 15 = 0.35 (95% CI:0.11, 0.59)
HADDTt points at Time4 , a total change of 0.31
HADDTt points.

Table IV. Time-varying estimated SNMM coefficients from Table III, for each time-point and level of education, describing short-term
causal effects over time on mental distress by depressive symptoms (HADDt + 1), anxiety (HADAt + 1), and sum (HADTt + 1) (separate
models) of hypothetical interventions on cognitive flexibility (IEDt).

HADDt+1 HADAt+1 HADT t+1

Low education High education Low education High education Low education High education

Time1 −0.055 −0.012 −0.065 −0.016 −0.14 −0.019

Time2 −0.035 0.008 −0.036 −0.012 −0.101 0.019
Time3 −0.021 0.022 −0.021 0.028 −0.067 0.054

Note. Estimates refer to c′ zt+1,t = (c0 + c1 HighEdu + c2 f (t)) (time-varying coefficients in Equation 3), where f (t) is the function in (1) with
l = 0.2, 0.1, 0.4 respectively.

Other dimensions of EF were tested similarly,
without any significant effects. Dividing the signifi-
cance level (0.05) by the number of tests to correct
for multiple testing (a conservative Bonferroni cor-
rection) would still result in significant effects for
HADAt and HADDt .
In the imputation, both prior and subsequent
measurements for each missing value were allowed
in the model. Stability in the effect estimates across
the imputations indicated successful imputation.
Estimated FMI ′ s were between 1.6 and 4.3%, con-
siderably smaller than the proportions of missing
IEDt values during follow-up (except at baseline)—
0.8%, 10.3% and 30.8%, reflecting that other vari-
ables succeeded in retaining information for the
missing IEDt values.
Selection bias from loss-to-follow-up was assessed
by comparison of results with and without censoring-
weights (Appendix). Estimation without weights
(without adjusting for loss-to-follow-up) showed an
underestimation of the magnitude (in the direction
of zero). Relative bias was largest for HADDt , with
8.5%, and 7.1% for c1 and c2 (significant coeffi-
cients) and smallest for HADAt with 0.2% and
1.1% for c1 and c2 , respectively. The difference in
bias was not explained by different samples.
Return-to-Work in Low vs. High Education
Testing whether those with low education differed
with regard to work, an independent samples t-test
indicated no significant difference in the percentage
of sick leave post-treatment to 12 months after com-
pleting rehabilitation t(175) = 0.2, p = .88.
Additional analyses were performed to check for
baseline differences in cognitive flexibility and
depressive and anxious symptoms in the RTW
versus non-RTW groups. Here, depressive symp-
toms were associated with RTW (p = 0.04), but no
substantial differences were detected for anxiety or
cognitive flexibility (p = .13).
Discussion
Causal effects of cognitive inflexibility on improve-
ment in symptoms of depression and anxiety were
found in this observational dataset. Despite limit-
ations in the data the model proved sufficient to
“tease out” strong causal effects. The inflexibility
domain of EF had a much stronger signal/noise
ratio than the other domains, consistent across all
the centers, and maintained significance when cor-
rected for multiple testing. The effect decreased
over time and was moderated by education.
Psychotherapy Research 1107
In the low-education group, cognitive inflexibility
was higher than in the high-education group over the
whole period and seemed to have a strong causal
effect on subsequent anxiety and depressive symptoms,
which was not the case for the high-education group.
The effect weakened after the treatment period. A
total change in HADDt (depressive symptoms) over
the whole observation period for the low-education
group was estimated to be around 2.4 points improve-
ment relative to perfect flexibility, which is clinically
relevant. The high education group did have a
similar, although weak effect during the treatment
period, and with a change of sign, indicating that
more cognitive inflexibility led to more depressive/
anxious symptoms after treatment. These results
might explain why there is no consistent relationship
between EF capacity and mood disorders, and why
some benefiting from rehabilitation while others do
not (Mikkelsen & Rosholm, 2018).
Education as a moderator was found in a recent
systematic review and meta-analysis on the effects
of antidepressants on executive functioning
(Gudayol-Ferré et al., 2021). Improvement in EFs
was overall stronger in the higher educated following
treatment (Gudayol-Ferré et al., 2021) indicating
that education sometimes can be a moderator in
this relationship.
Theoretically, the importance and positive benefits
of high cognitive flexibility are recognized in leading
models of cognitive behavioral therapy (CBT) (Clark
& Beck, 2010; Hayes et al., 2011). The more tra-
ditional CBT models state that with repeated acti-
vation, negative self-schemas can crystalize into a
depressive or anxious mode, which then generalizes
to more and milder stressful life events. Newer devel-
opments such as metacognitive therapy also state that
emotional disorders are maintained by the activation
of maladaptive thinking styles. Here, the cognitive
attentional syndrome is characterized by extended
negative thinking in the form of rumination and
worry and is associated with increased self-focused
attention and maladaptive coping behaviors (Wells,
2011). In both cognitive models, maladaptive infor-
mation processing is thought to result in a weakening
of cognitive control. This could be operationalized as
inflexibility or inability to access more adaptive,
alternative modes of thinking. What is surprising
with our current findings is that such inflexibility
seems to be beneficial.
Our results might therefore indicate an important
nuance in our understanding of cognitive flexibility
and symptoms of anxiety and depression. While it
is fairly established that shifting ability is central for
supporting flexible and effective self-regulation
(Kashdan & Rottenberg, 2010), the ability to shift
attention from a task at hand to new aspects of a
1108 H. B. Jacobsen et al.
situation more critical is highly adaptive, but it also
costly in the form of energy expenditure for the
brain. An automized cost–benefit analysis of shifting
between sets of stimuli in any demanding novel situ-
ation has previously been demonstrated (Szalma &
Matthews, 2015). This signifies that a given chal-
lenge must be significant or perceived as important
to motivate shifts in attention. Less flexibility may
thus be associated with good coping in less demand-
ing day-to-day situations, only becoming negatively
associated with good coping in crises or extended
challenging situations.
Following this reasoning, our results could be
explained by our participants’ perception of their
challenges related to sick leave. Challenges caused
by sick leave in the Nordic region are often con-
sidered manageable for the individual due to avail-
able, generous benefits, support and health care
systems (Elstad & Vabø, 2008). It could be that the
beneficial role of inflexibility appears in our study
due to the context of sick leave being less dramatic
than in countries with no or little welfare or sick
leave benefits.
Another interpretation also worth considering in
lieu of our results is our action variable testing cogni-
tive flexibility, IED set shift. In this test, a rule is
established by shaping a given participants’ behavior
with contingencies related to choosing an abstract
pattern (correct/wrong). Once a rule is established,
it is then altered without warning, and the speed
and accuracy of acquiring a new rule, accumulated
over several stages, is the outcome from the test. Its
design dictates that those who resist changes to
rules and are less sensitive to cues from their
context, score poorly on this test. But here it is also
these participants who improve in their symptoma-
tology the most. The counter-intuitive nature of
our data could also, therefore, be viewed considering
the behavioral construct of rule-governed behavior.
A conceptualization of EFs as a subset of rule-gov-
erned behavior characterized by flexibility was given
by Hayes et al in the mid-90s (Hayes et al., 1996).
When one compares learning behavior under the
control of instructions (i.e., rule-governed behavior)
with behaviors shaped by contingencies such as
reinforcement by trial and error, participants who
are instructed tend to show more insensitivity to
negative rewards than contingency-shaped partici-
pants (Vaughan, 1989). This entails that when
instructed to expect less symptoms from doing a be-
havior such as running, a less flexible participant
would ignore experience of non-relief and defer to
the instruction of “less symptoms.” Within rule-gov-
erned behavior, cognitive inflexibility, therefore,
would refer to a tendency to maintain current behav-
ior disregarding experience to a larger degree, which
could be adherence to instructed benefits from fol-
lowing “rules” set in a rehabilitation program, such
as expecting less symptoms from certain behaviors.
As an example of this, a previous randomized con-
trolled trial indicated support for our current
finding in showing that low-educated participants
benefited more from an intervention in smoking ces-
sation, due to increased adherence to instructive
stories (Strecher et al., 2008).
Indeed, the current rehabilitation programs would
involve multiple examples of increased behavioral
activation under the control of instructions from
therapists. Given these conditions, one could envi-
sion generalized tracking developing during the stay
from instructed functional relationships among
events (e.g., drawing out rules based on observation
and instructions from therapists) (Villatte et al.,
2015). This generalized tracking of a rule in the
abstract sense could then improve anxiety and
depression through rigidity over time and perceived
symptom relief disregarding evidence of the contrary.
This is substantiated by the HADS being a patient-
reported outcome, meaning that one does not know
if others in close personal relationships for example
would judge this perceived symptom relief in the
same way. Thus, this improvement could be strictly
subjective, something that is indicated by the lack
of substantive associations with return-to-work. Of
interest to the current findings is also a prior publi-
cation on this multicenter trial showing significant
effects from other cognitive functions on return-to-
work, but not between cognitive flexibility and
return-to-work (Johansen et al., 2019).
Methodological Discussion
The structural nested mean model with g-estimation
demonstrated efficient estimation in this application.
The study of causal reciprocal effects between two
variables (so-called cross-lagged effects), dates to
early time-series analysis, and was incorporated in
the SEM framework (Jöreskog, 1970; Jöreskog &
Sörbom, 1979), in the form of the cross-lagged
panel model (CLPM), which became especially
popular in the behavioral and psychological science
research. A recent review of medical journals found
270 papers published between 2009 and 2019 that
used CLPM (Usami et al., 2019b). The CLPM rep-
resents a different tradition for modeling causal
effects than the counterfactual framework in epide-
miology but shares the same fundamental “no
unmeasured confounding” assumption. The
CLPM’s sensitivity for this assumption has been cri-
ticized, e.g., yielding non-satisfactory adjustment for
stable trait factors (Lucas, 2022). Extensions of

CLPM and several alternatives with latent variables
have been developed (Usami et al., 2019a), but
they all have in common that they are severely
biased in their causal cross-lagged effect-estimate,
when the data-generating process deviates from
their strict parametric assumptions (Lüdtke &
Robitzsch, 2022). To date, it seems to be a lack of
consensus in psychological and behavioral science,
of whether to keep (Orth et al., 2021) or to
abandon (Lucas, 2022) the CLPM model.
Consequently, the counterfactual framework has
gained acceptance in defining the causal effects
(bias) in general (Lüdtke & Robitzsch, 2022; Rijn-
hart et al., 2021; Usami, 2022) and for comparison
of methods (Lüdtke & Robitzsch, 2022). In some
cases, authors from the SEM literature even rec-
ommend causal mediation analysis (counterfactual
framework) rather than the traditional mediation
models (SEM) (Rijnhart et al., 2021). The strength
of the SNMM with g-estimation has been recog-
nized, but unwillingness to depart from the latent
variable models makes a comparison between the
two traditions difficult. Usami used the SNMM as
part of fitting the SEM, to estimate joint effects of
time-varying treatments with improved control for
stable traits, due to the robustness of functional
form and misspecification in the SNMM (Usami,
2022). Despite of these advances in robustness, the
SEM component still relies on several parametric
(and distributional) assumptions, some of which are
untestable.
The assumption of “no unmeasured confound-
ing” is not testable from data, and the reason that
causal inference from observational data is hazar-
dous (Herńan & Robins, 2020). In the counterfac-
tual framework, rather than introducing further
assumptions on unmeasured factors that each
would need separate assessment (and not testable),
it seems more appealing to apply the already exist-
ing framework for sensitivity analysis (Robins et al.,
1999; VanderWeele & Arah, 2011; Klungsøyr et al.,
2009). In this framework, one needs a model (a
parsimonious function) for the association
between the mean of the counterfactual outcome
variable and the action, treatment or exposure,
within levels of measured confounders. This bias-
function which represents the magnitude of con-
founding due to unmeasured factors, is not ident-
ified from the data, and needs to be considered
plausible by subject-matter experts. The sensitivity
of the causal effect is described by varying this func-
tion (and its parameters). In this way, many fewer
sensitivity parameters are needed to be varied, and
the “impossible” decision as to whether to view U
as univariate or multivariate, continuous or dis-
crete, and its relation to the action variable,
Psychotherapy Research 1109
treatment or exposure and outcome is done away
with (Robins et al., 1999). The SNMM with g-esti-
mation is well suited for sensitivity analysis, because
g-estimation does not require no unmeasured con-
founding for the identification of parameters, it
merely requires that the magnitude of unmeasured
confounding is known (Robins et al., 1999; Van-
steelandt & Joffe, 2014; Yang & Lok, 2018). By
letting the bias-function be zero for no unmeasured
confounding and varying both parameters in this
function and the functional form itself, sensitivity
for unmeasured confounding can be assessed
(Yang & Lok, 2018). However, Yang and Lok
described this in the form of complicated estimating
equations (Yang & Lok, 2018), which have not
been translated to g-estimation by nested regression
models from Vansteelandt & Sjolander, used in this
application.
A practical problem is to determine if a certain
magnitude of unmeasured confounding (on the
scale of the parameters in the bias-function) rep-
resents sensitivity or not. Therefore, when the
association of interest is not well described, as in
the present case, such an undertaking might not
be worth the effort. Further, the degree of missing
data will necessarily affect the interpretation of the
results in all cases. Plausible significant causal
effects rely on “no misspecification” in the imputa-
tion models (as well as in other model-com-
ponents), which are strong assumptions. Adding
further assumptions, in the form of bias-functions
for unmeasured confounding might seem question-
able. Nevertheless, the g-estimator is itself robust
for unmeasured confounding by allowing for miss
specification in one of its components (e.g. unmea-
sured confounding in the outcome model) (doubly
robust). One gets two attempts of success, consist-
ent estimates are achieved if either the propensity
score model, or the outcome model, in addition
to the SNMM are correctly specified. Even effi-
ciency gain can be achieved by including a
“wrong” model. In an RCT with a dichotomous
action (or treatment or exposure), where the pro-
pensity score model is known (probability for treat-
ment), and consistent effect-estimate would be
expected, adding a mis specified outcome model
in a doubly-robust estimator usually leads to
lower standard error in the effect-estimate (Tsiatis
et al., 2020). Insight in one of the two models
will, in a doubly robust method, be of extra value.
In the present application, confidence in the com-
prehensive propensity score model (a series of
lasso-regressions with a rich set of covariates, both
time-constant and time-varying) indicates that the
estimated causal effect is not completely a result
of unmeasured confounding.
1110 H. B. Jacobsen et al.
Strengths and Limitations
Some limitations concerning a causal interpretation of
the results are: the observational nature of the data, a
limited observation period and number of repeated
measures, time-varying confounding, and substantial
missingness. On the other hand, the multicenter
design with four independent sites provided data
from many participants with a good geographic
spread in the most populous region of Norway. The
tests of executive functioning were selected on relevant
and important aspects of executive functioning and the
HADS is a widely used and suitable response variable.
The structural nested mean model with g-estimation is
appropriate causal inference in these types of data out-
performing several alternative methods.
Conclusion
The results from this study identify an unexpected
causal relationship between cognitive inflexibility
and improvement in symptoms of anxiety and
depression. It challenges the scientific consensus,
even though the causal interpretation must be
viewed as exploratory, due to limitations in the
design. Confirmation of these results from other
studies will contribute a nuance in the understanding
of which variables to target in therapy and why. With
higher acceptance in defining causal effects with
counterfactuals within the psychological and behav-
ioral science, and the continuing lack of consensus
for the most appropriate model for causal inference,
this application is meant to promote the structural
nested mean model with g-estimation. It has fewer
non-testable assumptions than the traditional latent
variable models and does not rely on specialized
software.
Disclosure Statement
No potential conflict of interest was reported by the
author(s).
Supplemental data
Supplemental data for this article can be accessed
online at https://doi.org/10.1080/10503307.2023.
2221808.
References
Aasvik, J. K., Woodhouse, A., Børsting Jacobsen, H.,
Borchgrevink, P. C., Stiles, T. C., & Landrø, N. I. (2015).
Subjective memory complaints among patients on sick leave
are associated with symptoms of fatigue and anxiety. Frontiers
in Psychology, 6, https://doi.org/10.3389/fpsyg.2015.01338
Aasvik, J. K., Woodhouse, A., Stiles, T. C., Jacobsen, H. B.,
Landmark, T., Glette, M., Borchgrevink, P. C., & Landrø,
N. I. (2017). Effectiveness of working memory training
among subjects currently on sick leave Due to complex symp-
toms. Frontiers in Psychology, 7, 2003. https://doi.org/10.3389/
fpsyg.2016.02003
Bjelland, I., Dahl, A. A., Haug, T. T., & Neckelmann, D. (2002).
The validity of the hospital anxiety and depression scale.
Journal of Psychosomatic Research, 52(2), 69–77. https://doi.
org/10.1016/S0022-3999(01)00296-3
Buuren, S. V., & Groothuis-Oudshoorn, K. (2011). mice: multi-
variate imputation by chained equations in R. Journal of
Statistical Software, 45(3), 1–67. https://doi.org/10.18637/jss.
v045.i03
Cambridge Cognition. (n.d.). Executive function. https://
cambridgecognition.com/executive-function/
Caspi, A., Houts, R. M., Ambler, A., Danese, A., Elliott, M. L.,
Hariri, A., Rasmussen, L. J. H., Reuben, A., Richmond-
Rakerd, L., Sugden, K., Wertz, J., Williams, B. S., & Moffitt,
T. E. (2020). Longitudinal assessment of mental health dis-
orders and comorbidities across 4 decades among participants
in the Dunedin birth cohort study. JAMA Network Open, 3(4),
e203221. https://doi.org/10.1001/jamanetworkopen.2020.3221
Clark, D. A., & Beck, A. T. (2010). Cognitive theory and therapy
of anxiety and depression: Convergence with neurobiological
findings. Trends in Cognitive Sciences, 14(9), 418–424. https://
doi.org/10.1016/j.tics.2010.06.007
Daniel, R. M., Cousens, S., De Stavola, B., Kenward, M. G., &
Sterne, J. (2013). Methods for dealing with time-dependent
confounding. Statistics in Medicine, 32(9), 1584–1618. https://
doi.org/10.1002/sim.5686
De Stavola, B. L., et al. (2015). Mediation analysis With inter-
mediate confounding: Structural equation modeling viewed
through the causal inference lens. American Journal of
Epidemiology, 181(1), 64–80. https://doi.org/10.1093/aje/
kwu239
Devanand, D. P., Pelton, G. H., Marston, K., Camacho, Y.,
Roose, S. P., Stern, Y., & Sackeim, H. A. (2003). Sertraline
treatment of elderly patients with depression and cognitive
impairment. International Journal of Geriatric Psychiatry, 18(2),
123–130. https://doi.org/10.1002/gps.802
Diamond, A. (2013). Executive functions. Annual Review of
Psychology, 64(1), 135–168. https://doi.org/10.1146/annurev-
psych-113011-143750
Elstad, J. I., & Vabø, M. (2008). Job stress, sickness absence and
sickness presenteeism in Nordic elderly care. Scandinavian
Journal of Public Health, 36(5), 467–474. https://doi.org/10.
1177/1403494808089557
Ertefaie, A., Small, D. S., Flory, J. H., & Hennessy, S. (2017). A
tutorial on the use of instrumental variables in pharmacoepide-
miology. Pharmacoepidemiology and Drug Safety, 26(4), 357–
367. https://doi.org/10.1002/pds.4158
First, M. B., Gaebel, W., Maj, M., Stein, D. J., Kogan, C. S.,
Saunders, J. B., Poznyak, V. B., Gureje, O., Lewis-
Fernández, R., Maercker, A., Brewin, C. R., Cloitre, M.,
Claudino, A., Pike, K. M., Baird, G., Skuse, D., Krueger, R.
B., Briken, P., Burke, J. D., Woods, D. W. (2021). An organ-
ization- and category-level comparison of diagnostic require-
ments for mental disorders inICD-11 andDSM-5. World
Psychiatry, 20(1), 34–51. https://doi.org/10.1002/wps.20825
Fresco, D. M., Segal, Z. V., Buis, T., & Kennedy, S. (2007).
Relationship of posttreatment decentering and cognitive reac-
tivity to relapse in major depression. Journal of Consulting and
Clinical Psychology, 75(3), 447–455. https://doi.org/10.1037/
0022-006X.75.3.447

Friedman, J., Hastie, T., & Tibshirani, R. (2010). Regularization
paths for generalized linear models via coordinate descent.
Journal of Statistical Software, 33(1), 1–22. https://doi.org/10.
18637/jss.v033.i01
Fu, Z., Brouwer, M., Kennis, M., Williams, A., Cuijpers, P., &
Bockting, C. (2021). Psychological factors for the onset
of depression: a meta-analysis of prospective studies. BMJ
Open, 11(7), e050129. https://doi.org/10.1136/bmjopen-
2021-050129
Grant, D. A, & Berg, E. A. (1993). Wisconsin card sorting test.
Journal of Experimental Psychology.
Gudayol-Ferré, E., Duarte-Rosas, P., Peró-Cebollero, M., &
Guàrdia-Olmos, J. (2021). The effect of second-generation
antidepressant treatment on the executive functions of patients
with major depressive disorder: a meta-analysis study with
structural equation models. Psychiatry Research, 296, 113690.
https://doi.org/10.1016/j.psychres.2020.113690
Hayes, S. C., Gifford, E. V., & Ruckstuhl Jr, L. (1996). Relational
frame theory and executive function: A behavioral approach.
Hayes, S. C., Luoma, J. B., Bond, F. W., Masuda, A., & Lillis, J.
(2006). Acceptance and commitment therapy: model, pro-
cesses and outcomes. Behaviour Research and Therapy, 44(1),
1–25. https://doi.org/10.1016/j.brat.2005.06.006
Hayes, S. C., Strosahl, K. D., & Wilson, K. G. (2011). Acceptance
and commitment therapy: The process and practice of mindful
change. Guilford Press.
Herńan, M., & Robins, J. (2020). Causal inference: What If.
Chapman & Hall/CRC.
Huizinga, M., Baeyens, D., & Burack, J. A. (2018). Editorial:
Executive function and education. Frontiers in Psychology, 9,
https://doi.org/10.3389/fpsyg.2018.01357
Jacobsen, H. B., Aasvik, J. K., Borchgrevink, P. C., Landrø, N. I.,
& Stiles, T. C. (2016). Metacognitions Are associated with sub-
jective memory problems in individuals on sick leave due to
chronic fatigue. Frontiers in Psychology, 7, https://doi.org/10.
3389/fpsyg.2016.00729
Joffe, M. M., & Brensinger, C. (2003). Weighting in instrumental
variables and G-estimation. Statistics in Medicine, 22(8), 1285–
1303. https://doi.org/10.1002/sim.1380
Johansen, T., Jensen, C., Eriksen, H. R., Lyby, P. S., Dittrich, W.
H., Holsen, I. N., Jakobsen, H., & Øyeflaten, I. (2019).
Occupational rehabilitation is associated with improvements
in cognitive functioning. Frontiers in Psychology, 10, 2233.
Jöreskog, K. G. (1970). A general method for analysis of covari-
ance structures. Biometrika, 57(2), 239–251. 5. https://doi.
org/10.1093/biomet/57.2.239
Jöreskog, K. G., & Sörbom, D. (1979). Advances in factor analysis
and structural equation models. Abt Books.
Kashdan, T. B., & Rottenberg, J. (2010). Psychological flexibility
as a fundamental aspect of health. Clinical Psychology Review, 30
(7), 865–878. https://doi.org/10.1016/j.cpr.2010.03.001
Klungsøyr, O., Sexton, J., Sandanger, I., & Nygård, J. F. (2009).
Sensitivity analysis for unmeasured confounding in a marginal
structural Cox proportional hazards model. Lifetime data analy-
sis, 15, 278–294.
Letkiewicz, A. M., Miller, G. A., Crocker, L. D., Warren, S. L.,
Infantolino, Z. P., Mimnaugh, K. J., & Heller, W. (2014).
Executive function deficits in daily life prospectively predict
increases in depressive symptoms. Cognitive Therapy and
Research, 38(6), 612–620. https://doi.org/10.1007/s10608-
014-9629-5
Levkovitz, Y., Caftori, R., Avital, A., & Richter-Levin, G. (2002).
The SSRIs drug Fluoxetine, but not the noradrenergic
tricyclic drug Desipramine, improves memory
performance during acute major depression. Brain Research
Bulletin, 58(4), 345–350. https://doi.org/10.1016/S0361-9230
(01)00780-8
Psychotherapy Research 1111
Liu, H., Funkhouser, C. J., Langenecker, S. A., & Shankman, S.
A. (2021). Set shifting and inhibition deficits as potential endo-
phenotypes for depression. Psychiatry Research, 300, 113931.
https://doi.org/10.1016/j.psychres.2021.113931
Lucas, R. E. (2022). It’s time To abandon the cross-lagged panel
model. Preprint.
Lüdtke, O., & Robitzsch, A. (2022). A comparison of different
approaches for estimating cross-lagged effects from a causal
inference perspective. Structural Equation Modeling: A
Multidisciplinary Journal, 29(6), 888–907.
Mac Giollabhui, N., Olino, T. M., Nielsen, J., Abramson, L. Y., &
Alloy, L. B. (2019). Is worse attention a risk factor for or a con-
sequence of depression, or are worse attention and depression
better accounted for by stress? A prospective test of three
hypotheses. Clinical Psychological Science, 7(1), 93–109.
https://doi.org/10.1177/2167702618794920
MacPherson, H. A., Kudinova, A. Y., Schettini, E., Jenkins, G.
A., Gilbert, A. C., Thomas, S. A., Kim, K. L., Radoeva, P.
D., Fenerci, R. L. B., Yen, S., Hower, H., Hunt, J., Keller,
M. B., & Dickstein, D. P. (2021). Relationship between cogni-
tive flexibility and subsequent course of mood symptoms and
suicidal ideation in young adults with childhood-onset bipolar
disorder. European Child & Adolescent Psychiatry, 31(2), 299–
312. https://doi.org/10.1007/s00787-020-01688-0
Madley-Dowd, P., Hughes, R., Tilling, K., & Heron, J. (2019).
The proportion of missing data should not be used to guide
decisions on multiple imputation. Journal of Clinical
Epidemiology, 110, 63–73. https://doi.org/10.1016/j.jclinepi.
2019.02.016
Mikkelsen, M. B., & Rosholm, M. (2018). Systematic review and
meta-analysis of interventions aimed at enhancing return to
work for sick-listed workers with common mental disorders,
stress-related disorders, somatoform disorders and personality
disorders. Occupational and Environmental Medicine, 75(9),
675–686. https://doi.org/10.1136/oemed-2018-105073
Miyake, A., & Friedman, N. P. (2012). The nature and organiz-
ation of individual differences in executive functions. Current
Directions in Psychological Science, 21(1), 8–14. https://doi.org/
10.1177/0963721411429458
Mulder, H., Verhagen, J., Van der Ven, S. H. G., Slot, P. L., &
Leseman, P. P. M. (2017). Early executive function at Age
Two predicts emergent mathematics and literacy at Age five.
Frontiers in Psychology, 8. https://doi.org/10.3389/fpsyg.2017.
01706
Naimi, A. I., Cole, S. R., & Kennedy, E. H. (2017). An introduc-
tion to g methods. International Journal of Epidemiology, 46(2),
756–762.
Neyman, J. (1923). Sur les applications de la thar des probabilities
aux experiences Agaricales: Essay des principle. Excerpts re-
printed (1990) in English (D. Dabrowska and T. Speed,
Trans.). Statistical Science, 5: 463–472.
Nolen-Hoeksema, S. (2011). Lost in thought: The perils of
rumination.
Orth, U., Clark, D. A., Donnellan, M. B., & Robins, R. W.
(2021). Testing prospective effects in longitudinal research:
Comparing seven competing cross-lagged models. Journal of
Personality and Social Psychology, 120(4), 1013–1034. https://
doi.org/10.1037/pspp0000358
Pearl, J. (1995). Causal diagrams for empirical research.
Biometrika, 82(4), 669–688. https://doi.org/10.1093/biomet/
82.4.669
Pearl, J. (2009). Causality: Models, reasoning, and inference (2nd
ed). Cambridge University Press.
Picciotto, S., & Neophytou, A. M. (2016). G-Estimation of struc-
tural nested models: Recent applications in Two subfields of
epidemiology. Current Epidemiology Reports, 3(3), 242–251.
https://doi.org/10.1007/s40471-016-0081-9
1112 H. B. Jacobsen et al.
Porter, R. J., Robinson, L. J., Malhi, G. S., & Gallagher, P. (2015).
The neurocognitive profile of mood disorders - a review of the
evidence and methodological issues. Bipolar Disorders, 17, 21–
40. https://doi.org/10.1111/bdi.12342
Price, R. B., & Duman, R. (2020). Neuroplasticity in cognitive
and psychological mechanisms of depression: an integrative
model. Molecular Psychiatry, 25(3), 530–543. https://doi.org/
10.1038/s41380-019-0615-x
R Core Team, R. (2014). A language and environment for statistical
computing. R Foundation for statistical computing. http://www.
r-project.org/.
Rijnhart, J. J. M., Valente, M., MacKinnon, D. P., Twisk,
J. W. R., & Heymans, M. W. (2021). The Use of traditional
and causal estimators for mediation models with a binary
outcome and exposure-mediator interaction. Structural
Equation Modeling: A Multidisciplinary Journal, 28(3), 345–
355. https://doi.org/10.1080/10705511.2020.1811709
Robins, J. M. (1994). Correcting for non-compliance in random-
ized trials using structural nested mean models.
Communications in Statistics - Theory and Methods, 23(8),
2379–2412. https://doi.org/10.1080/03610929408831393
Robins, J. M. (1997). Causal inference from complex longitudinal
data. In M. Berkane (Ed.), Latent variable modeling and appli-
cations to causality. Lecture notes in statistics (120) (pp. 69–
117). Springer-Verlag.
Robins, J. M., Blevins, D., Ritter, G., & Wulfsohn, M. (1992). G-
estimation of the effect of prophylaxis therapy for pneumocystis
Carinii pneumonia on the survival of aids patients.
Epidemiology, 3(4), 319–336. https://doi.org/10.1097/
00001648-199207000-00007
Robins, J. M., Rotnitzky, A., & Scharfstein, D. (1999). Sensitivity
analysis for selection bias and unmeasured confounding in
missing data and causal inference models. In M. E. Halloran,
& D. Berry (Eds.), In: Statistical models in epidemiology: The
environment and clinical trials (pp. 1–92). Springer-Verlag.
Rock, P. L., Roiser, J., Riedel, W. J., & Blackwell, A. (2014).
Cognitive impairment in depression: a systematic review and
meta-analysis. Psychological Medicine, 44(10), 2029–2040.
https://doi.org/10.1017/S0033291713002535
Rubin, D. B. (1974). Estimating causal effects of treatments in ran-
domized and nonrandomized studies. Journal of Educational
Psychology, 66(5), 688–701. https://doi.org/10.1037/h0037350
Rubin, D. B. (1978). Bayesian inference for causal effects: The
role of randomization. The Annals of Statistics, 6(1), 34–58.
https://doi.org/10.1214/aos/1176344064
Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys.
Wiley.
Sachs, G., Berg, A., Jagsch, R., Lenz, G., & Erfurth, A. (2020).
Predictors of functional outcome in patients With bipolar dis-
order: Effects of cognitive psychoeducational group therapy
after 12 months. Frontiers in Psychiatry, 11, https://doi.org/10.
3389/fpsyt.2020.530026
Schafer, J. L. (1999). Multiple imputation: a primer. Statistical
Methods in Medical Research, 8(1), 3–15. https://doi.org/10.
1177/096228029900800102
Snyder, H. R. (2013). Major depressive disorder is associated
with broad impairments on neuropsychological measures
of executive function: a meta-analysis and review.
Psychological Bulletin, 139(1), 81–132. https://doi.org/10.1037/
a0028727
Snyder, H. R., Miyake, A., & Hankin, B. L. (2015). Advancing
understanding of executive function impairments and psycho-
pathology: bridging the gap between clinical and cognitive
approaches. Frontiers in Psychology, 6, https://doi.org/10.3389/
fpsyg.2015.00328
Spirtes, P., Glymour, C., & Scheines, R. (1993). Causation, predic-
tion, and search. MIT Press.
Stange, J. P., Alloy, L. B., & Fresco, D. M. (2017). Inflexibility as
a vulnerability to depression: A systematic qualitative review.
Clinical Psychology: Science and Practice, 24(3), 245–276.
https://doi.org/10.1111/cpsp.12201
Stange, J. P., Connolly, S. L., Burke, T. A., Hamilton, J. L.,
Hamlat, E. J., Abramson, L. Y., & Alloy, L. B. (2016).
Inflexible cognition predicts first onset of major depressive epi-
sodes in adolescence. Depression and Anxiety, 33(11), 1005–
1012. https://doi.org/10.1002/da.22513
Strecher, V. J., McClure, J. B., Alexander, G. L., Chakraborty, B.,
Nair, V. N., Konkel, J. M., Greene, S. M., Collins, L. M.,
Carlier, C. C., Wiese, C. l. J., Little, R. J., Pomerleau, C. S.,
& Pomerleau, O. F. (2008). Web-Based smoking-cessation
programs. American Journal of Preventive Medicine, 34(5),
373–381. https://doi.org/10.1016/j.amepre.2007.12.024
Szalma, J. L., & Matthews, G. (2015). 12 Motivation and
Emotion in Sustained Attention.
Teasdale, J. D., Moore, R. G., Hayhurst, H., Pope, M., Williams,
S., & Segal, Z. V. (2002). Metacognitive awareness and preven-
tion of relapse in depression: empirical evidence. Journal of
Consulting and Clinical Psychology, 70(2), 275–287. https://doi.
org/10.1037/0022-006X.70.2.275
Torrent, C., Martinez-Arán, A., del Mar Bonnin, C., Reinares,
M., Daban, C., Solé, B., Rosa, A. R., Tabarés-Seisdedos, R.,
Popovic, D., Salamero, M., & Vieta, E. (2012). Long-term
outcome of cognitive impairment in bipolar disorder. The
Journal of Clinical Psychiatry, 73(7), e899–e905. https://doi.
org/10.4088/JCP.11m07471
Tsiatis, A. A., Davidian, M., Holloway, S. T., & Laber, E. B.
(2020). Dynamic treatment regimes: Statistical methods for pre-
cision medicine. CRC Press.
Usami, S., Murayama, K., & Hamaker, E. L. (2019a). A unified
framework of longitudinal models to examine reciprocal
relations. Psychological Methods, 24(5), 637–657. https://doi.
org/10.1037/met0000210
Usami, S., Todo, N., & Murayama, K. (2019b). Modeling reci-
procal effects in medical research: Critical discussion on the
current practices and potential alternative models. PLOS
ONE, 14(9), e0209133.
Usami, S. (2022). Within-person variability score-based causal infer-
ence: A two-step estimation for joint effects of time-varying treat-
ments. Psychometrika.
VanderWeele, T. J. (2012). Invited commentary: Structural
equation models and epidemiologic analysis. American Journal
of Epidemiology, 176(7), 608–612. 3. https://doi.org/10.1093/
aje/kws213
VanderWeele, T. J. (2015). Explanation in causal inference. Methods
for mediation and interaction. Oxford University Press.
VanderWeele, T. J., & Arah, O. A. (2011). Bias formulas for sen-
sitivity analysis of unmeasured confounding for general out-
comes, treatments, and confounders. Epidemiology, 22(1), 42–
52. https://doi.org/10.1097/EDE.0b013e3181f74493
VanderWeele, T. J., & Hernán, M. A. (2013). Causal inference
under multiple versions of treatment. Journal of Causal
Inference, 1(1), 1–20. https://doi.org/10.1515/jci-2012-0002
Vansteelandt, S., & Joffe, M. (2014). Structural nested models
and G-estimation: The partially realized promise. Statistical
Science, 29(4), 707–731. https://doi.org/10.1214/14-STS493
Vansteelandt, S., & Sjolander, A. (2016). Revisiting g-estimation
of the effect of a time-varying exposure subject to time-
varying confounding. Epidemiologic Methods, 5(1), 37–56.
https://doi.org/10.1515/em-2015-0005
Vaughan, M. (1989). Rule-governed behavior in behavior analy-
sis. In Rule-governed behavior (pp. 97–118). Springer.
Villatte, M., Villatte, J. L., & Hayes, S. C. (2015). Mastering the
clinical conversation: Language as intervention. Guilford
Publications.

Wallace, M. P., Moodie, E. E. M., & Stephens, D. A. (2016).
Model assessment in dynamic treatment regimen estimation
via double robustness. Biometrics, 72(3), 855–864. https://doi.
org/10.1111/biom.12468
Wells, A. (2011). Metacognitive therapy for anxiety and depression.
Guilford Press.
Wells, A., & Matthews, G. (1996). Modelling cognition in
emotional disorder: The S-REF model. Behaviour Research
and Therapy, 34(11-12), 881–888. https://doi.org/10.1016/
S0005-7967(96)00050-2
Wu, Y., Levis, B., Sun, Y., He, C., Krishnan, A., Neupane, D.,
Bhandari, P. M., Negeri, Z., Benedetti, A., & Thombs, B. D.
Psychotherapy Research 1113
(2021). Accuracy of the Hospital Anxiety and Depression
Scale Depression subscale (HADS-D) to screen for major
depression: systematic review and individual participant data
meta-analysis. BMJ, 373, n972.
Yang, S., & Lok, J. J. (2018). Sensitivity analysis for unmeasured
confounding in coarse structural nested mean models. Statistica
Sinica, 28(4), 1703–1723.
Zigmond, A. S., & Snaith, R. P. (1983). The hospital anxiety
and depression scale. Acta Psychiatrica Scandinavica, 67(6),
361–370. https://doi.org/10.1111/j.1600-0447.1983.
tb09716.x

Appendix
A1. G-estimation in Linear Structural Nested
Mean Models (SNMMs)
A linear structural mean model (SMM) (Robins 94;
Vansteelandt & Sjolander, 2016; Vansteelandt &
Joffe, 2014) is one of Robins’ g-methods [Robins
92a], a model for the conditional causal effect (con-
ditional on covariates) of any variable of interest
(e.g. treatment or exposure) on an outcome. It’s
linear in an unknown (finite-dimensional) parameter
c, which quantifies the causal effect of a hypothetical time-varying exposure, with time-varying covariates
level of the exposure on a counterfactual outcome,
denoted Y(a) (potentially unobserved outcome for
the hypothetical exposure level a). One of the basic
assumptions for causal inference is “consistency,”
that Y (a) = Y for those that had exposure level
A = a. Two others are “positivity” (not necessary
for the SNM) and “no unmeasured confounding”
(Vansteelandt & Sjolander, 2016). A linear SMM
may be formulated as
replaced by the propensity score, given by
E(Y |l, a) = b0 + b′1 l + b′2 zp + c′ za (A2)
The estimated causal parameter c is equivalent (in
large samples) to the g-estimator (Vansteelandt & Sjo-
lander, 2016) and has nice properties, like “double-
robustness” (Vansteelandt & Sjolander, 2016). The
generalization to a conditional causal effect of a
and outcome is based on the linear structural nested
mean model (SNMM) [Robins 94]. The exposure
and covariates At and Lt are thought to be assessed
at t = 1, . . . , T − 1 with the history up until time t
denoted by A
t and L

t . Similarly, the future of a vari-
ables is denoted by underbars, Lt = {Lt , Lt+1 , . . .}.
The SNMM can be formulated as
at , 0) − Ys (

E{Ys (
at−1 , 0)|
at−1 ,
l t } = c′ zst at (A3)

where s . t, and Ys (
at , 0) is the counterfactual

E(Y (a) − Y (0)|l) = c′ za
which describes the effect of setting the exposure to a
versus 0 (by some intervention) within subjects with
covariate values l, on the additive scale, where 0 is
some reference value (not necessarily “no
exposure”), and z is a covariate vector that may
depend on (e.g. be a subset of) l. This is the “point
exposure case,” which is the building block for the
general case.
The SMM is fitted in two steps: first, the exposure
is regressed on a vector of covariates L. Let the “pro-
pensity score” P be the fitted value from this
regression (e.g. a linear regression), whether the
exposure is continuous or categorical (even though
the term is usually reserved for a dichotomous
exposure). Second, regress the outcome on the
vector of covariates L, and the terms ZA from the
SMM, and ZP where the exposure A has been
(A1) outcome where the exposure is set to a
t up until time
t, and zero thereafter. The “no unmeasured confound-
ing” (sequentially conditional exchangeability)
assumption  is formulated as
Y t+1 (
at−1 , 0) At |L
t−1 = a

t, A
t−1 (conditional inde-
pendence between At and all future values of
Ys (

at , 0)).
This construct facilitates the estimation of, for
each observed outcome Ys at time point s, not only
the causal effect of the exposure immediately preced-
ing the outcome (time s − 1), but also earlier
exposures (times s − 2 . . . ) for which the effect of
later exposures are subtracted from the outcome.
This way the SNMM can be broken down into a
sequence of SMMs and fitted in the same way as in
the point exposure case.
First, the exposure on each time = 1, . . . , T − 1,
is regressed on the history of exposures A
t−1 , and
covariates L
t , for example in a linear regression,
and the fitted value from this regression, called the
1114 H. B. Jacobsen et al.
propensity score, is given by Pt
at−1 ,
l t ) = g0 + g1 at−1 + g′2 lt
E(At |

The causal effect of the exposure at each time on the
subsequent outcome is found by viewing the data as a
sequence of point-exposure SMMs, with the consist-
ency assumption that Yt (
at−1 ) equals Yt ,
t = 1, . . . , T for those with observed exposure
history A
t−1 = a

t−1 , and with the univariate
outcome regression in (A2) replaced with a repeated
measures regression model like the Generalized Esti-
mating Equation model (GEE) with independence
working correlation (Vansteelandt & Sjolander,
2016). The history A
t−2 and L

t−1 are considered
baseline covariates (possibly different than the
vector of covariates in the exposure regression) and
conditioned on, as well as the sequence of propensity
scores Pt−1
E(Yt |

t−2 + b′2
l t−1
at−1 ,
l t−1 ) = b0 + b1 a
+ b′3 zt,t−1 pt−1
+ c′ zt,t−1 at−1
This yields a preliminary estimate ĉ(0) . To estimate
the effect of At−2 , At−3 . . . again each one is con-
sidered a point exposure with the predictions of

s−2 , 0), Ys (A
Y s (A
s−3 , 0) . . . as outcomes. The
unbiased prediction of Ys (A
t , 0), here denoted by
Hst for arbitrary s and t, s . t, is found by the use
of ĉ(0) to subtract the cumulative effect of observed
exposure past time t from the observed outcome

s−1
Hst = Ys − ĉ(0) ′Zsu Au
u=t+1
The updated and improved estimate of c is found by
considering Hst for all s and t as repeatedly measured
outcomes, with a new independence GEE in the
model
at ,
l t ) = b0 + b1 a
E(Hst |

t−1 + b′2
l t + b′3 zst pt
+ c′ zst at
which yields ĉ(1) . Standard errors by bootstrap are
recommended (Vansteelandt & Sjolander, 2016).
The structural nested mean model (SNMMs)
[Robins 94] with g-estimation is particularly well
suited for estimating an effect of a time-varying con-
tinuous exposure, and to assess effect-modification
by time-varying covariates, not possible with the
more popular marginal structural model (MSM)
(Vansteelandt & Sjolander, 2016).
Different causal hypotheses can easily be assessed
with different SNMMs. In the application presented,
(A4) some candidates were:
at , 0) − Ys (

E(Ys (
at−1 ,
l t ) = c′ zst at
at−1 , 0)|

= (c0 I(s = t + 1) + c1 (s − t − 1))at (A8)
at , 0) − Ys (

E(Ys (
at−1 ,
l t ) = c′ zst at
at−1 , 0)|

= (c0 + c1 trt + c2 lt + c3 trt × lt )at (A9)
at , 0) − Ys (

E(Ys (
at−1 ,
l t ) = c′ zst at
at−1 , 0)|

= (c0 + c1 f (t))at (A10)
for = 1, 2, 3 s . t, and I(.) is the indicator function,
such that I(arg) = 1 if arg = true.
The SNMM in (A8) differentiates between short-
and long-term effects, e.g. how a long-term effect
may decrease over time. The SNMM in (A9) and
(A10) parameterizes short-term effects. (A9) allows
a treatment-group difference in effect, time-varying
effect-modification from lt , and also a treatment-
group difference in the effect-modification, in the
(A5) most general form. The SNMM in (A10) is a
special case of (A9) with effect modification from a
deterministic function of time.
Strengths of the SNMMs and the associated
method of g-estimation, compared to the MSM
and associated inverse probability weighting, are
more efficient effect estimates and the so-called
“doubly robustness” property, which provides some
protection against model-misspecification (Vanstee-
landt & Sjolander, 2016).
(A6)
A2. Multiple Imputation
Compared to a “complete case analysis,” both bias
reduction and efficiency gain can be achieved by
multiple imputation (MI). The availability of stan-
dard software leads to an increase in applications
of MI in medical research after 2005. An upper
limit for the proportion of missing values in key
variables has been postulated, but with little evi-
(A7) dence to support it (Madley-Dowd et al., 2019).
A more useful tool for determining potential effi-
ciency gain from MI is the fraction of missing
information (FMI), approximated by
FMI = r/(1 + r) (for a high number of imputa-
tions), where r is the relative increase in variance
due to the missingness given by (Madley-Dowd
2019; Schafer, 1999)


1 s2between
r = 1+ (A11)
m s2within
 Psychotherapy Research 1115

with m as number of imputations, the between variance on A
t−1 , L

t . Since the SNMM already conditions on
is the sample variance of the coefficients across imputa- exposure and confounder history up to time t, no
tions, and the within variance is the sample mean of further adjustment is needed for no loss-to-follow-up
the estimated variances across imputations. The FMI up to time t. Therefore, the weights are usually less vari-
is a parameter-specific measure that quantifies the loss able than the weights for the marginal structural model
of information due to missingness, while accounting (Vansteelandt & Sjolander, 2016). In the present appli-
for the amount of information retained by other variables cation, the weights for the outcomes Hst (in the GEE)
(Madley-Dowd et al., 2019). If the estimated FMI (in take the form (A0 = 0):
%), say for an exposure effect parameter, is less than
the proportion of missing values in the exposure, it
1)
Pr(C2 = 0|C1 = 0, A0 , L
means that other variables contain information about w21 = I(C2 = 0)
1)
Pr(C2 = 0|C1 = 0, A1 , L
this parameter which is recovered by the imputation,
even in the case of a high proportion of missing values.
1)
Pr(C2 = 0|C1 = 0, A0 , L
However, for a high proportion of missing, unbiasedness w31 = I(C3 = 0)
1)
Pr(C2 = 0|C1 = 0, A1 , L
in the MI relies heavily on the untestable assumptions of

1)
Pr(C3 = 0|C2 = 0, A0 , L
MAR, and on no misspecification in the imputation and ×
analysis models. A low number of imputations in MI is

2)
Pr(C3 = 0|C2 = 0, A2 , L
usually sufficient, by an argument based on relative effi-
ciency (variance compared to the case with a very large
2)
Pr(C3 = 0|C2 = 0, A1 , L
w32 = I(C3 = 0)
2 ) (A13)

2, L
number of imputations). For example, with an FMI of Pr(C3 = 0|C2 = 0, A
20%, 10 imputations correspond to a relative efficiency
above 98% (Schafer, 1999).
1)
Pr(C2 = 0|C1 = 0, A0 , L
w41 = I(C4 = 0)
1)
Pr(C2 = 0|C1 = 0, A1 , L

1)
Pr(C3 = 0|C2 = 0, A0 , L
A3. Censoring from Loss-to-Follow-up ×
2, L

2)
Pr(C3 = 0|C2 = 0, A
To adjust for potential selection-bias from loss-to-

1)
Pr(C4 = 0|C3 = 0, A0 , L
follow-up, inverse probability of censoring weighting ×
is possible, under the “missing at random assump-
3, L
Pr(C4 = 0|C3 = 0, A
3)
tion,” which in the SNMM model means that at
each time, the missingness in the outcome is inde-
2)
Pr(C3 = 0|C2 = 0, A1 , L
w42 = I(C4 = 0)

2)
pendent of future exposures, covariates and out- Pr(C3 = 0|C2 = 0, A2 , L
comes, conditional on exposures, covariates and
1)
Pr(C4 = 0|C3 = 0, A0 , L
outcomes measured up to that time (Vansteelandt ×
2, L

2)
Pr(C4 = 0|C3 = 0, A
& Sjolander, 2016). Adjustment for censoring by
loss-to-follow-up is achieved by weighting the
2, L
Pr(C4 = 0|C3 = 0, A
3)
outcome regression, with inverse probability w43 = I(C4 = 0)

3)
weights, similar (but not identical) to the weights Pr(C4 = 0|C3 = 0, A3 , L
used in fitting the marginal structural models. If
Ct = 1 is an indicator of a person being lost to
follow-up by time t, and zero otherwise, the “stabil-
ized weights” that are used and with the least varia-
A.4 The Lasso
bility, express the conditional probability of
Cs = 0, s . t (that the person will continue to stay The lasso is a “shrinkage” method with high per-
in the study) conditional on exposure and covariate formance with respect to minimizing prediction
history, divided by the same probability with error in a regression model. Increasing the number
updated exposure and covariate history (Vanstee- of predictors in a regression model usually means
landt & Sjolander, 2016) less bias and more variance, leading to overfitting.
To avoid overfitting, the lasso shrinks the coefficients
T
t−1 , L
Pr(Cs = 0|Cs−1 = 0, A
t) by minimizing the ℓ1 penalty function, which results
wTt = I(CT = 0)

in some of the coefficients are set equal to zero when
s=t+1 Pr(Cs = 0|Cs−1 = 0, As−1 , Ls−1 ) the predictors are found to have small influence. In
(A12) the linear regression model

The expression in (A12) reduces to 1 if censoring at 

p
each time s . t has no residual dependence on yi = b0 + xij bj + 1i (A14)
exposure and covariate values past time t, conditional j=1
1116 H. B. Jacobsen et al.

The lasso solution can be written as where s is the amount of shrinkage. The optimal s
that minimizes prediction error, can be found by
2
argmin  
n p cross-validation, so that the data is divided into let’s
b̂lasso = yi − b 0 − xij bj (A15) say 10 equal-sized parts, 9 of which is used to fit a
b i=1 j=1 model with a specific shrinkage and one part to calcu-
late the prediction error. When this is repeated for

p
each part, the mean prediction error is a good
subject to |bj | ≤ s
estimate.
j=1