The Digit Symbol Substitution Test (DSST):

Psychometric Properties and Clinical Utility in Major Depressive Disorder

Judith Jaeger1 and Silvia Zaragoza Domingo, MA2
1
Clinical Professor, Department of Psychiatry and Behavioral Sciences, Albert Einstein College of Medicine, New York, NY, USA
P.1.j.033 2
Neuropsychological Research Organization s.l, Barcelona, Spain

BACKGROUND
• Major depressive disorder (MDD) is characterized by cognitive
dysfunction of modest but clinically meaningful severity that can
fluctuate over the disease course independently of mood symptoms.1
Cognitive dysfunction has been associated with functional
impairment.2 Subjectively and behaviorally, this is manifested in ways such
as slowed responding, difficulty focusing or distractibility, impairments
holding information in working memory, and difficulty planning complex
activities. Clinicians are accustomed to monitoring shifts in mood symptoms
via self-report measures, and cognitive functioning has typically been
evaluated similarly through subjective complaints about cognitive dysfunction
and its functional consequences, as expressed by the patient.
• In contrast to subjective reports, neuropsychological tests permit
objective cognitive assessments that are less vulnerable to the
impacts of patient insight, values, and concurrent situational events (Figure
1). Importantly, subjective complaints often do not correlate with objective
Figure 1. Symptoms and Signs of Cognitive Dysfunction
cognitive dysfunction as measured using neuropsychological tests.
Figure 1. Symptoms and Signs of Cognitive Dysfunction
Objective:
• Neuropsychological tests
Evaluation Cognitive
filters: domains:
Attention
WHAT CONSTRUCT DOES THE DSST MEASURE? CLINICAL RELEVANCE OF CHANGES IN THE DSST
• The DSST was developed as an experimental tool for studying human associative • The DSST’s sensitivity to change both acutely and chronically (ie, with age)
learning.4 As a result of its favorable practical and psychometric qualities, the is one of its hallmark features. As a result, the DSST has been used as a
DSST was incorporated into the original Wechsler Bellevue IQ test battery.3 standard tool in clinical pharmacology studies from as early as the 1970s.12
• Its utility as a clinical tool in neuropsychology became evident when it was • The clinical meaningfulness of the magnitudes of change on the DSST can
shown to reliably distinguish patients with brain damage during the screening help clinicians understand utility of examining performance on the DSST as
of soldiers. an index of cognitive dysfunction and change.
• Good test performance on the DSST requires intact functioning on the • The importance of change for the individual might be considered through the
attributes of motor speed, attention, visuoperceptual functions (including lens of the DSST’s sensitivity to change in the context of clinical and experimental
scanning), and functions associated with writing or drawing (ie, basic manual conditions that could serve as useful benchmarks for the meaningfulness of
dexterity). change of a certain magnitude.
• Performance on the DSST may also be affected by associative learning: If • Patients diagnosed with either amnestic or non-amnestic mild cognitive
pairings are rapidly learned following the first few trials, performance speed impairment (MCI) (N=3117) were studied with respect to the impact of
will improve because the patient will not need to check the accuracy of each depressive versus cognitive symptoms on impaired functioning.13 In the
pairing. total sample and in the amnestic MCI subgroup sample (n=2488), performance on
the DSST partially mediated the effect of depression on and fully mediated
• The decision to consciously engage in this learning strategy to improve
performance speed calls upon the executive functions of planning and the effect of executive function on functional impairment (as measured by the
strategizing. 10-item Pfeffer Functional Activities Questionnaire) (P<0.0001).
Subjective: • Working memory - another executive function - is also likely required to keep • As mentioned before, in MDD patients studied 6 months after discharge from
• Physician rating task rules in mind during the test and for the continual updating of required hospital for an MDD episode, the DSST was strongly associated with the
• Self-report scales level of functioning achieved in work, school, and housing (odds ratio: 19.95).
digit-symbol pairs.
Commonly experienced • Laux and Lane5 noted in their review that DSST is often more highly correlated Meaning, a 1-SD difference on the DSST was associated with a nearly 20-
cognitive difficulties:
with verbal measures, suggesting that performance also involves some fold increase in the odds of obtaining a 1-point better rating on the 7-point
“I had trouble focusing on what you just
aspects of written language functioning as well. Multidimensional Scale of Independent Functioning.11
said; could you repeat it?”
Figure 4. Comparative Assessment of the Effect Sizes on the DSST (Cohen’s d)

Processing “I didn’t realize the traffic light turned

• In most populations, digit-symbol coding performance is highly correlated Figure 4. Comparative Assessment of the Effect Sizes on the DSST (Cohen’s d)
speed red until it was too late.” with overall cognitive performance. For example, in schizophrenia, Burton et
Working “I can’t remember the telephone
al6 found 3 principal cognitive domains (ie, processing speed, attention and Vortioxetine 20 mg (McIntyre et al, 2014)
memory number I just got from the operator.”
working memory, and learning) through a factor analysis of the MATRICS

Depressed Patients
Vortioxetine 10 mg (McIntyre et al, 2014)

Chronic Effects in
Executive
“I can’t figure out what I need from the Consensus Cognitive Battery and the BAC symbol-coding score correlated Vortioxetine 10–20 mg (Mahableshwarkar et al, 2015)
supermarket right now to make dinner
Clinician/rater functioning Patient
tonight.” with each of these factors at r values exceeding 0.80. Vortioxetine 5 mg in elders (Katona et al, 2012)
Episodic “Where did I put my house keys?” • Beres and Baron7 evaluated the effect of active training on the DSST as a way Duloxetine 60 mg (Mahableshwarkar et al, 2015)
of dissecting its cognitive components. In this study, 20 administrations of a
memory “I can’t remember how to get home.”
Duloxetine 60 mg in elders (Katona et al, 2012)
Clinicians/raters use a variety of tools to evaluate cognitive dysfunction and cognitive improvement (column 1) in various cognitive 30-second DSST were performed every day for 5 days, with new number-
domains (column 2). Examples of difficulties for each respective cognitive domain are given in column 3. Figure is being reprinted
with permission from Paul Maruff, Ph.D. code pairs each day. The study concluded that both elderly and young patients Acute caffeine 50 mg @ 30 min (Kassis et al, 2013)

improved their scores within each day and between days (Figure 3). Acute caffeine 50 mg @ 120 min (Kassis et al, 2013)

Healthy Patients
Acute Effects in
• Depressed patients sometimes report more or report less cognitive dysfunction Caffeine deprivation (Lane, 1997)

than is observable using objective tests. This relationship is complicated by Figure 3. Clinical Applicability of What the DSST Measures: Simple Motor Speed,
Diphenhydramine 150 mg (Roth et al, 1987)
Associative Learning, and Executive Functioning
the fact that disability arising from cognitive impairment can exacerbate Figure 3. Clinical Applicability of What the DSST Measures: Simple Motor Speed, Associative Learning, and Executive
Functioning 40 Lorazepam 2 mg (Pompeia et al, 2008)
subjective distress.
Day 1 Day 2 Day 3 Day 4 Day 5 Alcohol BAC 0.088 (Mattila et al, 1998)
• The relationship between change in mood and change in cognitive functioning
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
is also complicated. For example, an improvement of mood symptoms can 35 Effect Size
result in an improvement in subjective ratings of cognitive dysfunction in the Positive effect sizes indicate improvements in cognitive functioning; negative effect sizes indicate impairments in cognitive
absence of objective improvement in cognitive performance. Conversely,
Mean Number of Symbols Completed

functioning. The top portion of the graph shows the effects of antidepressant treatments on cognitive functioning, while the bottom
portion illustrates the effects of other agents known to affect cognitive functioning.
objective improvement in cognitive performance may not be directly perceived 30
by a patient, even with functional improvement, if mood symptoms persist.
• The DSST has recently been investigated in 2 clinical trials of vortioxetine
• Hence, objective testing offers clinically important information.
(an antidepressant for the treatment of MDD) in patients with MDD, with the
Neuropsychological tests, such as the Digit Symbol Substitution Test (DSST), 25
relative effect compared to placebo highlighted in Figure 4.14-16 The effect of
can be used by researchers and clinicians to objectively monitor change in
duloxetine (an antidepressant with multiple indications), utilized as an active
cognitive performance over time.
20
comparator in 2 studies,15,16 is also included.
• The DSST may be a valuable assessment tool for estimating the magnitude
• For a healthy person to appreciate what might be the subjective experience
of cognitive dysfunction and - perhaps more importantly - for tracking change
of a cognitive effect of comparable magnitudes, the second half of Figure 4
over time within a patient. Young (18–32 yrs)
15 shows the acute effects of a range of compounds relative to placebo with fairly
• Evans and colleagues2 conducted a recent systematic literature review of the Old (63–80 yrs)
well-understood effects on DSST performance in young healthy volunteers.17-22
relationship between cognitive and psychosocial functioning in patients with 0
MDD and found that (1) depressed patients had cognitive deficits in various 1 10 20 1 10 20 1 10 20 1 10 20 1 10 20 • By way of illustration, the acute effect of lorazepam 2 mg (a sedative
domains that were associated with different measures of psychosocial Reproduced from Beres & Baron, Reproduced The fromJournal of Gerontology,
Beres & Baron Volume
(1981). Points represent
Number of Trials
36, Issue
mean number 5, completed
of symbols pg 594.byCopyright © 1981 Gerontological
for the treatment of anxiety disorders or symptoms) resulted in a
functioning, and (2) cognitive functioning appears to be broadly associated
older and younger patients on each of the 100 training trials. Straight lines have been fitted to values
Society of America. Points represent
for the 20 trialsmean
of each ofnumber of days.
the 5 training symbols completed by older and younger patients on each of the 100 decline relative to placebo of 0.52 on the DSST, which is almost
with functional impairment in individuals with MDD (Table 1).
training trials. Straight lines have been fitted to values for the 20 trials of each of the 5 training days.
identical to the magnitude of improvement relative to placebo seen
Table 1. Systematic Review of Neurocognitive and Psychosocial Functioning in Patients with Depression after 8 weeks with vortioxetine in adults with MDD in the McIntyre et al
• Because learning of the number-code pairs could only improve performance
Table 1. Systematic Review of Neurocognitive and Psychosocial Functioning in (2014) trial. The more modest benefit seen in the Mahableshwarkar et
Patients with Depression between retests on the same day, it would appear that associative learning
al (2015) study is of a similar magnitude as the acute negative effect
X = Neurocognitive test assessed X = Significant association with functional outcome
contributes to performance but is not sufficient.
of 150 mg of diphenhydramine (an antihistamine with sedative
Study n Attention
Psychomotor
speed
Verbal
memory
Visual
memory
Executive
functioning
Functioning
• Hence, it is likely that some other cognitive operation is in force that does effects). Both of these compounds (lorazepam and diphenhydramine) at
Baune et al, 2010 70 X X X
Employment improve with practice. Simple motor speed does not typically show sustained these doses produce cognitive adverse effects considered to be sufficient to
status

Godard et al, 2011 16 X X X X LIFE-RIFT

improvement with practice. The executive function of strategizing or consciously receive a label indicating that it is not safe to drive or operate heavy machinery
Gupta et al, 2013 33 X X X X
LIFE-RIFT*, exerting effort to learn the pairings is most likely at play here. while under their effect. Although benchmarks from acute adverse effects in
SSPA, AFT

McCall and Dunn, 2003 77 X X IADL, PSMS • There is good agreement that the DSST is a polyfactorial measure whose healthy young people may not be comparable to the sustained 8-week benefit
Naismith et al, 2007 21 X
performance can be disrupted by impairments in a range of cognitive in patients with MDD, it may be possible to consider these benchmarks in
X X X BDQ†

domains, among which are motor speed, attention, and working memory/ trying to understand the subjective experience or meaningfulness of effects
Shimizu et al, 2013 43
of these magnitudes as reflected in DSST performance differences.
X X X X SF-36‡
executive functioning, domains that have shown impairment in studies of
Jaeger et al, 2006 48 X X X X X MSIF MDD patients.
Withall et al, 2009 48 X X X X SOFAS CLINICAL APPLICATIONS
Adapted from Evans et al. (2014). *Recreational subscale only. †Physical disability only. ‡General health perceptions only. AFT, Advanced Finances Task;
Adapted from
BDQ,Evans et alQuestionnaire;
Brief Disability (2014). *Recreational subscale
IADL, Instrumental Activities only.
of Daily †Physical
Living; disabilityInterval
LIFE-RIFT, Longitudinal only.Follow-Up
‡General health
Evaluation perceptions
– Range of Impaired only. DSST PERFORMANCE IN PATIENTS WITH MDD • The use of DSST for clinical decision making in MDD has not been widely
AFT, Advanced Finances
Functioning Task;
Tool; MSIF, BDQ, Brief
Multidimensional Disability
Scale of IndependentQuestionnaire; IADL,Self-Maintenance
Functioning; PSMS, Personal
and Occupational Functioning Assessment Scale; SSPA, Social Skills Performance Assessment
Instrumental
RIFT, Longitudinal Interval Follow-Up Evaluation – Range of Impaired Functioning Tool; MSIF, Multidimensional Scale of
Activities
Scale; of Daily
SF-36, Short-Form 36;Living; LIFE-
SOFAS, Social
• Studies have shown neuropsychological performance is impaired in patients tested, but given its known sensitivity to cognitive dysfunction (acknowledging
Independent
NOTES: (1) EVANS 2014 ALSO INCLUDES 2 MORE STUDIES NOT INCLUDED HERE. Most likely because they
Functioning;
included PSMS,
only separate Personal
analyses Self-Maintenance
of cognition Scale;
and functioning and itSF-36, Short-Form
would be misleading36; SOFAS,
to state there Social
was no and Occupational with MDD, even from the first episode.8 its lack of specificity for any single kind or domain of dysfunction) it may be
Functioningsignificant
Assessment Scale;if SSPA,
association the test Social Skills
was never Performance
performed. Assessment.
(2) Changed Naismith et al, 2007 — significant association
between processing/psychomotor speed and the BDQ physical disability subscale. No significant association with SF- • A recent meta-analysis on executive dysfunction in MDD estimated one of many potential tools worth exploring for clinical application.
12, so this has been removed. (3) Added footnote markers to denote that only subscales of specific functioning
assessments were significant (i.e., BDQ, SF-36). (4) Changed Shimizu et al, 2013 — significant association between magnitudes of impairment relative to healthy controls averaging about 0.5 • It is important to remember that the DSST is subject to modest practice effects.
INTRODUCTION TO THE DSST
verbal memory and the SF-36 General health perceptions subscale
standard deviations (SDs) on a wide array of neuropsychological tests and This has been shown in sports concussion literature; for example, a study
• The DSST is a paper-and-pencil cognitive test presented on a single sheet concluded that “MDD is reliably associated with impaired performance on by Collie et al23 reported that concussed symptomatic athletes showed no
that requires the patient to copy, into spaces below rows of numbers, the neuropsychological measures of executive functioning.”9 Interestingly, the significant change from baseline, in contrast to non-concussed (control) and
symbols that are matched to each number according to a key located on the magnitude of DSST performance deficit relative to controls studied in this concussed asymptomatic athletes, who displayed significant improvements.
top of the page (Figure 2). meta-analysis of 1904 patients (mean±SD age, 50.5±17.6 years) from 22 Hence, for an individual patient, improvement in performance over time is
studies yielded a mean standardized effect size comparable to the effect possible even in the absence of an effective treatment.
• Often referred to as “coding” or “symbol coding,” the paradigm of the DSST
across the domain of executive functioning as a whole (0.55; P<0.001 [95% • Computerized tests sensitive to the cognitive dysfunction observed
has survived almost unchanged in its 1939 form3 to the most recent version
confidence interval: 0.34–0.75]).9 in MDD may also be useful by reducing or eliminating practice effects, hence
(IV) of the Wechsler Adult Intelligence Scale. Comparable versions can be
seen in the Brief Assessment of Cognition in Schizophrenia (BACS) and the • Similar findings were reported in elderly depressed patients (mean±SD age, making them potentially even more sensitive to clinically meaningful change.
Repeatable Battery for the Assessment of Neuropsychological Status. 70.3±6.4 years), where DSST performance is worse in late versus early onset
of depression (P<0.04).10 CONCLUSIONS
• The DSST is perhaps the most commonly used test in all of neuropsychology •
due to its brevity and high discriminant validity; however, the question of • In patients with MDD (aged 18–59 years) who were studied for 6 months
“what it measures” still has no definitive answer. after discharge from hospital for an MDD episode, the DSST was strongly • The DSST is a valid and sensitive measure of cognitive
associated with the level of functioning achieved in work, school, and home,
• Although the DSST is not specific as to the nature of the deficit in the absence
yielding an odds ratio of 19.95 (scaled to SD units).11 dysfunction impacted by many domains, including cognitive
of further testing, it is still widely regarded as a clinically useful tool because
of its sensitivity to the presence of cognitive deficits in a wide range of patient -- A difference of 1 SD on the DSST was associated with a nearly 20-fold functions that are demonstrated to be impaired in patients
populations. increase in odds of obtaining a 1-point better rating on the 7-point global with MDD.
scale of the Multidimensional Scale of Independent Functioning, an index
• As a result of the DSST’s high level of sensitivity to general cognitive • The DSST is a valid indicator of change in cognitive
of real-life functioning in the community.
impairment in a range of brain diseases and conditions (including MDD), it
is Figure
sensitive to change and is widely used in clinical pharmacology trials and functioning, and performance on the DSST correlates with
2. The Digit Symbol Substitution Test
REFERENCES
clinical practice. 1. Kraepelin E. Manic-depressive insanity and paranoia. E. & S. Livingstone; 1921.
real-world functional outcomes (eg, the ability to accomplish
Figure 2. The Digit Symbol Substitution Test
2. Evans VC, Iverson GL, Yatham LN, Lam RW. The relationship between neurocognitive and psychosocial functioning in major depressive disorder: a systematic
review. J Clin Psychiatry. 2014;75(12):1359-1370.
3. Wechsler D. The measurement of adult intelligence. Baltimore: Williams & Wilkins; 1939.
everyday tasks) and recovery from functional disability.
4. Boake C. From the Binet-Simon to the Wechsler-Bellevue: tracing the history of intelligence testing. J Clin Exp Neuropsychol. 2002;24(3):383-405.
5. Laux L, Lane D. Information processing components of substitution test performance. Intelligence. 1985;9(2):111-136.
6. Burton CZ, Vella L, Harvey PD, et al. Factor structure of the MATRICS Consensus Cognitive Battery (MCCB) in schizophrenia. Schizophr Res. 2013;146(1-3):244-
248.
• Importantly, the DSST has been demonstrated to be
7. Beres CA, Baron A. Improved digit symbol substitution by older women as a result of extended practice. J Gerontol. 1981;36(5):591-597.
8. Lee RS, Hermens DF, Porter MA, Redoblado-Hodge MA. A meta-analysis of cognitive deficits in first-episode major depressive disorder. J Affect Disord. sensitive to change in cognitive functioning in patients with
2012;140(2):113-124.
9. Snyder HR. Major depressive disorder is associated with broad impairments on neuropsychological measures of executive function: a meta-analysis and review.
Psychol Bull. 2013;139(1):81-132. MDD and may offer an effective means to detect clinically
relevant treatment effects.
10. Nebes RD, Pollock BG, Houck PR, et al. Persistence of cognitive impairment in geriatric patients following antidepressant treatment: a randomized, double-blind
clinical trial with nortriptyline and paroxetine. J Psychiatr Res. 2003;37(2):99-108.
11. Jaeger J, Berns S, Uzelac S, Davis-Conway S. Neurocognitive deficits and disability in major depressive disorder. Psychiatry Res. 2006;145(1):39-48.
12. Mattila MJ, Aranko K, Mattila ME, Paakkari I. Effects of psychotropic drugs on digit substitution: comparison of the computerized symbol-digit substitution and
traditional digit-symbol substitution tests. J Psychopharmacol. 1994;8(2):81-87.
13. Brown PJ, Liu X, Sneed JR, et al. Speed of processing and depression affect function in older adults with mild cognitive impairment. Am J Geriatr Psychiatry.
2013;21(7):675-684.
14. McIntyre RS, Lophaven S, Olsen CK. A randomized, double-blind, placebo-controlled study of vortioxetine on cognitive function in depressed adults. Int J
Neuropsychopharmacol. 2014;17(10):1557-1567.
15. Katona C, Hansen T, Olsen CK. A randomized, double-blind, placebo-controlled, duloxetine-referenced, fixed-dose study comparing the efficacy and safety of Lu Funding: This study was funded by H. Lundbeck A/S and Takeda Pharmaceutical Company, Ltd.
AA21004 in elderly patients with major depressive disorder. Int Clin Psychopharmacol. 2012;27(4):215-223.
16. Mahableshwarkar AR, Zajecka J, Jacobson W, Chen Y, Keefe RS. A randomized, placebo-controlled, active-reference, double-blind, flexible-dose study of the Conflict of Interest: Judith Jaeger is a former employee of Cogstate Corporation and AstraZeneca R&D and now, as
efficacy of vortioxetine on cognitive function in major depressive disorder. Neuropsychopharmacology. 2015;22(8):2025-2037.
17. Roth T, Roehrs T, Koshorek G, Sicklesteel J, Zorick F. Sedative effects of antihistamines. J Allergy Clin Immunol. 1987;80(1):94-98. President of CognitionMetrics, LLC, consults to several pharmaceutical company clients, including Lundbeck and Takeda.
18. Tucker AM, Whitney P, Belenky G, Hinson JM, Van Dongen HP. Effects of sleep deprivation on dissociated components of executive functioning. Sleep.
2010;33(1):47-57. Silvia Zaragoza Domingo declares having no conflict of interest.
19. Kassis O, Katz N, Ravid S, Pillar G. Double-blind placebo and active (caffeine) controlled study to examine the effects of the herbal nutritional supplement beverage
“Wake up” on vigilance and function after lunch. Isr Med Assoc J. 2013;15(8):419-423.
20. Lane JD. Effects of brief caffeinated-beverage deprivation on mood, symptoms, and psychomotor performance. Pharmacol Biochem Behav. 1997;58(1):203-208.
21. Pompéia S, Pradella-Hallinan M, Manzano GM, Bueno OFA. Effects of lorazepam on visual perceptual abilities. Hum Psychopharmacol Clin Exp. 2008;23(3):183-
192. Poster presented at the 29th ECNP Congress, 17-20 September, 2016, Vienna, Austria
22. Mattila MJ, Vanakoski J, Kalska H, Seppala T. Effects of alcohol, zolpidem, and some other sedatives and hypnotics on human performance and memory. Pharmacol
The test is printed on a page which is handed to the patient with the instruction to copy into the blank below each numeral, Biochem Behav. 1998;59(4):917-923.
the symbol with which it is associated as shown in the key at the top of the page. 23. Collie A, Makdissi M, Maruff P, Bennell K, McCrory P. Cognition in the days following concussion: comparison of symptomatic versus asymptomatic athletes. J Neurol
Neurosurg Psychiatry. 2006;77(2):241-245.