Evidence of Validity and Accuracy of An Implicit Measure To Assess The Depressive Trait

Trends in Psychology (2023) 31:690–715
https://doi.org/10.1007/s43076-022-00145-y
ORIGINAL ARTICLE
Evidence of Validity and Accuracy of an Implicit Measure

to Assess the Depressive Trait
Nathalia Melo de Carvalho1 · Rafael Valdece Sousa Bastos1 ·

Jean Carlos Natividade1
Accepted: 17 January 2022 / Published online: 28 January 2022

© Associação Brasileira de Psicologia 2022
Abstract
In the Big5 model, the depressive trait is an underlying neuroticism factor that indi-
cates a tendency to experience negative emotions, low expectations for the future,
and loneliness. The present study sought to build an implicit measure to assess the
depressive trait and search for evidence of its validity. In study 1, we elaborated and
selected items through an implicit task that was based on accuracy and response
times. In study 2, the items were applied to an implicit association test. The results
indicated that the measure had satisfactory evidence of validity, including a one-
factor structure, and relationships with other theoretically expected variables. Fur-
thermore, the reliability indices of the measure were adequate. The test–retest cor-
relation coefficient was higher than the values that were found in the literature. This
research presented an implicit measure built with an innovative item selection pro-
cess and which proved to be suitable to access the depressive trait of personality.
Keywords Implicit assessment · Depressive trait · Personality traits · Big five

factors · Test validity · Test accuracy · Implicit social cognition
Since the first studies of personality, some traits have been shown to be especially
relevant because of their predictive power on behaviors and other outcomes in the
lives of individuals (Gregory, 2014). The depressive trait, for example, is consid-
ered predictive of well-being, professional performance, and mental disorders (Klein
et al., 2011; Quevedo & Abella, 2011; Slaughter & Kausel, 2009). People with
higher levels of the depressive trait and other traits that are associated with nega-
tive emotions are more likely to develop clinical conditions of depression, a public
health problem that affects 4.4% of the world’s population (World Health Organiza-
tion, 2017).
* Jean Carlos Natividade

jeannatividade@gmail.com
1
Department of Psychology, Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro,
Brazil
1Vol:.(1234567890)
3
Trends in Psychology (2023) 31:690–715 691
The most common way to detect levels of the depressive trait is through self-
report instruments (e.g., Costa & McCrae, 2007; Nunes et al., 2010; Soto & John,
2017). One limitation of these measures, however, is the influence of bias that results
from social desirability and the handling of self-image (Payne & Gawronski, 2010).
People who are highly motivated to build a favorable image of themselves find the
opportunity to falsify answers in self-report instruments because they have suffi-
cient time to reflect and deliberate on the presented items (Fazio & Olson, 2003). In
recent decades, implicit measures have emerged in an attempt to overcome this limi-
tation. They have been shown to be useful in capturing more spontaneous responses
(Payne & Gawronski, 2010). A recent meta-analysis showed that the implicit asso-
ciation test (IAT) is the most used measure to assess the implicit self-concept of
personality (De Cuyper et al., 2017). The meta-analysis also revealed, however,
that few studies developed empirical procedures to guarantee evidence of validity
based on the content of implicit measures of the self-concept. Most studies failed to
assess personality more narrowly by measuring facets that underlie the Big5 factors.
Thus, the present study sought to construct an IAT to assess the depression facet of
the Big5 model and search for evidence of its validity. Additionally, we developed
an empirical procedure based on accuracy and response times to select items for
implicit measures.
Depressive Trait in the Big5 Model
The Big5 model has been shown to be replicable and well accepted in studies of
personality (Barenbaum & Winter, 2010). This model has a basic assumption that
the idea that individual human differences can be identified in the natural language
of people (John et al., 1988). This can be assumed because, throughout evolu-
tion, people would have invented words to describe individual characteristics that
were important for social interaction (John et al., 1988). Based on this assumption,
researchers who searched for terms to describe individual differences in dictionaries
and books within the scope of different cultures found that personality characteris-
tics could be grouped into five explanatory dimensions (e.g., Goldberg, 1992; Hutz
et al., 1998).
In theory, the Big5 factors are independent of each other and refer to a continuum.
Every person is at some level (low, medium, or high) in the latent trait spectrum of
each of the five factors (Costa & McCrae, 2007; John et al., 2010). In short, they can
be defined as follows: extraversion (a tendency to seek stimulation in interactions
with others), agreeableness (a tendency to behave in a prosocial manner), consci-
entiousness (a tendency to be obstinate and organized to achieve established goals),
neuroticism (a tendency to experience negative emotions and emotional instabil-
ity), and openness to experience (a tendency to seek new and enriching experiences;
Costa & McCrae, 2007; John et al., 2010; Natividade & Hutz, 2015).
With some exceptions (e.g., Ashton et al., 2004; Benet & Waller, 1995), research-
ers from different cultures agree that the five-factor structure is the best arrangement
of personality descriptors (McCrae et al., 2004). No consensus has been reached,
however, on the nomenclature of these factors or the facets (or subfactors) that
13
692 Trends in Psychology (2023) 31:690–715
underlie each of them. The number of facets may vary according to the instrument
that is chosen to approach personality. The NEO-PI-R, for example, presents a struc-
ture of 30 facets (Costa & McCrae, 2007), and the BFI-2 has 15 facets (Soto & John,
2017). An instrument that was built specifically for the Brazilian context revealed a
structure of 17 facets (Nunes et al., 2010).
Although the number of facets varies, the depressive trait in all these instruments
underlies the neuroticism factor and is grouped in a subfactor called depression
(Costa & McCrae, 2007; Nunes et al., 2010; Soto & John, 2017). The depressive
trait is understood as a tendency to present negative emotions, low expectations for
the future, feelings of loneliness, boredom, the absence of clear goals, and a monoto-
nous and emotionless life (Nunes et al., 2010; Soto & John, 2017). People with high
scores on the depression facet consider themselves unable to deal with difficulties
that arise in their lives, whereas low levels of this facet indicate happier and more
hopeful people with regard to the future because they believe in their own ability to
deal with any problems that may arise throughout their lives (Nunes et al., 2010).
Also, some authors emphasize happiness as a facet of neuroticism that would be
opposite to depressive characteristics, as happier people rarely feel sad, constantly
look to the brighter side of life, and feel comfortable about themselves (Goldberg,
1999; Hofstee et al., 1992).
High scores on the depression facet are associated with several negative life out-
comes, such as lower levels of happiness, a worse quality of life, and shorter longev-
ity (Terracciano et al., 2008). Among all the subfactors of neuroticism, the depres-
sion facet is most strongly related to subjective well-being. Individuals with high
levels of the depressive trait tend to experience more negative than positive affect
while evaluating their own lives as less satisfactory (Quevedo & Abella, 2011).
Additionally, the traits that comprise the neuroticism factor are generally associ-
ated with different psychiatric conditions, such as depression and anxiety disorders
(Lahey, 2009). Thus, neuroticism and its facets can be considered personality traits
that significantly impact both physical and mental health conditions (Lahey, 2009).
Implicit Measurement of Personality
Measuring the depression facet may help identify people who would benefit from
preemptive interventions that minimize the risk of the onset of mental disorders
(Klein et al., 2011). The evaluation of the depressive trait, however, faces the same
difficulty as measuring other constructs that are considered socially undesirable, in
which people are not always willing to reveal negative aspects of themselves (Payne
& Gawronski, 2010). Some people may also be unaware of their own characteristics
or tend to protect self-esteem and be reluctant to acknowledge that they are depres-
sive people (Hepper et al., 2010). Aware of these difficulties, researchers have devel-
oped in recent decades implicit (or indirect) methods to assess disorder-relevant
contents, such as self-esteem and depression, which proved to predict some specific
psychopathological behaviors (Meites et al., 2008; Roefs et al., 2011).
In contrast to self-report measures, implicit measures aim to assess psychological
constructs without directly asking people what one wants to know (De Houwer &
13
Moors, 2010). The implicit association test (IAT) and Go/No-Go task are examples
of such measures (Greenwald et al., 1998; Nosek & Banaji, 2001). Although they
have some particularities, these tests are similar regarding the way they use response
time as a unit of measure for the strength of an association between categories,
avoiding the deliberative processing of information by respondents. This character-
istic makes it less strongly influenced by self-presentation goals, although scores on
the implicit measures are also susceptible to faking (Greenwald et al., 2002; Stef-
fens, 2004).
Among the available implicit measures, the IAT is notable because it is the most
used measure of personality traits (De Cuyper et al., 2017). This test assesses con-
structs whose definitions involve associations between concepts, such as attitudes
(associations between objects and affects), stereotypes (associations between groups
and non-valence attribute concepts), and self-concept (associations between the
self and non-valence attribute concepts; Greenwald et al., 2002). Personality meas-
ures have traditionally been based on the self-concept to measure traits (Kim et al.,
2019). In the impossibility of accessing the traits themselves, one tries to identify
how strongly people believe that certain characteristics adequately describe them
(Kim et al., 2019). These beliefs constitute a small part of complex networks of
associations that form the self-concept and thus are subject to measurement through
the IAT (Greenwald et al., 2002). There is no consensus, however, about which spe-
cific constructs underlie the IAT’s responses (Corneille & Hütter, 2020).
The implicit measurement of personality is still at an early stage compared with
the long history of explicit measures, but some satisfactory results have already been
reported. For example, the factorial structure of the IAT that was built to assess the
five major factors was shown to be adequate and have internal consistency (Grumm
& von Collani, 2007; Schmukle et al., 2008). However, more considerable prob-
lems are found with regard to the temporal stability of the measures and evidence
of validity based on relationships with other variables (De Cuyper et al., 2017; Hof-
mann et al., 2005; Schimmack, 2019). Few studies have produced results about the
temporal stability of implicit measures of personality. These measures have also
shown weak correlations with self-report instruments and little predictive power
regarding behaviors (Costantini et al., 2015; Hofmann et al., 2005; Schmukle et al.,
2008; Meissner et al., 2019).
With regard to specifically implicit measures of depression, studies have built
measures to assess depression as a mood state (Glashouwer & De Jong, 2010;
Meites et al., 2008) or negative self-schema (Dentale et al., 2016), with no intent to
assess it as a stable characteristic (i.e., trait) in the scope of personality. However,
depression as a trait also is a relevant disorder variable (Klein et al., 2011). This gap
in measures of depression trait meets the argument of De Cuyper et al. (2017) that
implicit measures of personality would benefit from a narrower approach to person-
ality by considering not only the Big5 factors but also the facets that underlie them,
such as organization and depression (Costantini et al., 2015). Cuyper et al. argued
that a more specific (or facet) approach can result in the greater predictive power of
implicit measures on behavior, especially if the type of behavior is considered (i.e.,
automatic or deliberate). They also emphasize the importance of further studies to
13
develop empirical procedures to select items that comprise implicit tests to ensure
that the selected words or images accurately represent the target construct.
Present Study and Hypothesis
This study was designed to construct an IAT to assess the depression facet of neu-
roticism and search for evidence of its validity. We also sought to develop an empiri-
cal procedure to select items for implicit measures. Therefore, two studies were per-
formed. The first study consisted of the elaboration of items and execution of an
item selection procedure that considered the accuracy and speed (i.e., response time)
of associations between words and the target category. No specific hypotheses were
tested because this was an exploratory and methodological study.
In the second study, evidence of the validity of the constructed measure was
sought based on the internal structure and relationships with other variables. The
instrument’s reliability was tested by internal consistency and test–retest coeffi-
cients. In this study, we expected the items to be grouped in a one-factor solution in
the EFA, which would be coherent with the definition of the depression facet (Costa
& McCrae, 2007; Nunes et al., 2010; Soto & John, 2017). We also tested the best
arrangement for implicit and explicit depression items together through confirma-
tory factor analysis (CFA) to contribute to the discussion about whether implicit
and explicit measures assess the same construct (Fazio & Olson, 2003; Payne et al.,
2008), distinct but related constructs (Nosek & Smyth, 2007), or constructs that
would be best explained by a common second-order factor. For criterion validity,
we mainly hypothesized that implicit depression would positively correlate with
explicit neuroticism, depression, and negative affect and negatively correlate with
self-esteem, satisfaction with life, positive affect, and positive expectations toward
the future (Dentale et al., 2016; Lahey, 2009; Quevedo & Abella, 2011). We also
expected that the participants who behaved in a depressed or neurotic manner, such
as thinking that life was not worth living or having trouble not crying, would present
higher levels of implicit depression (Lahey, 2009; Nunes et al., 2010).
Study 1
Method
Participants
The participants were 176 university students. An equivalent proportion of men

and women was sought, with students from different courses. The participants were
divided into two groups according to the task they completed: 105 participants com-
pleted the task that elaborated categories that are related to the individual, and 71
participants completed the task for categories that belonged to the depression facet.
No detailed information about the participants was obtained in this study.
13
Procedures
Data Collection
The participants were personally approached on the university campus and invited
to participate in a survey on personality. When the participants arrived in the labo-
ratory, they were informed about their rights and how the research would be per-
formed. They received noise mufflers and were directed to computers that were
separated by partitions so that a quiet data collection environment with few visual
stimuli could be ensured. The survey was answered entirely via computer. Informed
consent was obtained from all individual participants included in the study.
Elaboration of Items
We first elaborated a list of words that represented the individual and depression
facet, in their high and low extremes, according to previous recommendations (e.g.,
Greenwald & Farnham, 2000). We named categories that were related to the indi-
vidual as Me (target category) and Other (opposing category) and the representa-
tive categories of the depressive trait as Depression (target category) and Cheerful-
ness (opposing category). Two researchers then independently used the list that was
developed in the previous step to select words that best represented the four catego-
ries based on word content. A third researcher compiled the two previous selections
and judged the adequacy of the words for the definitions of the constructs. Finally, a
list of 26 words was generated to represent the individual, and 33 words represented
the depression facet.
Empirical Test of Items
Based on the sets of items that were elaborated in the previous step, a Go/No-Go
task (Nosek & Banaji, 2001) was configured to empirically test associations between
the words and categories. The task was performed on a computer. In the upper
right corner of the screen, a category appeared. In the center of the screen, a word
appeared that would or would not be representative of the category. In the task that
was developed for the categories that were related to the individual (Me and Other),
the participants provided their names and surnames before starting the task so that
this information could be used as representative stimuli of the Me category. For
these categories, each word remained on the screen for 0.7 s. For the categories of
the depression facet, each word remained on the screen for 0.9 s. The time lapse dur-
ing which the words remained on the screen was selected according to the time that
is commonly used in Go/No-Go tasks (Nosek & Banaji, 2001). Three judges also
evaluated whether the time lapse that was used was sufficient to read and understand
the content of the words. If the word was representative of the category, then the
participant should press the space bar on the computer keyboard as quickly as pos-
sible before the maximum time was reached. If the word was not representative, then
13
the participant should not press any key. The task was configured to randomly dis-
play 90 trials and repeat each word at least once. All of the participants’ responses
were considered in the data analysis.
Analyses
The average response time of making associations between the words and their
respective categories and accuracy rates was calculated. The criteria for selecting
the items, based on responses in the task, were as follows: eight words in each cat-
egory with the highest accuracy rates and the shortest association time (in cases
of identical values) between the word and category. The data that were generated
to produce the results are open and available for download on the Open Science
Framework (https://osf.io/7nre6/).
Results
A total of 32 words were selected (16 for each pair of categories, eight for each
target category and eight for each opposing category). Table 1 shows the words that
were selected for the Me, Other, Depression, and Cheerfulness categories, together
with their respective accuracy rates and average response times.
Study 2
Method
Participants
A total of 98 participants were included in study 2, 62.2% were women (n = 61),

and 37.8% were men (n = 37), with an average age of 21.9 years (SD = 6.14 years,
Min = 18 years, Max = 61 years), who lived in the city of Rio de Janeiro, Brazil. For
education, 13.3% had completed high school (n = 13), 79.6% had incomplete higher
education (n = 78), and 7.1% ranged from completed higher education to completed
graduate education (n = 7). The sample size was aligned with the recommendation
of a minimum of 10 participants per item to run a factorial analysis (Osborne et al.,
2008).
Instruments
The participants completed the IAT that was constructed for this study to assess the
depressive trait and an online questionnaire that asked sociodemographic questions
(gender, age, and education), presented scales of self-reported personality, self-esteem,
and subjective well-being and asked behavioral criterion questions about typical behav-
iors of depressed and high neurotic people. The behavioral criterion questions asked
13
Table 1 Words selected for the categories Me, Other, Depression, and Cheerfulness
Me and Other Depression and Cheerfulness
Words Corrects (%) M (ms) SD Words Corrects (%) M (ms) SD
Namea 93.9 486.23 71.99 Negativista 98.3 524.21 86.25

b,c
So-and-so 91.9 503.57 74.79 Crestfallena 97.7 551.42 90.18
Ia 91.8 465.33 87.58 Pessimistica 97.3 524.31 82.68
Surnamea 91.7 503.78 70.23 Hopelessa 97.2 556.19 92.88
So-and-sob,c 91.0 505.24 76.56 Depresseda 97.1 519.28 90.47
Mya,c 90.4 473.73 80.69 Depressivea 96.8 521.91 95.66
Mea,d 89.4 472.69 78.16 Sada 96.7 527.02 86.94
Mea,d 89.3 469.73 89.59 Disillusioneda 96.3 525.92 88.40
Theirb 89.2 497.44 80.53 Smilingb 96.0 512.68 81.11
Otherb 87.5 508.21 78.68 Joyfulb 95.8 492.91 92.51
So-and-sob,c 86.7 513.52 75.40 Happyb 95.6 477.67 81.50
Theyb 86.4 496.34 91.03 Positiveb 95.1 505.25 82.02
Theirsb 86.3 508.16 86.85 Radiantb 94.6 556.20 105.40
Anotherb 85.7 518.36 93.81 Jovialb 94.2 537.92 93.57
a
With Me 85.6 509.55 82.18 High Spiritb 94.1 570.88 99.01
Minea 81.7 501.24 84.86 Gladb 94.0 530.34 95.04
a
Word selected for the target category
b
Word selected for the opposing category
c
Words were different in the Portuguese language, but there is no translation for them in English and they
are equivalent in meaning to “so-and-so”
d
Words were different in the Portuguese language, but there is no translation for them in English and they
are equivalent in meaning to “me”
how strongly the participants believed that good things would happen in their future
(0 = I do not believe that good things will happen in my life, 100 = good things will def-
initely happen in my life). They were also asked if they had trouble holding back tears
at some point in the previous week (yes and no), came to think that life was not worth
living at some point in the previous week (yes and no), and if they felt so helpless they
could not leave the house the day before (yes and no). More general criterion questions
were also asked, including how many friends they have (0 to 100 or more), how many
people (friends, family, etc.) they believed they could count on (0 to 100 or more), and
how many times (approximately) they had unprotected sex in the last month (0 to 100
or more). Additionally, control questions were asked to guarantee that the participants
were not answering the scales randomly. These questions asked the participants to mark
a specific response option (e.g., “This is a control question. Please mark number four”).
Implicit Measure of the Depressive Trait
The words that were selected in study 1 were introduced in a computerized version
of the IAT that was built and applied using Inquisit software (Draine, 2001). We
13
chose a seven-block structure for the measurement, with three blocks having the
training function (single categorization sessions) and the other four blocks having
the test function (double categorization sessions). The participants first provided
their names and surnames so that these words appeared as stimuli that were repre-
sentative of the Me category. They were then informed about which words belonged
to each target category (Me and Other) and each attribute category (Depression and
Cheerfulness) and how they should proceed to answer the test. The participants’
task was to associate the words that appeared in the center of the screen with the
correct categories as quickly as possible and make as few mistakes as possible.
When the word referred to a category that was shown in the upper left corner of
the screen (e.g., Me or Depression), the participant should press the “E” key. If the
word belonged to a category that was shown in the upper right corner, then the par-
ticipant should press the “I” key (e.g., Other or Cheerfulness). The words appeared
in random order in both the training blocks and test blocks. In blocks 3 and 6, each
word appeared only once for the participant (one trial). In blocks 4 and 7, each word
appeared twice (two trials). Adjectives with different forms for women and men
(i.e., a common feature of the Portuguese language) were inflected according to the
participant’s gender to avoid bias (Natividade et al., 2012). All of the participants
completed the seven blocks in the same order. The order, categories, and number of
trials of the blocks are presented in Table 2.
Factorial Battery of Personality (FBP; Nunes et al., 2010)
This instrument measures the five personality factors and their 17 facets: extraver-
sion (communication, self-confidence, dynamism, social interactions), agreeableness
(kindness, pro-sociability, trust in people), conscientiousness (competence, cautious-
ness/prudence, endeavor/commitment), neuroticism (vulnerability, emotional instabil-
ity, passivity, depression), and openness to experience (openness to ideas, liberalism,
search for novelties). This is a personality test that was built specifically for Brazil. It
has 126 items in the form of statements, and the participants should estimate the extent
to which they agree with each of them on a 7-point scale. Examples of the items are as
follows: “I’m tired of living” and “I’m an irritable person.” The alpha coefficients that
were reported in the original study ranged from 0.74 to 0.89. In the present study, the
Table 2 Sequences of blocks and trials of the implicit measure of depressive trait
Block Number of Function Categories on the left Categories on the right
trials
1 20 Training Me Other
2 20 Training Depression Cheerfulness
3 32 Test Me and Depression Other and Cheerfulness
4 64 Test Me and Depression Other and Cheerfulness
5 20 Training Other Me
6 32 Test Other and Depression Me and Cheerfulness
7 64 Test Other and Depression Me and Cheerfulness
13
alpha coefficients were 0.92 for extroversion, 0.81 for agreeableness, 0.82 for conscien-
tiousness, 0.91 for neuroticism, and 0.79 for openness.
Reduced Scale of Personality Descriptors (RED5; Natividade & Hutz, 2015)
This measure assesses the five personality factors through 20 items in the form of adjec-
tives or short expressions, such as “anxious,” “calm,” and “open to new experiences.”
Each personality factor consists of four items. Participants must judge how adequately
each adjective or expression describes them on a 7-point scale. The alpha coefficients
that were reported in the original study ranged from 0.59 to 0.84. In the present study,
the alpha coefficients were 0.88 for extroversion, 0.79 for agreeableness, 0.78 for con-
scientiousness, 0.75 for neuroticism, and 0.58 for openness.
Rosenberg’s Self‑Esteem Scale (Hutz & Zanon, 2011; Brazilian version

of Rosenberg’s Original Scale, 1965)
This single-factor scale measures self-esteem through 10 items in the form of state-
ments. Participants are asked to state how much they agree with each of the items on
a 4-point scale. The alpha coefficient that was reported in Hutz & Zanon (2011) was
0.90. In the present study, the alpha coefficient was 0.88.
Life Satisfaction Scale (LSS; Hutz et al., 2014; Brazilian version of the Original
Scale by Diener et al., 1985)
This instrument assesses the cognitive component of subjective well-being through five
items, explained by a single factor. The items are answered on a 7-point scale. Partici-
pants must indicate how strongly they agree with each item. The alpha coefficient was
not reported in Hutz et al. (2014). In the present study, the alpha coefficient was 0.82.
Positive and Negative Affect Schedule (PANAS; Zanon & Hutz, 2014; Brazilian
Version of the Scale by Watson et al., 1988)
This scale assesses the affective dimensions (positive affect and negative affect) of
subjective well-being. Each factor is composed of 10 items that are presented in the
form of adjectives and represent emotions that are commonly experienced by people.
The items are answered on a 5-point scale. Participants must indicate how strongly the
adjectives describe the way they have been feeling lately. The alpha coefficients were
not reported in Zanon & Hutz (2014). In the present study, the alpha coefficient was
0.87 for positive affect and 0.89 for negative affect.
13
Procedures
Data Collection
The data collection procedures were identical to study 1, with the exception that
we invited the participants to return to the laboratory after 30 days for a second
wave of data collection. Of the initial participants, 34 returned to the laboratory
and responded again to the implicit measure of the depressive trait. Informed con-
sent was obtained from all individual participants included in the study.
Analyses
Participants who answered the control questions incorrectly, had an accuracy rate
of less than 70% on the IAT, and emitted more than 10% of responses with a
time lapse of less than 300 ms were excluded from the analyses (Greenwald et al.,
2003; Grumm & von Collani, 2007). Overall, we excluded six participants and
ran the analyses with N = 98. The logarithmic transformation of response times
was then performed according to standard procedures in the literature (Schmukle
et al., 2008). For the calculation of IAT items, words from the Depression and
Cheerfulness categories were transformed into pairs (e.g., negative and positive)
by considering the best matches with regard to opposite content in the Portuguese
language and according to the procedure that was suggested in previous stud-
ies (Grumm & von Collani, 2007; Schmukle et al., 2008). The value of the item
consisted of subtracting the average response time of each pair in the congruent
blocks (6 and 7) from the average response time of that same pair in the incon-
gruous blocks (3 and 4). Thus, for each word pair, the participants had an associa-
tion score, which was used as the item’s value.
To test the structure of the measure, an exploratory factor analysis (EFA) was
performed using the principal axis method. We also tested three models using
CFA with a robust maximum likelihood estimator. In model 1, we specified
only one factor to explain all items from the explicit facet of depression (from
the FBP) and implicit depression. In model 2, we specified two correlated factors
to explain explicit and implicit corresponding items. Model 3 was specified with
one second-order factor to explain both explicit and implicit factors of the depres-
sive personality trait. We used R software (R Development Core Team, 2012) and
the Lavaan statistical package (Rosseel, 2012) for these analyses. The reliability
indices of the measurements were obtained by calculating the alpha, omega, and
Pearson correlation coefficients between the two moments of the test application.
To obtain an average association for each participant, the IAT effect (D score)
was calculated according to the procedure that was suggested by Greenwald et al.
(2003). Finally, Pearson’s correlation coefficients were calculated, and Student’s
t-tests were performed to obtain evidence of validity based on relationships with
other variables. These analyses were conducted using jamovi 1.0 software (The
jamovi project, 2019). The data are available for download at the same link that is
provided in study 1.
13
Results
Structure and Accuracy
The procedure for calculating the items that are described above resulted in a set
of eight pairs of words, which underwent a factor analysis with principal axis and
oblimin rotation methods. Initially, the adequacy of data for factoring was found,
KMO = 0.81; Bartlett’s sphericity test: χ2(28, N = 98) = 127, p < 0.001. A parallel
analysis of random eigenvalues, with 500 bootstrap samples, showed that the last
observed eigenvalue that was greater than the simulated one was for factor I (factor
I: observed eigenvalue = 2.17, simulated eigenvalue = 0.54; factor II: observed eigen-
value = 0.27, simulated eigenvalue = 0.35). This factor explained 27.4% of the data
variance. The scree plot also showed the emergence of a single factor. Considering
these results, the extraction of a single factor was considered adequate. The data
adequacy indices for the single factor model were also satisfactory (Brown, 2006),
χ2(20, N = 98) = 15.5, p = 0.75; χ2/df = 0.77; Tucker–Lewis index (TLI) = 1.07,
root mean square error of approximation (RMSEA) = 0.00 [90% confidence inter-
val = 0.00–0.06); standardized root mean square residual (SRMR) = 0.07. The items,
their respective factor loadings, and communalities are presented in Table 3.
We also tested three models through confirmatory factor analysis (CFA). The
results showed that the model with two correlated factors was the best solution for
the data compared with a single factor model and a second-order factor model. The
correlation between the latent implicit depression trait and latent explicit depression
trait was 0.50. The adjustment indices for all the models are shown in Table 4. With
regard to the reliability indicators for the implicit measure, the alpha and omega
coefficients were 0.75. Additionally, the Pearson correlation coefficient that was
obtained between the two moments of application of the implicit measure (30-day
interval) was 0.62 (p < 0.001).
Table 3 Factor loadings and Items Factor Loadings h2

communalities of the items
from a principal axis factoring Negativist/Positive .61 .37
analysis
Disillusioned/Radiant .59 .35
Pessimist/Jovial .56 .32
Crestfallen/High Spirit .50 .26
Hopeless/Smiling .50 .25
Sad/Joyful .49 .24
Depressive/Glad .47 .23
Depressed/Happy .45 .20
Eigenvalue 2.15
% of explained variance 27.4
Alpha coefficient .75
Omega coefficient .75
Test–retest coefficienta .62
a
Approximately 30 days between test–retest, n = 34. N = 98
13
Table 4 Adjustment coefficients Single-factor Two-factor Second-order factor

of the models tested from
confirmatory factor analyses χ2 180.5 113.2 112.1
with robust maximum likelihood
estimator Df 104 103 102
P < .001 .23 .23
χ2/df 1.73 1.10 1.10
CFI 0.80 0.97 0.97
TLI 0.77 0.97 0.97
GFI 0.77 0.87 0.87
RMSEA 0.087 0.032 0.032
CI 90% RMSEA 0.067–0.111 0.000–0.063 0.000–0.063
AIC 1427.3 1355.6 1357.6
Single-factor means model specified with a single factor explain-

ing all of the depression personality facet items. Two-factor means
model specified with two correlated factors, one explaining implicit
depression items, and the other explicit depression items. Second-
order factor means model specified with one second-order factor
explaining both explicit and implicit factors of depressive personal-
ity facet. N = 98
χ2, chi-square; df, degrees of freedom; χ2/df, chi-square/degrees
of freedom ratio; CFI, robust comparative fit index; TLI, robust
Tucker–Lewis index; GFI, goodness-of-fit index; RMSEA, robust
root mean square error of approximation; CI 90% RMSEA, 90% con-
fidence interval; AIC, robust Akaike information criterion
Relationships with Other Variables
To search for more evidence of validity, Pearson correlations between the con-
structed measure and other variables were calculated. Table 5 shows correlations
between the implicit depressive trait (IAT), the five personality factors that were
assessed with an extensive contextualized-statements test (FBP) and a reduced-
adjective measure (RED5), the dimensions of subjective well-being and self-esteem,
and the behavioral criterion questions. Implicit depression positively correlated with
the Neuroticism factor (RED5) and negatively correlated with satisfaction with life,
self-esteem, and positive expectations for the future.
Table 6 shows correlations between the constructed measure and facets of the
FBP. Negative correlations were found between implicit depression and communi-
cation (FBP), self-confidence (FBP), dynamism (FBP), and trust in people (FBP)
facets, and a positive correlation was found between implicit depression and depres-
sion facet (FBP).
Finally, differences between groups in levels of the implicit and explicit depres-
sive trait were tested. No significant difference was found between men (n = 37)
and women (n = 61) in levels of implicit depression, t(96) = − 0.55, p = 0.58,
d = 0.10, Hedge’s g = 0.10). This result was also found for levels of explicit depres-
sion, t(96) = 0.84, p = 0.40, d = 0.16, Hedge’s g = 0.16. Participants who declared
that they had not been able to stop crying at some point in the previous week had
higher levels of implicit depression (n = 49, M = − 0.42, SD = 0.37) compared with
13
Table 5 Pearson correlation coefficients between implicit depression (IAT), personality (FBP and RED5), subjective well-being, self-esteem, and criterion questions
M SD 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
1. Implicit − 0.50 0.38 (.75)

depres-
sion
(IAT)
2. Neu- 3.67 1.04 .19 (.91)
roticism
(FBP)
3. Extro- 4.04 0.98 − .25* − .22* (.92)
version
(FBP)
4. Agreea- 5.34 0.56 − .05 − .27** .06 (.81)
bleness
(FBP)
5. Consci- 4.76 0.66 .02 − .36** .11 .13 (.82)
entious-
ness
(FBP)
6. Open- 5.12 0.71 − .001 .003 .48** .01 .01 (.79)
ness
(FBP)
7. Extro- 4.47 1.46 − .18 − .20* .82** .09 .02 .39** (.88)
version
(RED5)
8. Agreea- 5.85 0.86 − .07 − .27** .48** .34** .21* .31** .49** (.79)
bleness
(RED5)
9. Neu- 3.82 1.15 .26** .62** .04 − .22** − .21* − .07 .02 − .20* (.75)
roticism
(RED5)
10. 4.92 1.18 − .03 − .38** − .09 .26** .51** − .13 − .03 .16 − .31** (.74)
Consci-
entious-
703
13
ness
(RED5)
Table 5 (continued)
704
M SD 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
11. Open- 5.11 0.90 − .04 − .06 .46** .08 − .11 .63** .44** .22* − .05 − .05 (.58)
13
ness
(RED5)
12. Life 4.48 1.50 − .28** − .49** .43** .23* .37** − .004 .24* .22* − .24* .23* .06 (.82)
satisfac-
tion
13. 3.29 0.82 − .19 − .50** .56** .12 .41** .23* .39** .30** − .27** .18 .24* .56** (.87)
Positive
affect
14. Nega- 2.30 0.79 .19 .66** .06 − .19 − .18 .03 .01 − .09 .64** − .37** − .02 − .38** − .20* (.89)
tive
affect
15. Self- 2.92 0.64 − .34** − .77** .38** .20* .40** .13 .25* .28** − .57** .22* .13 .63** .65** − .57** (.88)
esteem
16. CQ1 74.6 25.3 − .34** − .54** .38** .27** .30** .12 .26** .17 − .36** .15 .17 .57** .68** − .45** .71** -
17. CQ2 9.05 7.55 − .10 − .16 .12 .02 .03 .17 .10 .31* − .20 .21* .01 − .12 − .02 − .12 .06 .04 -
18. CQ3 9.59 12.2 − .04 − .19 .02 .07 .13 − .08 − .07 .13 − .21* .15 − .02 .22* .10 − .17 .17 .28* .24* -
19. CQ4 3.23 14.2 .15 − .09 .01 .21* .01 − .11 − .07 .10 .12 .01 − .02 .07 .11 .03 .002 − .05 .11 − .01
CQ1 = Regarding your future, how much do you believe that good things will happen? CQ2 = How many close friends do you have? CQ3 = Regarding the people who are
part of your life (friends, family, etc.), how many of them do you believe you can really count on? CQ4 = In the last month, how many times, approximately, have you had
unprotected sex? Alpha coefficients of the scales are in the main diagonal, within parentheses. N = 98
*
p < .05
**
p < .01
Table 6 Pearson correlation coefficients between implicit depression (IAT) and FBP facets
M SD 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
1. Implicit − 0.50 0.38 (.75)

depres-
sion
(IAT)
2. N1 – 4.06 1.23 .13 (.82)
vulner-
ability
3. N2 – 3.72 1.47 .03 .50** (.84)
emotional
instabil-
ity
4. N3 – pas- 4.26 1.30 .07 .60** .50** (.79)
sivity
5. N4 – 2.75 1.23 .33** .60** .40** .49** (.79)
depres-
sion
6. E1 – 3.94 1.31 − .21* − .45** − .02 − .24* − .31** (.84)
commu-
nication
7. E2 – self- 3.38 1.14 − .23* − .13 .28** .24* − .06 .58** (.75)
confi-
dence
8. E3 4.42 1.14 − .22* − .48** − .15 − .33** − .57** .64** .41** (.70)
– dyna-
mism
9. E4 – 4.52 1.22 − .17 − .12 .20* .06 − .26** .56** .57** .49** (.82)
social
interac-
tions
10. A1 – 5.68 0.67 .09 .10 .19 − .02 − .14 .14 .08 .23* .41** (.80)
kindness
705
13
Table 6 (continued)
706
M SD 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
11. A2 – 5.46 0.92 .02 .03 − .13 − .22* − .19 − .22* − .33** − .07 − .25* .16 (.72)
13
pro-socia-
bility
12. A3 – 4.71 0.98 − .22* − .32** − .36** − .26* − .46** .14 − .04 .15 .14 .15 .24* (.76)
trust in
people
13. C1 4.68 0.84 − .07 − .42** − .24** − .46** − .51** .28** .17 .52** .17 .06 − .001 .11 (.82)
– compe-
tence
14. C2 – 5.10 1.17 .16 .13 − .27** − .01 .12 − .26* − .32** − .14 − .34** .09 .17 − .01 − .23* (.62)
cautious-
ness/
prudence
15. C3 – 4.69 1.06 .01 .03 − .01 − .23* − .16 .04 − .02 .21* − .03 .12 .18 − .13 .43** .11 (.71)
endeavor/
commit-
ment
16. O1 – 5.25 0.98 − .08 − .20* − .05 − .05 − .15 .35** .30** .32** .38** .18 − .26** .13 .09 − .09 − .001 (.75)
openness
to ideas
17. O2 5.53 0.87 .08 .14 .21* .11 .09 .14 .17 .06 .29** .36** − .27** − .04 − .01 .001 .06 .36** (.59)
– liberal-
ism
18. O3 – 4.43 1.17 .03 − .09 .11 .20* .06 .26 .28** .21* .37** .22* − .30** − .05 − .01 .03 − .09 .19 .22* (.65)
search for
novelties
Alpha coefficients of the scales are in main diagonal, within parentheses. N = 98

N, neuroticism Factor; E, extroversion factor; A, agreeableness factor; C, conscientiousness factor; O, openness factor
*
p < .05
**
p < .01
those who claimed not to have had this experience (n = 49, M = − 0.58, SD = 0.37),
t(96) = 2.13, p = 0.03, d = 0.43, Hedge’s g = 0.43. However, no difference in levels
of explicit depression was found between these two groups, t(96) = 1.05, p = 0.30,
d = 0.21, Hedge’s g = 0.21.
Moreover, participants who claimed they felt so helpless they could not leave
the house the day before had higher levels of explicit depression (n = 26, M = 3.53,
SD = 1.27) compared with those who did not say so (n = 72, M = 2.46, SD = 1.09),
t(96) = 4.10, p < 0.001, d = 0.90, Hedge’s g = 0.94. Participants who said they
thought life was not worth living (n = 25, M = 3.95, SD = 1.12) at some point in the
previous week had higher levels of explicit depression compared with those who did
not have this thought (n = 73, M = 2.33, SD = 0.97), t(96) = 6.90, p < 0.001, d = 1.54,
Hedge’s g = 1.60. However, there were no differences between these groups in lev-
els of implicit depression, with regard to both feeling helpless to leave the house,
t(96) = 0.99, p = 0.32, d = 0.23, Hedge’s g = 0.24, and having thought that life was not
worth living, t(96) = 1.99, p = 0.05, d = 0.42, Hedge’s g = 0.45. For this last result, it
is worth noting that we found a moderate effect size (d = 0.42; Hedge’s g = 0.45)
with a 0.05 p-value. Thus, although it did not achieve statistical significance, report-
ing differences between groups might allow further attention to this behavioral ques-
tion. As expected, people who endorsed having though that life was not worth living
had higher level of implicit depression (n = 25, M = − 0.38, SD = 0.46) compared to
those who declared not having this thought (n = 73, M = − 0.55, SD = 0.34).
General Discussion
The main goal of the present study was to build an implicit measure to assess the
depressive trait and search for evidence of its validity. The starting point was the
definition of the construct, and a list of words that represented four categories was
generated: Me, Other, Depression, and Cheerfulness. The adequacy of the items for
these categories was judged by three researchers who were experts on the construct
and using a computerized task. This procedure sought to ensure that the chosen
words, in addition to adequately representing the construct, were easily identifiable
as belonging to their respective category by the target population of the test. This
task was developed based on previous studies that highlighted the need for more
robust procedures to select the items that comprise IATs (De Cuyper et al., 2017).
We believe that this was an innovative proposal in implicit measurement because
previous studies selected items with no empirical procedures (e.g., Glashouwer &
De Jong, 2010) or using self-report methods (e.g., Costantini et al., 2015).
The advantage of this procedure for selecting items compared with strategies
that were used in previous studies is that it minimizes a possible source of error
variance in scores that are obtained through implicit instruments. Like any psy-
chological measure, the score that is generated by the IAT not only is related
to the respondents’ latent trait level but also carries associated errors (Gawron-
ski, 2019). Some errors are random and cannot be predicted or controlled; others,
however, can be identified and corrected. The item selection procedure that was
developed in the present study provides an additional guarantee that respondents
13
strongly associate the chosen words with their respective categories. In practical
terms, this can prevent the participant, when answering the test, from getting con-
fused about the appropriate category with which to associate a word. An implicit
measure whose items are not strongly associated with categories by the target
population of the test may require additional effort by participants to associate
the words with the correct categories. This effort, therefore, may cause part of
the observed score not to be related to the individual’s latent trait level but rather
to be related to the time the participant takes to remember that an item belongs
to a certain category. In this sense, although eliminating errors that are associ-
ated with psychological measures is not possible, selecting items that are strongly
associated with categories can help increase the accuracy of implicit measures.
Future studies, however, may improve this procedure by controlling possible con-
founding effects of stimulus selection, such as word length or familiarity.
In a second step, an IAT was created with categories and words that were pre-
viously selected. The instrument then underwent empirical testing so that more
evidence of validity could be sought, and reliability indicators were obtained. The
EFA revealed a single factor structure for the measurement, which was congruent
with the definition of the construct, which conceives it as one of the subfactors
of neuroticism (Costa & McCrae, 2007; Nunes et al., 2010; Soto & John, 2017).
This result indicates that the measure that was built presented evidence of validity
based on the internal structure. The CFA also revealed that implicit and explicit
depression items were explained by distinct but correlated factors, reinforcing the
idea that implicit and explicit measures might assess distinct but correlated con-
structs (Nosek & Smyth, 2007), although structural misfit between measures can
also explain the size of the correlation (0.50) and the emergence of independent
latent factors (see Payne et al., 2008).
Additionally, the instrument’s accuracy was tested using alpha, omega, and
test–retest coefficients. The alpha and omega coefficients were higher than 0.70,
indicating satisfactory internal consistency (Nunnally, 1978). The test–retest
correlation coefficient, however, was 0.62, which was slightly below the recom-
mended value. Even if this value was found to be higher than those that were
obtained in other studies that built implicit personality tests (De Cuyper et al.,
2017), a value that is considered adequate, according to psychometric manuals,
would be greater than 0.70 (Nunnally, 1978; Pasquali, 2011).
Previous studies found the same pattern of results with regard to the reliability
of implicit measures (i.e., levels of internal consistency that were comparable to
explicit measures) but lower temporal stability (De Cuyper et al., 2017). Some
critics relied on these results to argue that the scores that are obtained by implicit
measures would be uninformative because they might indicate random values
(e.g., Mitchell, 2018). Arguably, however, this would be a misinterpretation
when considering the dual model of information processing (Gawronski, 2019).
According to this model, implicit associations tend to be less (and not equally)
stable over time compared with explicit associations because they vary depend-
ing on how much content is accessible in memory (Gawronski & Bodenhausen,
2011). Thus, low or moderate temporal stability would not pose a threat to the
13
accuracy of implicit instruments but would reveal that, given the way constructs
are designed, lower test–retest correlations are expected (Gawronski, 2019).
In the specific case of personality, however, low temporal stability may represent
a limitation of the use of implicit measures. This is because implicit associations
are only a means of accessing the target construct and reflect different patterns of
the way people tend to think, feel, and behave (traits). Although implicit associa-
tions have more variations over time than explicit associations, personality traits are
expected to remain stable (Pervin, 1994). Thus, low or moderate temporal stability
may indicate that implicit personality measures are highly susceptible to the influ-
ence of contextual variables, which is an undesirable characteristic in instruments
that evaluate traits. Future studies may explore ways to increase the temporal stabil-
ity of the measure that is constructed (e.g., by testing the influence of contextual
cues on the test–retest correlation coefficient; Gschwendner et al., 2008).
Relationships between the constructed measure and other variables were also
tested to search for more evidence of validity. As expected, levels of the implicit
depressive trait positively correlated with the depression facet (FBP) and neuroti-
cism factor (RED5). Once again and in the expected direction, negative correlations
were found between implicit depression and life satisfaction and between implicit
depression and self-esteem. These relationships are important evidence of the valid-
ity of the constructed measure because, according to previous studies, depressive
people tend to evaluate themselves negatively with regard to both their own self
and to the living conditions they believe they have (Klein et al., 2011; Quevedo &
Abella, 2011). One caveat must be made, however, about the relationships that were
found between the constructed measure and self-esteem. Although the strength of
the correlation was only moderate, it proved to be superior compared with corre-
lations that were obtained between the implicit measure and explicit measures of
personality. Other studies should further investigate whether an affective load is
assigned to items of the measure that is constructed in this research (Grumm & von
Collani, 2007).
Additionally, a negative correlation was found between implicit depression and
how strongly people believed that good things would happen in their future lives.
This result was expected when considering that high hopelessness about the future
is one of the main characteristics of people with high levels of the depressive trait
according to the definition of the construct (Nunes et al., 2010). In contrast, some
expected relationships were not found. For example, no significant correlation was
found between the constructed measure and FBP neuroticism factor or affective
dimensions of subjective well-being. There are several reasons for weak or non-
significant correlations between implicit and explicit tests, which do not necessar-
ily concern the validity of the tests (Hofmann et al., 2005). Some of these reasons
imply that these measures would evaluate different mental processes, whereas others
emphasize methodological issues (Fazio & Olson, 2003; Gawronski, 2019).
In the present study, the lack of conceptual convergence between personality
measures may have been the main reason for weak or, in the case of the neuroti-
cism (FBP) factor, nonsignificant correlations. The measure that was constructed
in the present study assesses personality through adjectives that represent catego-
ries, whereas the FBP uses affirmatives. This may represent a possible source of
13
conceptual divergence between the two measures. Additionally, the IAT structure
itself may have contributed to dissonant results between implicit and explicit meas-
ures. Using the IAT that was built in the present study, it is not possible to know
how strongly respondents associate the depressive trait with themselves individually;
instead, it only allows comparisons with a nonspecific “other” (Karpinski, 2004).
Self-report instruments, in contrast, seek to assess how strongly someone associ-
ates a certain characteristic with himself, regardless of how firmly he believes that
another person has that characteristic. Future studies should explore other types of
implicit (e.g., single-category IAT; Karpinski & Steinman, 2006) and explicit (e.g.,
semantic differential) measures of the depressive trait by expanding conceptual con-
vergence between the two forms of measurement.
Finally, differences between groups in levels of explicit and implicit depression
were tested. Compared with the implicit measure, the explicit measure of the depres-
sive trait generally proved to more efficiently discriminate groups in the expected
direction. The opposite was observed only for one of the behavioral criterion ques-
tions, more specifically with regard to the participant not having been able to stop
crying at some point in the previous week. Participants who avowed this weakness
had higher levels of implicit but not explicit depression. This behavioral criterion
question was elaborated to assess an automatic behavior because it refers to non-ver-
bal and uncontrollable behavior. Consistent with results from previous studies, this
result reinforces the idea that implicit measures would more efficiently predict auto-
matic behaviors (e.g., Back et al., 2009; Steffens & Schulze-König, 2006). Future
studies should replicate this finding and elaborate more complex research designs to
differentiate automatic and deliberate behaviors, preferably using behavioral obser-
vation. Also, we recommend further attention to the behavioral question related
to having thought that life was not worth living. Despite the difference in implicit
depression did not achieve statistical significance (p-value was not inferior to 0.05,
but equal to this value), people who endorsed this item had higher implicit depres-
sion than people who did not with a moderate effect size (d = 0.42). Future studies
may clarify the importance of this behavior in the context of implicit depression by
testing the difference between groups in a larger sample.
The present results substantiate findings in the literature on implicit measures that
less efficiently predict behaviors, even those that are obtained through self-report
methods, that are considered typical of the target construct (Costantini et al., 2015;
Schmukle et al., 2008). Notably, however, this issue is part of a broader debate, and
these results do not necessarily indicate problems of validity of the tests but may
indicate methodological flaws (Gawronski, 2019). To organize knowledge that has
been accumulated about implicit measurement to date, Gawronski (2019) showed
that studies have failed to consider moderators of relationships between implicit
associations and behaviors. According to this author, there is no consistent theo-
retical support for expecting unconditional relationships between implicit tests and
behavioral outcomes. Limitations of this research include the absence of moderation
analyses and the retrospective measurement of behaviors, which is subject to distor-
tions that result from memory failures.
Another possible explanation for relationships that were not found con-
cerns the sample size. Current guidelines in psychology have emphasized the
13
importance of adopting good research practices to avoid false-positive and false-

negative results, including the selection of sufficiently large samples. A previous
sample calculation for all analyses (not only for EFA) and the inclusion of more
participants could have increased the statistical power of this study while mini-
mizing possible inferential errors. In addition to the sample size, the characteris-
tics of the participants were not very representative of the Brazilian population.
The sample was mostly composed of undergraduate students from only one uni-
versity. Other studies should endeavor to find evidence of validity in larger and
more diverse samples.
Compared with previous studies, the measure that was built in this study has
several strengths. We sought to assess depression that is conceived as a trait
rather than a mood state (Glashouwer & De Jong, 2010; Meites et al., 2008). The
depressive trait is well known to be a relevant predictor of mental disorders (Klein
et al., 2011). Thus, this measure can be used not only in test batteries to evaluate
clinical conditions of depression but also in a preventive manner by identifying
people whose personality amplifies the risks of mental disorders. The measure
that is presented herein also has the advantage of assessing personality more nar-
rowly, which can increase its predictive power toward behavior (De Cuyper et al.,
2017), compared with single IATs that are designed to assess the major factors
of personality (e.g., Grumm & von Collani, 2007; Schmukle et al., 2008). We
also tested the measure’s structure through factor analysis and obtained a tempo-
ral stability coefficient, an analysis that has rarely been performed previously in
studies of implicit depression (e.g., Dentale et al., 2016; Glashouwer & De Jong,
2010; Meites et al., 2008).
The measure that was constructed in the present study provided satisfactory evi-
dence of validity and reliability. We found a structure that aligned with the defi-
nition of the construct, had some expected relationships with other variables, and
had adequate reliability indices. Although the test–retest correlation coefficient was
slightly below an ideal value, the obtained value was higher than the values that
were reported in previous studies (De Cuyper et al., 2017). This result may have
been mainly a consequence of the empirical procedure that was designed to select
items, which sought to reduce measurement errors that are caused by weak asso-
ciations between items and categories that comprise the implicit tests. Future stud-
ies should expand the evidence of validity that was found in the present study by
testing moderating roles of other variables in relationships between the constructed
measure and behaviors (Garownski, 2019). The accumulation of more evidence of
validity may indicate that the constructed measure can be a reliable tool in both
research and applied contexts. In clinical and organizational contexts, for example,
the possibility of measuring the depressive trait allows predictions of the risk of the
onset of mental disorders and professional performance (Klein et al., 2011; Slaugh-
ter & Kausel, 2009). The use of an implicit measure can capture more spontaneous
responses and be less prone to distortions, even when people are highly motivated to
build a favorable image of themselves. For this purpose, however, more evidence of
validity and reliability is needed, especially regarding the power of the measure to
predict life outcomes and produce consistent results over time.
13
Author Contribution Nathalia Melo de Carvalho: conceptualization, methodology, software configura-

tion, data collection, data analysis, writing – original draft. Rafael Valdece Souza Bastos: software con-
figuration, data analysis, data collection. Jean Carlos Natividade: conceptualization, methodology, soft-
ware configuration, data collection, data analysis, writing – review and editing, supervision, funding
acquisition.
Funding This work was supported by the Conselho Nacional de Desenvolvimento Científico e Tec-
nológico (CNPq); the Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro
(FAPERJ); and the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) –
Finance Code 001.
Data Availability The study was not preregistered. The data of this research are available at the Open Sci-
ence Framework (https://osf.io/7nre6/).
Declarations
Ethics Approval and Consent to Participate The data collection for this research was approved by the
Human Research Ethics Committee of the Federal University of Rio Grande do Sul (UFRGS) under pro-
tocol number 712724. All procedures met the recommendations of Resolution 466/2012 of the National
Health Council (Brazil), which deals with the ethical aspects of research with human beings.
Conflict of Interest The authors declare no competing interests.
References
Ashton, M. C., Lee, K., Perugini, M., Szarota, P., De Vries, R. E., Di Blass, L., et al. (2004). A six-factor
structure of personality descriptive adjectives: Solutions from psycholexical studies in seven lan-
guages. Journal of Personality and Social Psychology, 86, 356–366. https://doi.org/10.1037/0022-
3514.86.2.356
Back, M. D., Schmukle, S. C., & Egloff, B. (2009). Predicting actual behavior from the explicit and
implicit self-concept of personality. Journal of Personality and Social Psychology, 97, 533–548.
https://doi.org/10.1037/a0016229
Barenbaum, N. B., & Winter, D. G. (2010). History of modern personality theory and research. In O. P.
John, R. W. Robins, & L. A. Pervin (Eds.), Handbook of personality: Theory and research (3rd ed.,
pp. 114–158). Guilford Press.
Benet, V., & Waller, N. G. (1995). The Big Seven factor model of personality description: Evidence for
its cross-cultural generality in a Spanish sample. Journal of Personality and Social Psychology, 69,
701–718. https://doi.org/10.1037/0022-3514.69.4.701
Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York, NY: Guilford.
Costa, P. T., & McCrae, R. R. (2007). NEO PI-R: Inventário de personalidade NEO revisado e inventário
de cinco fatores NEO revisado NEO-FFI-R [Versão curta] [NEO PI-R: Revised NEO personal-
ity inventory and NEO revised five factor inventory NEO-FFI-R (Short version)]. São Paulo: Vetor
Editora Psico-Pedagócica.
Costantini, G., Richetin, J., Borsboom, D., Fried, E. I., Rhemtulla, M., & Perugini, M. (2015). Develop-
ment of indirect measures of conscientiousness: Combining a facets approach and network analysis.
European Journal of Personality, 29, 548–567. https://doi.org/10.1002/per.2014*
Corneille, O., & Hütter, M. (2020). Implicit? What do you mean? A comprehensive review of the delu-
sive implicitness construct in attitude research. Personality and Social Psychology Review, 24, 212–
232. https://doi.org/10.1177/1088868320911325
De Cuyper, K., De Houwer, J., Vansteelandt, K., Perugini, M., Pieters, G., Claes, L., & Hermans, D.
(2017). Using indirect measurement tasks to assess the self-concept of personality: A systematic
review and meta-analyses. European Journal of Personality, 31(1), 8–41. https://doi.org/10.1002/
per.2092
13
De Houwer, J., & Moors, A. (2010). Implicit measures: Similarities and differences. In B. Gawronski &
B. K. Payne (Eds.), Handbook of implicit social cognition: Measurement, theory, and applications
(pp. 176–193). Guilford Press.
Dentale, F., Grano, C., Muzi, M., Pompili, M., Erbuto, D., & Violani, C. (2016). Measuring the auto-
matic negative self-schema: New evidence for the construct and criterion validity of the Depression
Implicit Association. Self and Identity, 15(5), 599–613. https://doi.org/10.1080/15298868.2016.
1179218
Diener, E., Emmons, R., Larsen, R., & Griffin, S. (1985). The satisfaction with life scale. Journal of Per-
sonality Assessment, 49, 91–95. https://doi.org/10.1207/s15327752jpa4901_13
Draine, S. C. (2001). Inquisit: Version 1.32. Seattle, WA: Millisecond Software.
Fazio, R. H., & Olson, M. A. (2003). Implicit measures in social cognition research: Their meaning and
uses. Annual Review of Psychology, 54, 297–327. https://doi.org/10.1146/annurev.psych.54.101601.
145225
Gawronski, B. (2019). Six lessons for a cogent science of implicit bias and its criticism. Perspectives on
Psychological Science, 14(4), 574–595. https://doi.org/10.1177/1745691619826015
Gawronski, B., & Bodenhausen, G. V. (2011). The associative-propositional evaluation model: Theory,
evidence, and open questions. In J. M. Olson & M. P. Zanna (Eds.), Advances in experimental social
psychology (Vol. 44, pp. 59–127). Academic Press.
Glashouwer, K. A., & De Jong, P. J. (2010). Disorder-specific automatic self-associations in depression
and anxiety: Results of The Netherlands study of depression and anxiety. Psychological Medicine,
40, 1101–1111. https://doi.org/10.1017/S0033291709991371
Goldberg, L. R. (1992). The development of markers for the Big-Five factor structure. Psychological
Assessment, 4(1), 26–42. https://doi.org/10.1037/1040-3590.4.1.26
Goldberg, L. R. (1999). A broad-bandwidth, public-domain, personality inventory measuring the lower-
level facets of several five-factor models. In I. Mervielde, I. Deary, F. De Fruyt, & F. Ostendorf
(Eds.), Personality Psychology in Europe (Vol. 7, pp. 7–28). Tilburg University Press.
Greenwald, A. G., & Farnham, S. D. (2000). Using the Implicit Association Test to measure self-esteem
and self-concept. Journal of Personality and Social Psychology, 79, 1022–1038. https://doi.org/10.
1037/KK122-3514.79.6.10?2
Greenwald, A. G., Banaji, M. R., Rudman, L. A., Farnham, S. D., Nosek, B. A., & Mellot, D. S. (2002).
A unified theory of implicit attitudes, stereotypes, self-esteem, and self-concept. Psychological
Review, 109, 3–25. https://doi.org/10.1037//0033-295X.109.1.3
Greenwald, A. G., McGhee, D. E., & Schwartz, J. (1998). Measuring individual differences in implicit
cognition: The Implicit Association Test. Journal of Personality & Social Psychology., 74, 1464–
1480. 0022-3514/98/$3.00.
Greenwald, A. G., Nosek, B., & Banaji, M. R. (2003). Understanding and using the Implicit Association
Test: I. An improved scoring algorithm. Journal of Personality & Social Psychology, 85, 197–216.
https://doi.org/10.1037/0022-3514.85.2.197
Gregory, R. J. (2014). The history of psychological testing. In R. J. Gregory (Ed.), Psychological testing:
History, principles, and applications (7th ed., pp. 1–28). Upper Saddle River, NJ: Pearson.
Grumm, M., & von Collani, G. (2007). Measuring Big-Five personality dimensions with the implicit
association test: Implicit personality traits or self-esteem? Personality and Individual Differences,
43, 2205–2217. https://doi.org/10.1016/j.paid.2007.06.032
Gschwendner, T., Hofmann, W., & Schmitt, M. (2008). Differential stability. The effects of acute and
chronic construct accessibility on the temporal stability of the Implicit Association Test. Journal of
Individual Differences, 29, 70–79. https://doi.org/10.1027/1614-0001.29.2.70
Hepper, E. G., Gramzow, R., & Sedikides, C. (2010). Individual differences in selfenhancement and self-
protection strategies: An integrative analysis. Journal of Personality, 78, 781–814. https://doi.org/
10.1111/j.1467-6494.2010.00633.x
Hofmann, W., Gawronski, B., Gschwendner, T., Le, H., & Schmitt, M. (2005). A meta-analysis on the
correlation between the Implicit Association Test and explicit self-report measures. Personality and
Social Psychology Bulletin, 31, 1369–1385. https://doi.org/10.1177/0146167205275613
Horn, J. L. (1965). A rationale and test for estimating the number of factors in factor analysis. Psycho-
metrika, 30(1), 179–185. https://doi.org/10.1007/BF02289447
Hofstee, W. K., De Raad, B., & Goldberg, L. R. (1992). Integration of the Big Five and circumplex
approaches to trait structure. Journal of Personality and Social Psychology, 63, 146–163. https://
doi.org/10.1037/0022-3514.63.1.146
13
Hutz, C. S., & Zanon, C. (2011). Revisão da adaptação, validação e normatização da escala de autoestima
de Rosenberg [Revision of adaptation, validity and normatization of Rosenberg self-esteem scale].
Avaliação Psicológica, 10(1), 41–49.
Hutz, C. S., Nunes, C. H., Silveira, A. D., Serra, J., Anton, M., & Wieczorek, L. S. (1998). O desen-
volvimento de marcadores para a avaliação da personalidade no modelo dos cinco grandes fatores
[Development of markers to evaluate personality in the big five factor model]. Psicologia: Reflexão
e Crítica, 11(2), 395–411. https://doi.org/10.1590/S0102-79721998000200015
Hutz, C. S., Zanon, C., & Bardagi, M. P. (2014). Satisfação de vida [Life satisfaction]. In C. S. Hutz
(Org.), Avaliação em Psicologia Positiva (pp. 43–47). Porto Alegre, RS: Artmed.
IBM (2015). IBM SPSS Statistics for Windows, Version 23.0. Armonk, NY: IBM Corp.
John, O. P., Angleitner, A., & Ostendorf, F. (1988). The lexical approach to personality – A historical
review of trait taxonomic research. European Journal of Personality, 2(3), 171–203. https://doi.org/
10.1002/per.2410020302
John, O. P., Naumann, L. P., & Soto, C. J. (2010). Paradigm shift to the integrative Big Five trait tax-
onomy: History, measurement, and conceptual issues. In O. P. John, R. W. Robins, & L. A. Pervin
(Eds.), Handbook of personality: Theory and research (3rd ed., pp. 114–158). Guilford Press.
Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and Psycho-
logical Measurement, 20(1), 141–151. https://doi.org/10.1177/001316446002000116
Karpinski, A. (2004). Measuring self-esteem using the Implicit Association Test: The role of the other.
Personality and Social Psychology Bulletin, 30, 22–34. https://doi.org/10.1177/0146167203258835
Karpinski, A., & Steinman, R. B. (2006). The single category Implicit Association Test as a measure of
implicit social cognition. Journal of Personality and Social Psychology, 91, 16–32. https://doi.org/
10.1037/0022-3514.91.1.16
Kim, H., Di Domenico, S. I., & Connelly, B. S. (2019). Self-other agreement in personality reports: A
meta-analytic comparison of self- and informant-report means. Psychological Science, 30(1), 129–
138. https://doi.org/10.1177/0956797618810000
Klein, D. N., Kotov, R., & Bufferd, S. J. (2011). Personality and depression: Explanatory models and
review of the evidence. Annual Review of Clinical Psychology, 7, 269–295. https://doi.org/10.1146/
annurev-clinpsy-032210-104540
Lahey, B. B. (2009). Public health significance of neuroticism. American Psychologist, 64, 241–256.
https://doi.org/10.1037/a0015309
McCrae, R. R., Costa, P. T., Jr., Martin, T. A., Oryol, V. E., Rukavishnikov, A. A., Senin, I. G., et al.
(2004). Consensual validation of personality traits across cultures. Journal of Research in Personal-
ity, 38, 179–201. https://doi.org/10.1016/S0092-6566(03)00056-4
Meissner, F., Grigutsch, L. A., Koranyi, N., Müller, F., & Rothermund, K. (2019). Predicting behavior
with implicit measures: Disillusioning findings, reasonable explanations, and sophisticated solu-
tions. Frontiers in Psychology, 10https://doi.org/10.3389/fpsyg.2019.02483
Meites, T. M., Deveney, C. M., Steele, K. T., Holmes, A. J., & Pizzagalli, D. A. (2008). Implicit depres-
sion and hopelessness in remitted depressed individuals. Behaviour Research and Therapy, 46,
1078–1084. https://doi.org/10.1016/j.brat.2008.05.008
Mitchell, G. (2018). An implicit bias primer. Virginia Journal of Social Policy & the Law, 25, 27–59.
Natividade, J. C., & Hutz, C. S. (2015). Escala reduzida de descritores dos cinco grandes fatores de per-
sonalidade: prós e contras [Reduced scale of descriptors of big five personality factors: Pros and
cons]. Psico (PUC-RS), 46(1), 79–89. https://doi.org/10.15448/1980-8623.2015.1.16901
Natividade, J. C., Barros, M. C., & Hutz, C. S. (2012). Influência da flexão de gênero dos adjetivos em
instrumentos psicológicos em português [Influence of gender flexion of adjectives on psychological
instruments in portuguese]. Avaliação Psicológica, 11(2), 259–264.
Nosek, B. A., & Banaji, M. R. (2001). The Go/No-Go association task. Social Cognition, 19(6), 625–666.
https://doi.org/10.1521/soco.19.6.625.20886
Nosek, B. A., & Smyth, F. L. (2007). A multitrait–multimethod validation of the Implicit Association
Test: Implicit and explicit attitudes are related but distinct constructs. Experimental Psychology, 54,
14–29. https://doi.org/10.1027/1618-3169.54.1.14
Nunes, C. H. S. S., Hutz, C. S., & Nunes, M. F. O. (2010). Bateria Fatorial de Personalidade (BFP):
manual técnico [Factorial battery of personality: Technical manual]. São Paulo: Casa do Psicólogo.
Nunnally, J. C. (1978). Psychometric Theory (2nd ed.). McGraw Hill.
Osborne, J. W., Costello, A. B., & Kellow, J. T. (2008). Best practices in exploratory factor analysis In J.
Osborne (Ed) Best practices in quantitative methods. Los Angeles, Sage Publications.
13
Pasquali, L. (2011). Psicometria: teoria dos testes na psicologia e na educação [Psychometry: Theory of
educational and psychological tests]. Petrópolis: Vozes.
Payne, B. K., Burkley, M. A., & Stokes, M. B. (2008). Why do implicit and explicit attitude tests diverge?
The role of structural fit. Journal of Personality and Social Psychology, 94, 16–31. https://doi.org/
10.1037/0022-3514.94.1.16
Payne, B. K., & Gawronski, B. (2010). A history of implicit social cognition: Where is it coming from?
Where is it now? Where is it going? In B. Gawronski & B. K. Payne (Eds.), Handbook of implicit
social cognition: Measurement, theory, and applications (pp. 1–15). Guilford Press.
Pervin, L. A. (1994). A critical analysis of current trait theory. Psychological Inquiry, 5, 103–113. https://
doi.org/10.1207/s15327965pli0502_1
Quevedo, R. J. M., & Abella, M. C. (2011). Well-being and personality: Facet-level analyses. Personality
and Individual Differences, 50, 206–211. https://doi.org/10.1016/j.paid.2010.09.030
R Development Core Team. (2012). R: A language and environment for statistical computing. R Founda-
tion for Statistical Computing.
Roefs, A., Huijding, J., Schmulders, F. T. Y., MacLeod, C. M., de Jong, P. J., Wiers, R. W., & Jansen, A.
T. M. (2011). Implicit measures of association in psychopathology research. Psychological Bulletin,
137, 149–193. https://doi.org/10.1037/a0021729
Rosenberg, M. (1965). The measurement of self-esteem. In M. Rosenberg (Ed.), Society and the Adoles-
cent Self-Image (pp. 16–36). Princeton University Press.
Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Soft-
ware, 48(2), 1–36.
Schimmack, U. (2019) The implicit association test: A method in search of a construct. Perspectives on
Psychological Science, 1-19https://doi.org/10.1177/1745691619863798
Schmukle, S. C., Back, M. D., & Egloff, B. (2008). Validity of the five-factor model for the implicit self-
concept of personality. European Journal of Psychological Assessment, 24, 263–272. https://doi.
org/10.1027/1015-5759.24.4.263
Slaughter, J. E., & Kausel, E. E. (2009). The neurotic employee: Theoretical analysis of the influence of
narrow facets of neuroticism on cognitive, social, and behavioral processes relevant to job perfor-
mance. In J. Martocchio & H. Liao (Eds.), Research in personnel and human resources management
(pp. 265–341). Bingley, UK: Emerald Insight.
Soto, C. J., & John, O. P. (2017). The next Big Five Inventory (BFI-2): Developing and assessing a hier-
archical model with 15 facets to enhance bandwidth, fidelity, and predictive power. Journal of Per-
sonality and Social Psychology, 113(1), 117–143. https://doi.org/10.1037/pspp0000096
Steffens, M. C. (2004). Is the Implicit Association Test immune to faking? Experimental Psychology, 51,
165–179. https://doi.org/10.1027/1618-3169.51.3.165
Steffens, M. C., & Schulze-König, S. (2006). Predicting spontaneous Big Five behavior with Implicit
Association Tests. European Journal of Psychological Assessment, 22, 13–20. https://doi.org/10.
1027/1015-5759.22.1.13
Terracciano, A., Löckenhoff, C. E., Zonderman, A. B., Ferrucci, L., & Costa, P. T., Jr. (2008). Personality
predictors of longevity: Activity, emotional stability, and conscientiousness. Psychosomatic Medi-
cine, 70, 621–627. https://doi.org/10.1097/PSY.0b013e31817b9371
The jamovi project (2019). jamovi (Version 1.0) [Computer Software]. Retrieved from https://www.jam-
ovi.org
Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of pos-
itive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54,
1063–1070. https://doi.org/10.1037/0022-3514.54.6.1063
World Health Organization (WHO). (2017). Depression and other common mental disorders: Global
health estimates. Retrieved from http://apps.who.int/iris/bitstream/10665/254610/1/WHO-MSD-
MER-2017.2-eng.pdf
Zanon, C., & Hutz, C. S. (2014). Escala de afetos positivos e negativos (PANAS) [Positive and negative
affects scale (PANAS)]. In C. S. Hutz (Org.), Avaliação em Psicologia Positiva (pp. 63–67). Porto
Alegre, RS: Artmed.
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published
maps and institutional affiliations.
13

Evidence of Validity and Accuracy of An Implicit Measure To Assess The Depressive Trait

Uploaded by

Copyright:

Available Formats

You might also like

Evidence of Validity and Accuracy of An Implicit Measure To Assess The Depressive Trait

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Evidence of Validity and Accuracy of An Implicit Measure To Assess The Depressive Trait

Uploaded by

Copyright:

Available Formats

Trends in Psychology (2023) 31:690–715

Evidence of Validity and Accuracy of an Implicit Measure

Nathalia Melo de Carvalho1 · Rafael Valdece Sousa Bastos1 ·

Accepted: 17 January 2022 / Published online: 28 January 2022

Keywords Implicit assessment · Depressive trait · Personality traits · Big five

* Jean Carlos Natividade

Depressive Trait in the Big5 Model

Implicit Measurement of Personality

Present Study and Hypothesis

The participants were 176 university students. An equivalent proportion of men

Empirical Test of Items

A total of 98 participants were included in study 2, 62.2% were women (n = 61),

Namea 93.9 486.23 71.99 Negativista 98.3 524.21 86.25

Implicit Measure of the Depressive Trait

Factorial Battery of Personality (FBP; Nunes et al., 2010)

Reduced Scale of Personality Descriptors (RED5; Natividade & Hutz, 2015)

Rosenberg’s Self‑Esteem Scale (Hutz & Zanon, 2011; Brazilian version

Structure and Accuracy

Table 3 Factor loadings and Items Factor Loadings h2

Table 4 Adjustment coefficients Single-factor Two-factor Second-order factor

Single-factor means model specified with a single factor explain-

Relationships with Other Variables

1. Implicit − 0.50 0.38 (.75)

1. Implicit − 0.50 0.38 (.75)

Alpha coefficients of the scales are in main diagonal, within parentheses. N = 98

importance of adopting good research practices to avoid false-positive and false-

Author Contribution Nathalia Melo de Carvalho: conceptualization, methodology, software configura-

Conflict of Interest The authors declare no competing interests.

You might also like