Professional Documents
Culture Documents
Development and Validation of The Heidelberg Form For Emotion Regulation Strategies (HFERST) : Factor Structure, Reliability, and Validity
Development and Validation of The Heidelberg Form For Emotion Regulation Strategies (HFERST) : Factor Structure, Reliability, and Validity
research-article2017
ASMXXX10.1177/1073191117720283AssessmentIzadpanah et al.
Article
Assessment
Abstract
The Heidelberg Form for Emotion Regulation Strategies (HFERST) was developed to complement previous emotion
regulation (ER) questionnaires and addresses some of their limitations by measuring eight ER strategies. An initial item
pool was developed in German following expert discussions and adaptation of the items from existing ER scales. Following
a stepwise selection based on the experts’ judgments, 32 items with the best content validity were chosen. Results of an
exploratory factor analysis in Study 1 (N = 399) and two confirmatory factor analyses in Studies 2 (N = 715) and 3 (N =
408) supported the eight-factor structure of the HFERST and provided evidence for its internal consistency, construct and
criterion validity, and clinical utility. Study 4 supported a 2-week stability of the HFERST and Study 5 showed that, following
an ER intervention, the HFERST captured a reduction in dysfunctional and an improvement in functional ER strategies,
which again supports the clinical utility of the HFERST.
Keywords
emotion regulation, assessment, social support, suppression, acceptance, rumination, problem solving
Emotion regulation (ER) is defined as the use of cognitive psychotherapeutic procedures. The Heidelberg Form for
or behavioral strategies to modify the circumstances in Emotion Regulation Strategies (HFERST) is a newly devel-
which an emotion occurs, the experience of an emotional oped questionnaire in German that includes eight ER strate-
response (including its intensity and duration), or the way in gies: rumination, experience suppression, expressive
which an emotion is overtly expressed (Gross, 2002). suppression, avoidance, activity and social support, reap-
Disturbance in the regulation of positive and negative emo- praisal, problem solving, and acceptance. The development
tions is a symptom shared by the majority of mental disor- of the HFERST was based on previous ER literature and
ders (e.g., Barnow & Stopsack, 2012; Carl, Soskin, Kerns, aimed at tackling the limitations of the existing ER mea-
& Barlow, 2013; Cisler & Olatunji, 2012; D’Avanzato, sures. Several tools, including the well-known Cognitive
Joormann, Siemer, & Gotlib, 2013; Izadpanah et al., 2016; Emotion Regulation Questionnaire (CERQ; Garnefski &
Marroquin & Nolen-Hoeksema, 2015). Past studies show Kraaij, 2007), Emotion Regulation Questionnaire (ERQ;
that some ER strategies have proven to be more effective in Gross, 2003), and Difficulties in Emotion Regulation Scale
modifying affect and have, therefore, been conceptualized (DERS; Gratz & Roemer, 2004), have been developed to
as functional strategies (Aldao & Christensen, 2015). measure ER. However, there are several limitations to previ-
Among these strategies, reappraisal, problem solving, and ous questionnaires that need to be considered when develop-
acceptance have been studied most extensively (Barnow, ing new ER tools. First, some questionnaires only measure a
2012; Campbell-Sills & Barlow, 2007). Other groups of ER
strategies have been shown to be less effective in modifying
1
affect, and findings increasingly support the significance of University of Heidelberg, Heidelberg, Germany
*These authors contributed equally to this work. The sequence of other
these dysfunctional strategies (e.g., suppression, rumina- coauthors is based on the level of contribution.
tion, and avoidance) for the development and treatment of
various psychopathologies (e.g., Aldao & Nolen-Hoeksema, Corresponding Author:
Shahrzad Izadpanah, Institute of Psychology, Department of Clinical
2010; Reinecke et al., 2015). Psychology and Psychotherapy, University of Heidelberg, Hauptstraße
These findings indicate the importance of well- 47-51, Heidelberg 69117, Germany.
designed ER measurement tools that assist preventive and Email: shahrzad.izadpanah@psychologie.uni-heidelberg.de
2 Assessment 00(0)
limited variety of ER strategies. For example, the ERQ Therefore, in keeping with Salters-Pedneault et al. (2010),
merely assesses reappraisal and suppression, whereas other we go beyond this well-known but narrow perspective of
strategies such as rumination, acceptance, and avoidance suppression (behavioral or expressive suppression) and
have also proved significant for psychopathology (Aldao, define experience suppression as “an attempt to act directly
Nolen-Hoeksema, & Schweizer, 2010; Francisco, 2010; on an emotional response by putting it away or putting it out
Izadpanah et al., 2016). The development of ER question- of mind.” With this perspective, we aimed to include both
naires, which include the most significant ER strategies is experience and expressive suppression in the HFERST.
essential in capturing a full picture of the habitual use of ER Fifth, previous ER questionnaires have largely neglected
strategies, especially in the context of psychological situation selection, which is the first stage of the emotion
disorders. generative process in the process model of ER (Gross, 2003).
Second, some ER questionnaires which measure various This might explain why strategies related to situation selec-
ER strategies included subscales that show significant theo- tion have been understudied so far (Quoidbach, Mikolajczak,
retical and empirical overlap with psychopathology. For & Gross, 2015). Considering this point, we included the ER
example, while self-blame has been considered as an ER strategy avoidance in the HFERST.
strategy in CERQ, it seems to be part of the depression pro- Sixth, previous studies show that behavioral strategies
file as feeling bad about oneself, displaying excessive self- such as seeking social support (Marroquin & Nolen-
blame or the belief that one is a failure are characteristics of Hoeksema, 2015) and engaging in other activities such as
depression rather than ER strategy (Fried, Nesse, Zivin, sport (Penedo & Dahn, 2005) are effective in regulating
Guille, & Sen, 2014; Zahn et al., 2015). Similarly, catastro- emotions and contribute to mental health. For this reason,
phizing has been included as an ER strategy in CERQ, we have added a behavioral scale in the HFERST, which
while it is regarded as one of the cognitive distortions within measures the extent to which individuals respond to their
the depression profile (Leung & Poon, 2001). emotional experiences by seeking social support and engag-
Third, the assessment of specific strategies needs to be ing in activities. By considering these critical points, we
elaborated in order to test different aspects of a single strat- aimed to develop an ER questionnaire that not only inte-
egy. For instance, previous scales have mostly limited the grates different important strategies into a unified multidi-
assessment of acceptance to the acceptance of the situation mensional questionnaire but also complements the existing
(e.g., CERQ), while an important aspect of “acceptance” is measures and addresses the aforementioned limitations.
being tolerant toward emotional experiences and embracing
them (Hayes, Luoma, Bond, Masuda, & Lillis, 2006).
Although the DERS measures nonacceptance of negative
Item Development and Content Validity
emotional experiences, the acceptance of both situation and We designed the HFERST in three steps based on the sug-
emotion needs to be considered as different aspects of the gestions of Haynes, Richard, and Kubany (1995). Given
same strategy; therefore, new scales need to include items that we had the opportunity of exploring a rich literature on
for acceptance of both situation and emotion. Furthermore, ER, we applied a conceptual developmental strategy rely-
to our knowledge, the only ER questionnaire that measures ing on current ER theories and scales. In the first phase,
acceptance as an ER strategy (namely, the CERQ) shows a following regular discussions between two ER experts with
low score for internal reliability and is often weakly or advanced knowledge of ER literature, an item pool (in
insignificantly related to various psychopathology symp- German) was developed on the basis of reviewing previous
toms (Domínguez-Sánchez, Lasa-Aristu, Amor, & Holgado- literature and the existing measures of ER. This pool
Tello, 2011; Jermann, Linden, d’Acremont, & Zermatten, included items that were generated by the experts based on
2006; Tuna & Bozo, 2012). Previous research has also their knowledge of ER as well as adapted items, which were
found a small positive relation between CERQ-acceptance selected from previously validated measures. The selection
subscale and psychopathology (Tuna & Bozo, 2012). It has of items from previous measures was based on the decision
been argued that the items in the CERQ seem to measure of both experts about how well the items represent each
thoughts about acceptance or a maladaptive version of strategy. In most cases, the formulation and content of the
acceptance (e.g., passive resignation), when acceptance is items were significantly modified during the second and the
considered to be an adaptive strategy (Jermann et al., 2006). third stages of the item development, to improve the content
Fourth, emotional suppression has been defined as an validity and formulation of items on the basis of the ER lit-
attempt to reduce unwanted emotional experiences but it can erature. The initial item pool included 124 items that asked
also take different forms (Salters-Pedneault, Steenkamp, & about four dysfunctional strategies (rumination, experience
Litz, 2010). While previous measures have mostly focused suppression, expressive suppression and avoidance), and
on “expressive suppression,” suppression of experienced four functional strategies (activity and social support, reap-
feeling is a related yet distinct aspect of suppression that is praisal, problem solving, and acceptance). Strategies were
surprisingly understudied (Salters-Pedneault et al., 2010). included on the basis of both the ER literature regarding the
Izadpanah et al. 3
significance of various strategies for mental health (Aldao on the opinion of experts in the field of ER. We aimed at
et al., 2010; Barnow, 2012) and the limitations of previous keeping idiosyncratic effects minimal by (a) building our
measures as mentioned above. In the HFERST, we distin- work on a very rich theoretical end empirical literature on
guished between ER strategies which are mostly dysfunc- ER, (b) starting with established measures capturing various
tional and those which are mostly functional. However, it is facets of ER, and (c) developing the item pool in three steps
of note that this distinction, though supported by empirical with independent sets of experts (with the exception of one
research (Campbell-Sills & Barlow, 2007; Reinecke et al., expert, SB). We decided that this conceptual strategy would
2015), is not an absolute one because the functionality of be an economical and reasonable solution for building an
ER strategies can be influenced by factors relevant to the initial pool of 32 items that we would then subject to empiri-
ER context (Aldao, 2013). cal tests of their factor structure and validity.
In the second phase, all items were presented to a group of
five clinical psychologists with advanced knowledge of the
ER literature (each of whom had published at least once on
Overview
ER). Only one of these five experts was involved in the first The HFERST was designed with the aim of broadening ER
stage of item development. They proceeded to discuss how assessment by providing a measurement tool which, since it
well each item represented the relevant scale, before sorting includes various theoretical and empirical-based ER strate-
the items into each scale based on its content validity (the gies, not only has economical value but also enables research-
scale extended from 1 to 5). The psychologists were also ers to capture a full profile of the habitual use of ER strategies.
asked to note their suggestions for modifying the formulation Second, the HFERST includes both expressive and experien-
of items, with the aim of linguistic improvement and increas- tial type of suppression, as well as embracing a broader
ing item validity. Based on the experts’ agreements, the six understanding of the ER strategy “acceptance,” which
best items were selected for each scale. In the third phase, includes items for both situational and emotional types. It is
items were further analyzed by a psychologist with extensive further consistent with the theoretical background to accep-
clinical and research experience in the field of ER (SB), who tance as an adaptive ER strategy. In the following sections,
modified the item formulation based on suggested modifica- we provide results from a series of studies investigating the
tions in the second phase (for more information on the applied psychometric properties of the HFERST, supplementing the
modifications on final items in the HFERST, see Appendix theory-driven approach in item development with empirical
A). Based on the experts’ evaluations and the suggestions fol- data collected in more than 1,500 participants. Study 1 in this
lowing the second phase, he further selected five items for article conducts an exploratory factor analysis (EFA) and
activity and social support, three items for acceptance and tests the scale and item characteristics of the HFERST. In
four items for each of the other scales (rumination, experi- Study 2 and Study 3, we conduct two confirmatory factor
ence suppression, expressive suppression, avoidance, reap- analyses (CFA) using two separate samples, to investigate
praisal, and problem solving). further the reliability and validity of the HFERST. Study 4
In the HFERST, participants are asked to indicate to investigates the test–retest reliability of the HFERST, and in
what extent the items applied to them over the past 4 weeks, the Study 5, we apply the HFERST to examine changes in
with responses ranging from 1 to 5 (1 = never, 2 = occasion- ER after conducting an ER intervention.
ally, 3 = about half the time, 4 = usually, 5 = always). Higher
responses indicate a more frequent use of the respective
Study 1
scales.
In order to further evaluate the content validity of the This study had three main aims: First, we conducted an EFA
HFERST, we asked seven clinical psychologists with to investigate the factor structure of the HFERST. Second,
advanced knowledge in ER literature and research (with ER we investigated item–total correlations to test the suitability
as the central research topic and at least one publication on of each item for the questionnaire. Third, we examined the
ER) to rate the items. They received a paper version of the construct validity of the HFERST by investigating the cor-
HFERST and were asked to rate each of the items for how relation coefficients among the HFERST, depression, and a
well it reflected the respective scale, using a 5-point scale well-established ER questionnaire, the CERQ.
from 1 (not at all) to 5 (very well). The average rating for
the 32 items was 4.39 (SD = 0.69) indicating that, on aver-
Method
age, raters believed that items represented their respective
scales well. Apart from one of the experts (SB) who was Participants and Procedure. Data for this sample were gath-
involved in all steps of item development, we consulted ered through two online surveys using Unipark software.
each expert only at one of the developmental stages. The first sample included 240 individuals (188 females)
In conclusion, we pursued a purely conceptual strategy from 16 to 70 years old (M = 29.60 years) and the second
for developing the initial item pool of the HFERST, building sample consisted of 159 individuals (109 females) aged 18
4 Assessment 00(0)
to 65 years (M = 24.01 years). Participants were compen- Bayesian information criterion reached its minimum for the
sated by being given the chance to win coupons of €20. eight-factor solution (−1113 compared with −1001, for
Since both samples were collected by using the same seven factors and −1033 for nine factors, respectively).
method and the same population was targeted through col- Given these diverging results, we ran the EFA with six,
lecting data from similar social networks (e.g., Facebook), seven, eight, and nine extracted factors, respectively (princi-
we combined data sets to conduct the factor analysis (N = pal axis factoring with oblimin rotation). For the six- and
399; 297 women). The majority of participants were Ger- seven-factor solutions, the items supposedly assessing
man (88.5%). Their age ranged from 16 to 70 years (M = expressive suppression and experience suppression loaded
27.37 years, SD = 9.81) and 45.4% were in a committed on one common factor, but the four items from the postu-
relationship. Regarding education level, 62.4% had received lated experience suppression scale showed substantial cross-
their high school diploma, and 29.6% had at least a bache- loadings (absolute factor loadings >.30) on the avoidance
lor’s degree. factor. Additionally, two activity items (“When I experience
a strong emotion, I let it out by doing sports [e.g., jogging,
Measures. Participants in Samples 1 and 2 completed the fitness training, boxing, . . .,” “I relieve tension and negative
HFERST with 32 items that measure rumination (4 items), emotion by relaxation [e.g., yoga, relaxation techniques,
avoidance (4 items), expressive suppression (4 items), mindfulness]”) did not have absolute loadings >.30 on any
experience suppression (4 items), reappraisal (4 items), factor. In the six-factor solution, the two social support items
activity and social support (5 items), acceptance (3 items), loaded negatively on the suppression factor, while one activ-
and problem solving (4 items). Furthermore, we applied the ity item (“When I feel bad, I treat myself to something nice
Beck Depression Inventory (BDI), a self-rating measure for in order to feel better”) was associated with the acceptance
depression with 21 items and good reliability and stability factor; in the seven-factor solution, these three items all
(Beck, Steer, & Carbin, 1988). The BDI was applied to both loaded on one common factor. The solution with eight fac-
Samples 1 and 2. The Cronbach’s alpha of the inventory for tors provided evidence for a simple structure, with no abso-
our sample was .91. In addition, participants in Sample 2 lute cross-loadings greater than .29. However, two of the
completed the CERQ (Garnefski, Kraaij, & Spinhoven, activity items did not load on any of the factors (absolute
2001), which consists of 36 Likert-type items ranging from loadings <.28 for all). Finally, in the nine-factor solution,
1 (sometimes) to 5 (always) and measures nine cognitive these two items loaded on one common factor, but the eigen-
ER strategies: self-blame, rumination, catastrophizing, value of this ninth factor was less than 1 (.87). Taken
other-blame, positive reappraisal, acceptance, positive refo- together, our findings suggest that the eight-factor solution
cusing, refocus on planning, and putting into perspective. is preferred: Extracting fewer factors results in cross-load-
The German translation of CERQ has shown adequate ings and extracting nine factors yields one factor with eigen-
internal consistency (.60 < α < .73; Loch, Hiller, & Witthöft, value less than 1. Table 1 depicts the pattern matrix of the
2011). solution with eight factors. Given that only one activity item
loaded on any of the factors, we decided to remove all three
activity items and rename the respective factor as social sup-
Results
port for the following analyses. Furthermore, Item 25 (“I am
Factor Analysis. Prior to conducting the factor analyses, the afraid of strong intense emotions”) exhibited comparatively
response distributions of all individual items were exam- low loading on its designated factor (avoidance) and at the
ined. As none of these variables were excessively skewed or same time showed substantial cross-loading on the experi-
kurtotic (Kendall & Stuart, 1958), no items were excluded ence suppression factor. To ascertain a simple structure of
from the analyses on the basis of their response distributions. the scales, we also removed this item for all further
In a first step, we performed an EFA to investigate the struc- analyses.
ture of the HFERST. Although we had specific expectations
about the factor structure of this scale, we sought to explore Correlations With BDI and CERQ. We computed correlation
possible alternatives as well. To that end, we first deter- coefficients after excluding the aforementioned four items
mined the number of factors to be retained using three crite- following the EFA. To control the effect of gender on our
ria: parallel analysis, the minimum average partial (MAP) results, we conducted a partial correlation analysis. In line
method, and the Bayesian information criterion. Parallel with our expectations, depression was positively correlated
analysis suggested nine factors to be retained although the with rumination, expressive/experience suppression, and
adjusted eigenvalue of the ninth factor was only slightly avoidance, while it was negatively correlated with social
larger than zero (.021) and noticeably smaller than the eigen- support, problem solving, acceptance, and reappraisal (see
value of the eighth factor (.267). The MAP test suggested six Table 2). In general, HFERST subscales showed stronger
factors, but the MAP score was (rounded to the third deci- correlation with depression in comparison with the CERQ
mal) identical for seven or eight factors (.017). Finally, the subscales. Furthermore, the correlation coefficients between
Izadpanah et al. 5
Table 1. Factor Structure of the Heidelberg Form for Emotion Regulation Strategies in Study 1.
Activity/
Problem Expressive Experience Social
Items Rumination Reappraisal Acceptance solving suppression suppression Avoidance support
1. When I have negative .73
feelings, I often brood over
the question, why I am
feeling this way.
2. After emotional experiences/ .67
situations, I think a lot about
what I did and said, so as
to understand my feelings
better
3. I realize, again and again, .80
that I have to think about
something that made me
angry or sad
4. I remember past conflicts .70
often and think about what I
could have done differently
5. When I feel bad, I try to .72
see the positive aspects of a
situation.
6. When I find myself in a .77
stressful situation, I change
my thoughts about the
situation in such manner,
that I get calm.
7. When I want to feel better, .79
I concentrate on the good
aspects of a situation.
8. I change my feelings by .91
thinking differently about my
current situation.
9. When I cannot change .75
something, I accept the
situation as it is.
10. I am able to tolerate and .59
endure uncomfortable
situations.
11. I am able to accept things as .95
they are.
12. When looking for solutions .74
for a problem, I do (not
only) rely on my instinct but
try to think as rationally as
possible.
13. I think about possible .76
solutions of how to change
the situation.
14. When I am confronted .75
with problems, I think very
carefully about how I can
deal best with the situation.
15. When I have to make a .72
decision, I carefully weigh the
different alternatives against
each other.
(continued)
6 Assessment 00(0)
Table 1. (continued)
Activity/
Problem Expressive Experience Social
Items Rumination Reappraisal Acceptance solving suppression suppression Avoidance support
16. When something makes me .78
angry or sad, I try to hide my
feelings from others.
17. I hide physical expressions of .80
my feelings.
18. It is very hard for other .75
people to tell how I am
feeling at the moment.
19. Even when I am very .58
agitated, I am able to keep a
calm exterior.
20. I try not to allow for .64
negative emotions to begin
with.
21. I rarely let my emotions run .58
high, but keep them down.
22. Whenever possible, I avoid it .71
to realize my feelings.
23. When I have strong .77
emotions, I immediately push
them aside.
24. I prefer to avoid situations, .74
which could cause negative
emotions in me.
25. I am afraid of strong, .29 .35
intensive emotions.
26. I try to avoid thoughts about .65
things that weigh me down.
27. Whenever possible, .82
I take care that I am
not confronted with
uncomfortable situations.
28. When I feel bad, I treat .22 −.26 .40
myself to something nice to
feel better.
29. I like to share negative and .78
positive feelings by calling or
meeting up with others and
talking about them.
30. When I experience a strong −.25 .17
emotion, I let it out by doing
sports (e.g., jogging, fitness
training, boxing, etc.).
31. I often talk about my .70
emotions with my partner or
my close friends.
32. I relieve tension and negative .25 .28
emotion by relaxation (e.g.,
yoga, relaxation techniques,
mindfulness).
Note. Loadings >.30 are in boldface. The items are originally in German as presented in Appendix A.
Izadpanah et al. 7
Table 2. Partial Correlation Coefficients Among HFERST, CERQ, and BDI Controlling for Gender, Together With Their Means,
Standard Deviations, and Cronbach’s Alphas in Study 1.
9 10 11 12 13 14 15 16 17 18
HFERST (N = 399)
1. Rumination .24** .54*** .29*** .16* .24** −.09 −.05 .07 −.12 .26***
2. Experience suppression −.07 −.38*** −.09 .07 −.07 −.10 −.04 −.17* −.08 .30***
3. Expressive suppression .04 −.17* .02 .13 .04 −.15 −.07 −.07 −.05 .36***
4. Avoidance −.07 −.19* −.03 .11 −.10 −.01 −.07 −.17* −.20** .42***
5. Reappraisal −.11 .07 −.21** −.14 .02 .44*** .24** .40*** .60*** −.45***
6. Acceptance −.09 −.03 −.20** −.05 .24** .13 .07 .22** .30*** −.38***
7. Problem solving .06 .13 −.14 −.05 .14 .19* .19* .58*** .22** −.34***
8. Social support −.03 .28** −02 −.14 .10 .18* .11 .31*** .20* −.41***
CERQ (N = 159)
9. Self-blame .30*** .30*** −.12 .48*** −.11 .02 .03 −.04 .35***
10. Rumination .35*** .13 .40*** .06 .12 .31*** .11 .09
11. Catastrophizing .50*** .26** −.06 −.11 −.09 −.18* .35***
12. Other-blame .02 .06 −.05 .05 −.06 .10
13. Acceptance .09 .20* .30*** .20* .18*
14. Positive refocusing .32*** .29*** .40*** −.14
15. Putting into perspective .29*** .50*** −.17*
16. Refocus on planning .43*** −.18*
17. Positive reappraisal −.27**
BDI (N = 399)
18. Depression
Mean 2.53 3.10 1.90 1.86 3.05 2.42 3.44 3.62 3.11 13.08
Standard deviation 0.94 0.94 0.79 0.79 0.82 0.82 0.99 1.00 1.04 13.36
Cronbach’s alpha .74 .71 .72 .88 .84 .80 .72 .87 .82 .91
Note. HFERST = Heidelberger Form for Emotion Regulation Strategies; CERQ = Cognitive Emotion Regulation Questionnaire; BDI = Beck Depression
Inventory.
*p < .05. **p < .01. ***p < .001.
the HFERST subscales of reappraisal and rumination and supported our hypothesized eight-factor solution for the
the same subscales in CERQ were stronger than their cor- HFERST. Following this analysis, we decided to remove
relation coefficients with BDI. Again, in line with our the activity items as they failed to load on any factor. Given
expectations, the acceptance subscale from the HFERST that activity items measured engaging in different specific
showed a negative correlation with depression, while accep- activities, individuals might have tended to use one activity
tance subscale from the CERQ showed a small positive cor- (e.g., mindfulness/relaxing) but not another activity (e.g.,
relation with depression. sport) as an ER strategy, which resulted in low interitem
correlation of the activity items. Furthermore, one avoid-
Internal Consistency. Table 4 (upper panel) presents internal ance item was also removed due to substantial cross-load-
consistency, interitem, and corrected item–total correlations ing. The rest of the items showed the expected loadings on
for each HFERST subscale. The HFERST subscales showed their respective factors. Results also supported very good
good reliability (greater than .80 for all subscales). According internal consistency of the HFERST. A comparison of
to Kline (1979), the mean interitem correlations between .3 Cronbach’s alpha of the HFERST with the Cronbach’s
and .7 indicate an ideal balance between item specificity and alpha of the CERQ in our findings and previous studies sup-
scale homogeneity. Except for social support that slightly ports the superior internal consistency of the HFERST sub-
exceeded this value (.71), the average interitem correlations scales (e.g., Domínguez-Sánchez et al., 2011; Loch et al.,
for all other subscales were in this acceptable range, which 2011; Tuna & Bozo, 2012). In particular, similar subscales
supports the homogeneity of the HFERST subscales. on both questionnaires (i.e., acceptance, rumination, and
reappraisal) show a better internal consistency in the case of
the HFERST than that of CERQ (see Loch et al., 2011).
Brief Discussion Correlation analyses supported the validity and clinical util-
In this study, we investigated the factor structure of the ity of the HFERST, showing significant correlation coeffi-
HFERST with eight ER strategies. Results of the EFA cients between depression and the HFERST subscales in the
8 Assessment 00(0)
expected directions. Furthermore, those subscales of the goal-directed behavior (α = .91), impulse control difficul-
HFERST, which were also included in the CERQ showed a ties (α = .89), lack of emotional awareness α = .86), limited
stronger correlation with each other than with depression access to ER strategies (α = .93), and lack of emotional
(e.g., acceptance, rumination, reappraisal, problem solving, clarity (α = .90). The Cronbach’s alpha of the total scale in
and refocus on planning). An exception was the HFERST- our sample was .96. Furthermore, the Brief Symptoms
acceptance subscale, which was less strongly correlated Inventory (BSI) was used, which includes 53 items rated
with the CERQ-acceptance than with depression. However, on a 5-point Likert-type scale, ranging from not at all (0) to
while the correlation between the HFERST-acceptance and extremely (4). The BSI measures the presence of nine psy-
depression was moderate and in the expected direction, chopathology symptoms over the past 7 days and has
similar to other previous study, we found a small positive shown excellent reliability and validity (Franke, 2000).
correlation between depression and acceptance subscale The Cronbach’s alphas for each subscale in our sample
from CERQ (Tuna & Bozo, 2012). As Jermann et al. (2006) were as follows: somatization = .88, depression = .92,
argue, it seems that items from acceptance subscale in the obsessive compulsive = .87, interpersonal sensitivity = .88,
CERQ do not capture acceptance as an adaptive strategy anxiety disorders = .89, hostility = .83, phobic anxiety =
and they also partly measure thoughts about acceptance .89, paranoid anxiety = .84, and psychoticism = .84.
rather than acceptance as an ER strategy. Furthermore, anal-
yses of item characteristics in this study supported the
homogeneity of the subscales showing no extreme low or
Results
high mean interitem correlation. Factor Analysis. Using CFA procedure, the final eight-factor
model from Study 1 was investigated. We used three fit
indices: model chi square divided by degrees of freedom
Study 2 (χ2/df), comparative fit index (CFI; Bentler, 1990), and root
This second study had two main objectives: First, we con- mean square error of approximation (RMSEA; Marsh,
ducted a CFA to evaluate the final eight-factor structure of Balla, & Hau, 1996). In general, χ2/df values less than 2 are
the HFERST from Study 1, using a larger sample. Second, considered good and values less than 5 are acceptable
reliability, convergent, and construct validity of the ques- (Schumacker & Lomax, 2004). Furthermore, values greater
tionnaire were further investigated. than .90 for CFI and values less than .08 for RMSEA repre-
sent an acceptable fit. The model showed adequate fit: χ2 =
1117.38, df = 322, χ2/df = 3.47, RMSEA = .06, CFI = .92.
Method Mean and standard deviation for each item, along with their
Participants. A total of 734 participants (547 women) rang- factor loadings, are presented in Table 3.
ing from 14 to 81 years old (M = 27.83 years, SD = 10.01)
completed an online survey designed via Unipark software. Reliability. We evaluated internal consistency of the eight
On investigation, the data showed that 19 participants had subscales using Cronbach’s alpha. Table 4 (lower panel)
Mahalanobis distance at p < .001. These individuals were shows alpha coefficients and item and scale information,
therefore recognized as multivariate outliers and removed along with the correlations among HFERST subscales. As
from further analysis (Tabachnick & Fidell, 2007), result- illustrated in Table 4, the HFERST subscales showed good
ing in a total of 715 remaining participants. From this final internal consistencies and were also fairly homogenous
sample, 54.7% had finished high school and 31.7% had at with inter-item correlations ranging from .46 to .72
least a bachelor’s degree. Furthermore, 67% were students
at the time of assessment, and 55.2% reported being in a Convergent and Discriminant Validity. Table 5 shows associa-
committed relationship. tions among HFERST, DERS, and BSI subscales. Results
showed a good concurrent validity for the HFERST. In line
Measures. Participants completed the final version of the with our expectations, there was a positive correlation
HFERST from Study 1 with 28 items measuring rumina- between experience/expressive suppression, avoidance,
tion, experience suppression, expressive suppression, rumination (known as mainly dysfunctional), and DERS
avoidance, acceptance, reappraisal, problem solving, and scores, while DERS subscales were negatively correlated
social support. The DERS was also applied that includes 36 with problem solving, social support, reappraisal, and
items rated on a 5-point Likert-type scale from 1 (almost acceptance (known as mainly functional). Furthermore, as
never) to 5 (almost always). Higher scores indicate greater we expected, HFERST dysfunctional ER strategies were
impairment in ER. The DERS demonstrates good psycho- positively correlated with psychopathology symptoms,
metric characteristics (Gratz & Roemer, 2004). It measures while functional strategies (i.e., reappraisal, social support,
ER difficulties on six dimensions of nonacceptance of acceptance, and problem solving) were negatively corre-
emotional responses (α = .90), difficulties engaging in lated with all psychopathology symptoms.
Izadpanah et al. 9
Table 3. Result of the Confirmatory Factor Analysis in Study 2 Representing Standardized Regression Weights, Means, and Standard
Deviations for Each Item of the Final Version of the Heidelberg Form for Emotion Regulation Strategies.
Problem Expressive Experience Social
Items M (SD) Rumination Reappraisal Acceptance solving suppression suppression Avoidance support
Table 3. (continued)
Note. The items are originally in German as presented in Appendix A. All factor loadings are significant (p < .001).
Testing a Two-Dimensional Model. As mentioned in the intro- exhibit acceptable model fit to the HFERST items as well.
duction, ER strategies can be broadly classified into the two To that end, we set up a model in which all items from the
categories of functional and dysfunctional. We tested supposedly functional ER subscales (reappraisal, accep-
whether such a two-dimensional measurement model would tance, social support, and problem solving) loaded on one
Table 4. Partial Correlation Coefficients Among HFERST Subscales Controlling for Gender, Together With Internal Consistencies and Item Charcteristics in Study 1 and
Study 2.
IIC CITC
Note. HFERST = Heidelberg Form for Emotion Regulation Strategies; IIC = interitem correlation; CITC = corrected item–total correlation.
* p < .05. **p < .01. ***p < .001.
11
12
Table 5. Partial Correlation Coefficients Among HFERST, BSI, and DERS Scales Controlling for Gender, Together With Their Means and Standard Deviations in Study 2.
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
HFERST
1. Rumination .21*** .24*** .29*** .29*** .24*** .25*** .20*** .25*** .25*** .28*** .32*** .29*** .26*** −.00 .38*** .20*** .31***
2. Experience .24*** .29*** .31*** .31*** .30*** .17*** .28*** .32*** .35*** .33*** .35*** .18*** .19*** .56*** .27*** .43*** .41***
suppression
3. Expressive .28*** .32*** .35*** .37*** .33*** .23*** .35*** .35*** .38*** .38*** .38*** .17*** .22*** .45*** .31*** .40*** .40***
suppression
4. Avoidance .32*** .32*** .37*** .33*** .34*** .30*** .38*** .35*** .34*** .38*** .38*** .34*** .35*** .31*** .39*** .37*** .45***
5. Reappraisal −.25*** −.31*** −.36*** −.40*** −.29*** −.33*** −.31*** −.34*** −.33*** –.37*** −.32*** −.29*** −.34*** −.36*** −.51*** −.35*** −.46***
6. Acceptance −.30*** −.36*** −.39*** −.37*** −.34*** −.37*** −.33*** −.36*** −.37*** −.40**** −.33*** −.43*** −.41*** −.23*** −.45*** −.26*** −.45***
7. Problem solving −.26*** −.31*** −.29*** −.31*** −.25*** −.26*** −.26*** −.28*** −.33*** −.33*** −.24*** −.28*** −.34*** −.33*** −.33*** −.30*** −.38***
8. Social support −.35*** −.37*** −.39*** −.44*** −.36*** −.29*** −.39*** −.40*** −.46*** −.44*** −.36*** −.20*** −.30*** −.56*** −.40*** −.47*** −.47***
BSI
9. Somatization .73*** .68*** .69*** .81*** .64*** .73*** .67*** .71*** .84*** .49*** .48*** .62*** .35*** .59*** .52*** .64***
10. Obsessive .76*** .82*** .76*** .71*** .69*** .71*** .80*** .89*** .56*** .60*** .64*** .43*** .67*** .61*** .74***
Compulsive
11. Interpersonal .84*** .75*** .74*** .72*** .79*** .82*** .89*** .61*** .56*** .64*** .40*** .73*** .61*** .75***
Sensitivity
12. Depression .75*** .70*** .73*** .75*** .87*** .92*** .62*** .58*** .66*** .45*** .77*** .64*** .78***
13. Anxiety disorder .71*** .81*** .72*** .75*** .90*** .53*** .52*** .67*** .38*** .64*** .57*** .70***
14. Hostility .66*** .75*** .72*** .82*** .48*** .48*** .69*** .31*** .64*** .51*** .65***
15. Phobic anxiety .72*** .75*** .86*** .51*** .50*** .65*** .39*** .63*** .55*** .68***
16. Paranoid .78*** .86*** .51*** .49*** .61*** .42*** .63*** .54*** .67***
thoughts
17. Psychoticism .91*** .61*** .54*** .67*** .48*** .70*** .66*** .77***
18. Global Severity .63*** .60*** .74*** .46*** .76*** .66*** .81***
Index
DERSa
19. Non acceptance .53*** .57*** .45*** .72*** .59*** .82***
20. Goal oriented .64*** .26*** .71*** .47*** .77***
21. Impulse control .38*** .73*** .58**v .82***
22. Awareness .42*** .62*** .64***
23. Strategies .62*** .89***
24. Clarity .81***
25. Total
M 1.53 1.97 1.99 1.88 1.73 1.63 1.51 1.66 1.63 1.72 2.42 2.74 1.96 2.55 2.31 2.28 2.38
SD 0.72 0.92 1.03 1.04 0.83 0.75 0.88 0.85 0.86 0.77 1.01 1.05 0.91 0.89 1.04 0.94 0.77
Note. HFERST= Heidelberg Form for Emotion Regulation Strategies; BSI = Brief Symptom Inventory; DERS = Difficulties in Emotion Regulation Scale.
a
Higher scores represent more difficulties in emotion regulation.
***p < .001.
Izadpanah et al. 13
factor, while the other items belonging to the four dysfunc- and the internal consistency of the HFERST using BDI and
tional ER strategies (rumination, avoidance, expressive DERS.
suppression, and experience suppression) loaded on the
other factor (both factors were allowed to correlate). Results
Method
showed inacceptable fit for this model, χ2(349) = 5196.28,
χ2/df = 14.89, RMSEA = .14, CFI = .51, suggesting that the Participants. A total of 418 participants completed an online
multifaceted structure of ER use as assessed via the survey designed via Unipark software. Following calcula-
HFERST does not justify a mere classification of these tion of Mahalanobis distance, 10 participants were recog-
strategies as functional and dysfunctional. nized as multivariate outliers (p < .001) and were removed
from further analysis (Tabachnick & Fidell, 2007). This
Gender Effects. In an additional analysis, we checked for final sample included 408 participants (305 women) rang-
measurement invariance across genders. Having separated ing from 18 to 73 years old (M = 26.74 years, SD = 10.92).
the analyses by gender, findings showed that weak/metric A majority of the sample reported to be student at the time
invariance was tenable, since the model fit did not decrease of assessment (77.11%).
when item loadings were constrained to equality across the
genders, χ2(20) = 14.8, p > .05 (for information on the issue Measures. Participants completed the final version of the
of testing for measurement invariance, see Dimitrov, 2010; HFERST with 28 items measuring rumination, experience
Meredith, 1993). Further constraining the item intercepts to suppression, expressive suppression, avoidance, accep-
equality resulted in statistically significant deterioration of tance, reappraisal, problem solving, and social support. The
model fit, χ2(20) = 87.89, p < .001. Taken together, these DERS (Gratz & Roemer, 2004) was also applied that
results suggest that the HFERST shows evidence for weak/ includes 36 items measuring ER on six dimensions of non-
metric, but not strong measurement invariance across gen- acceptance of emotional responses, difficulties engaging in
ders. Of note, a vast body of empirical studies suggest that goal-directed behavior, impulse control difficulties, lack of
there are gender differences in applying ER strategies (e.g., emotional awareness, limited access to ER strategies, and
Nolen-Hoeksema & Aldao, 2011; Zlomke & Hahn, 2010). lack of emotional clarity. Furthermore, we applied the BDI,
Given that our analyses were mainly correlational based, a self-rating measure for depression with 21 items and good
the result of strong factorial invariance does not signifi- reliability and stability (Beck et al., 1988). Cronbach’s
cantly influence the validity of our findings as long as the alpha for all these scales in our sample are represented in
assumption of metric invariance holds (Dimitrov, 2010). Table 6.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
HFERST
1. Rumination −.15** .02 .09 −.18*** −.22*** .12* .08 .27*** .35*** .24*** −.20*** .35*** .15** .30*** .12*
2. Experience .53*** .41*** −.07 .05 −.14** −.51*** .17*** −.03 .02 .36*** .10* .23*** .18*** .06
suppression
3. Expressive .29*** −.09 −.06 −.04 −.52*** .20*** .003 −.02 .27*** .15** .26*** .19*** .12*
suppression
4. Avoidance −.15** −.04 .01 −.24*** .27*** .25*** .11* .15** .26*** .12* .28*** .12*
5. Reappraisal .32*** .22*** .15** −.25*** −.30*** −.32*** −.35*** −.48*** −.31*** −.47*** −.26***
6. Acceptance .14** .07 −.26*** −.31*** −.30*** −.17*** −.40*** −.31*** −.41*** −.26***
7. Problem solving .15** −.06 −.13** −.23*** −.28*** −.17*** −.28*** −.25*** −.12*
8. Social support −.20*** −.05 −.08 −.34*** −.21*** −.24*** −.25*** −.10*
DERSa
9. Nonacceptance .43*** .43*** .20*** .65*** .43*** .76*** .27***
10. Goal oriented .60*** .05 .64*** .32*** .75*** .26***
11. Impulse control .09 .65*** .39*** .75*** .25***
12. Awareness .20*** .40*** .42*** .06
13. Strategies .52*** .88*** .41***
14. Clarity .69*** .23***
15. Total .36***
BDI
16. Depression
M 3.70 2.18 3.07 2.99 3.23 3.34 3.96 3.72 2.30 2.71 1.88 2.37 2.13 2.04 2.24 .99
SD .84 .80 .97 .89 .88 .88 .73 1.18 .85 .95 .71 .67 .82 .64 .55 .65
Cronbach’s alpha .78 .83 .84 .78 .86 .80 .82 .86 .86 .89 .84 .78 .89 .79 .93 .95
Note. HFERST = Heidelberg Form for Emotion Regulation Strategies; BSI = Brief Symptom Inventory; DERS = Difficulties in Emotion Regulation Scale. N = 418.
a
Higher scores represent more difficulties in emotion regulation.
*p < .05. **p < .01. ***p < .001.
Izadpanah et al. 15
control difficulties correlated with lower scores on experi- only experience/expressive suppression scales and social sup-
ence and expressive suppression scales (r = −.24, p = .01; r port did not show a significant association with difficulties
= .19, p = .05, respectively) among men but not women. engaging in goal-directed behavior. Furthermore, the relation
Interestingly, impulse control difficulties were associated between these three HFERST subscales and impulse control
with higher social support among men (r = .21, p = .03) but difficulties was gender dependent, while all other correlations
with lower social support among women (r = −.15, p < .01). were significant and in the expected directions, also after con-
As we expected, there was a positive correlation between trolling for gender. Testing measurement invariance across
dysfunctional ER strategies and depression, while more genders supported weak factorial invariance but not strong
functional strategies (i.e., reappraisal, social support, accep- factorial invariance. However, considering that our analyses
tance, and problem solving) were negatively correlated with were mainly correlational in nature, and given that a vast body
depression. An exception to this was the case of experience of empirical studies support gender differences in applying
suppression, which did not show a significant correlation ER strategies (e.g., Nolen-Hoeksema & Aldao, 2011; Zlomke
with BDI. Further analysis of the data showed that experi- & Hahn, 2010), the result of strong factorial invariance does
ence suppression was positively correlated with depression not significantly influence the validity of our findings
among women (r = .13, p = .03) but not men (r = −.15, r). (Dimitrov, 2010). In general, the results of this study provided
further support for good construct validity, internal reliability,
Testing a Two-Dimensional Model. In a similar manner to and criterion validity of HFERST.
Study 2, we conducted a CFA to investigate a model in
which ER strategies were categorized into two dimensions
of functional and dysfunctional ER strategies. We tested
Study 4
whether such a two-dimensional measurement model would This fourth study was conducted in order to assess test–
exhibit an acceptable model fit to the HFERST items. Simi- retest reliability of the HFERST.
lar to our findings in Study 2, results showed an inacceptable
fit for this model, χ2(349) = 3199.15, χ2/df = 9.17, RMSEA =
Method
.14, CFI = .45, suggesting that this categorization does not
justify the multifaceted structure of ER use as it is assessed This study sample consisted of 60 graduate students in the
in the HFERST. psychology department who took part in a psychology
class, each of whom completed a paper-and-pencil version
Gender Effects. Similar to Study 2, we conducted an addi- of the HFERST during one of their classes. Participants
tional analysis to investigate measurement invariance across completed the questionnaire at the beginning of their course
genders. Having separated the analysis by gender, findings session, following the lecturer’s request. There was no fur-
supported the weak/metric invariance was tenable as model ther compensation for their participation. Two weeks later,
fit did not decrease when item loadings were constrained to the students were asked to complete the HFERST for a sec-
equality across the genders, χ2(20) = 22.48, p > .05 (see ond time. Of the 60 original participants, 37 completed the
Dimitrov, 2010; Meredith, 1993). Similar to Study 2, the questionnaire the second time. Those who did not partici-
model fit showed a statistically significant deterioration fol- pate in the second assessment scored significantly higher on
lowing further constraints to the item intercepts to equality experience suppression t(58) = 2.23, p = .03, but they did
χ2(20) = 51.41, p < .001. When taken together, these results not significantly differ from the rest of the sample in any
suggest that the HFERST shows evidence for weak/metric other HFERST subscales. The sample was predominantly
but not strong measurement invariance across genders. female (86.48%) and all participants were German.
Table 7. Results of t Tests Analysis for Test–Retest Reliability of HFERST From Time 1 to Time 2 (Study 4).
Time 1 Time 2
Note. All effects were insignificant (p > .05). N = 37. HFERST = Heidelberg Form for Emotion Regulation Strategies.
results remained unchanged after controlling for the effect therefore included in the final analysis. This sample con-
of gender. Findings of this study support the stability of the sisted of 39 women and 13 men, aged from 17 to 69 years
HFERST at an interval of 2 weeks. (M = 39.14 years, SD = 13.50). The intervention consisted
of a systematic group therapy for ER training consisting of
eight sessions. The content of the sessions was focused on
Study 5 psychoeducation of ER (e.g., what are emotions, what kinds
We conducted this final study to investigate the utility of the of ER strategies exist and why are they required, different
HFERST in measuring treatment outcomes. For this aim, facets of emotional experience), specific ER strategies and
we applied the HFERST before and after an 8-week group skills (rumination, reappraisal, avoidance, problem solving,
therapy for ER training to test if the HFERST captures ER engaging in activities, suppression, distraction and accep-
changes following this intervention. tance), and their positive/negative consequences. Sessions
were also focused on increasing awareness toward the envi-
ronmental triggers of the emotions and the relation between
Method bodily gesture and feelings (Barnow & Reichenbacher,
Our study sample included 55 individuals, 51.9% of whom 2013). Each participant had the chance to practice the skills
agreed to take part in our pilot study after being informed both in the sessions and between the sessions through hand-
about it through the outpatient clinic to which they were outs and homework. Further details on the structure, aims,
referred for psychotherapy. The other participants were and method of the intervention are provided in Barnow,
informed about the study through public announcements in Reinelt, and Sauer (2016).
the newspaper. All participants were informed that they
would receive training, which had been developed to
Results
improve ER. Each participant paid a participation fee of
€40 without receiving any further compensation for taking We conducted a series of paired t-test analyses to compare
part in the program. Diagnostic information was obtained the HFERST scores measured before and after the interven-
through the Structured Clinical Interview for DSM-IV Axis tion. As shown in Table 8, our results revealed that partici-
I and II (SCID-I and SCID-II; Wittchen, Zaudig, & Fydrich, pants showed a significant increase in applying functional
1997) and was available for 51.9% of the participants, strategies of reappraisal and acceptance after the interven-
77.8% of whom were diagnosed with affective disorders or tion. However, they showed only a trend in applying more
anxiety disorders, and 22.2% of whom were diagnosed with problem solving (p = .09) and no significant change in
eating disorders, personality disorders or somatoform disor- social support. Regarding dysfunctional strategies, partici-
der. Furthermore, 69.2% of the sample were receiving indi- pants showed a strong decrease in rumination, and a mar-
vidual psychotherapy at the time of our assessment and ginal decrease in avoidance (p = .06). There was no
11.5% reported to have had a history of receiving psycho- significant decrease in experience/expressive suppression.
therapy in the past. Before starting the group intervention, These results remained unchanged after controlling for the
all participants completed a paper version of the HFERST effect of parallel psychotherapy. Furthermore, the conduct
along with several other questionnaires. After conducting of our analyses for men and women separately showed the
the intervention, they were investigated for a second time, same pattern for both groups with significant changes in
when they completed the HFERST again. At this point, 52 rumination, acceptance and reappraisal, and no significant
participants took part in this second measurement and were changes in other strategies for both groups.
Izadpanah et al. 17
Table 8. Results of t Test Analysis for Comparing Scores on the HFERST Scales Before and After Intervention (Study 5).
Pretest Posttest
also refers to an attempt to suppress the subjective experi- changes in ER after conducting an ER training course.
ence of emotions. In line with our expectations, our results Participants showed an increase in functional strategies, such
supported the importance of experience suppression consid- as reappraisal and acceptance, and a decrease in dysfunc-
ering its significant relationship to psychopathology symp- tional strategies, such as rumination and avoidance. In gen-
toms and various difficulties in ER. eral, it seems that dysfunctional strategies are more resistant
Furthermore, we designed an acceptance subscale for the to intervention, and they therefore might require more
purpose of encompassing both situational and emotional extended training programs. This can be explained by the
types of acceptance. The stronger correlation between psy- more habitual nature of dysfunctional strategies and a more
chopathology symptoms and the acceptance subscale of the conscious nature of functional strategies, which facilitates
HFERST in comparison with the acceptance subscale from change to the latter category through learning procedures.
CERQ in our study was possibly an outcome of taking this The significant reduction of rumination might be explained
broader approach and supports the advantage of the by the availability of more functional strategies following the
HFERST over the CERQ, when applied within the context intervention that protected individuals from falling into a
of psychopathology. In general, when compared with the vicious circle of negative moods in response to negative
CERQ, HFERST subscales were more strongly related to experiences that in turn triggers greater rumination. An
depression, which suggests a better criterion validity of the extended repertoire of ER strategies enables individuals to
HFERST. Given that the formulation of items was based on apply other more conscious alternative strategies before
the awareness that items should not overlap with psychopa- engaging automatically in rumination. Furthermore, rumina-
thology symptoms, the content overlap with psychopathol- tion is associated with positive beliefs about the usefulness or
ogy cannot explain the stronger correlation between the even the necessity for rumination in solving current problems
HFERST and depression in comparison with CERQ. While and preventing future ones (Papageorgiou & Wells, 2001;
correlation coefficients between HFERT subscales and Weber & Exner, 2013). The intervention highlighted the
depression were not unexpectedly high, the correlation adverse effects of rumination and therefore might have con-
coefficients between CERQ subscales and depression were tributed to less rumination by correcting these beliefs. These
unexpectedly low. At the same time, similar subscales in findings show that the HFERST can capture changes in ER
both HFERST and CERQ showed a stronger correlation after an intervention, as well as showing sufficient 2-week
with each other than they did with depression (e.g., accep- stability when there is no intervention as found in Study 4.
tance, rumination, reappraisal, problem solving, and refo- Several limitations of this study should be noted. First,
cus on planning), which supports the construct validity of the samples included were largely convenience samples
the HFERST. An exception to this pattern was that the from online studies (Studies 1, 2, and 3), or samples from a
HFERST-acceptance subscale was more strongly correlated student population (Study 4). Furthermore, although stu-
with depression than it was with CERQ-acceptance. dents in Study 4 were not aware of the aim of the study,
However, previous findings show that the strategy accep- their knowledge of psychology might have resulted in a bias
tance from CERQ is often insignificantly or weakly corre- toward good test–retest reliability. In addition, our samples
lated with depression (e.g., Jermann et al., 2006; Öngen, were predominantly female and a small percentage of par-
2010), while based on the theoretical background, a nega- ticipants in samples from Study 1 and Study 2 (10% and
tive relation is expected (Liverant, Brown, Barlow, & 5.6%, respectively) were in their late adolescence. We
Roemer, 2008). On the basis of our results, the acceptance accounted for the gender effect by controlling it in all cor-
subscale of the CERQ even showed a small positive corre- relation analysis and by testing the model among both
lation with depression in Study 1, which was similar to find- women and men. Nevertheless, having more female than
ings from previous researchers (e.g., Tuna & Bozo, 2012). male participants might have reduced the representative-
This is potentially related to the formulation of items in this ness of the samples. More than this, Study 4 suggests that
subscale, which seems to measure passive resignation (“I there are no overall changes in the HFERST subscales
think that I cannot change anything about it”), while accep- across time if no ER intervention is implemented. However,
tance is considered to be an adaptive strategy. Furthermore, it remains an open question whether we would observe no
this subscale seems partly to measure thoughts of accep- change in these subscales in a sample of participants with
tance (as that expressed in the statement: “I think I have to mental disorders who do not receive an intervention.
accept that this has happened”) rather than acceptance as an Second, although Study 5 provided tentative evidence
ER strategy (see Jermann et al., 2006). that an ER intervention can significantly influence both
Furthermore, our results supported good internal consis- functional and dysfunctional strategies, regression to the
tency of the HFERST subscales and provided a consistent mean due to the lack of a randomized control group needs
pattern of relationship between the HFERST subscales and to be considered as a limitation to the findings of this study.
various psychopathology symptoms in predicted directions. Third, although we discuss ER strategy use as a constituting
In addition, in Study 5, using HFERST, we found significant factor for mental disorders, this causal direction cannot be
Izadpanah et al. 19
investigated in the present study, given the cross-sectional questionnaire so far measures ER along a continuum of
nature of our data in Studies 1 to 3. Fifth, although validat- functionality since capturing this quality is limited by sev-
ing this scale with a predominantly German sample is eral factors. First, the functionality of ER strategies can be
advantageous to studies that are conducted in Germany, the moderated by contextual factors such as the characteristics
HFERST should be tested for applicability in other lan- of individuals or the situation in which ER strategies are
guages and other cultural groups. We are currently working deployed (D’Avanzato et al., 2013; Liverant et al., 2008).
on investigating the intercultural validity of the HFERST, Second, subjective instruments mainly rely on the perspec-
including validation of an English version of the question- tive of the individuals. Therefore, although a strategy might
naire. This will facilitate intercultural comparisons and con- be applied in the wrong context or to an extreme level, indi-
tribute to cross-cultural research (see Appendix B and C for viduals might still find it functional from their own perspec-
German and English versions of the HFERST). Finally, tive. Therefore, the use of ecological momentary assessment
given that our conceptual method did not only rely on for measurement of ER should be complementary to apply-
experts’ opinion but also on previous literature and scales ing ER questionnaires in further research as it enables
on ER, we decided that a conceptual strategy will be an eco- researchers to obtain context-related information. In order
nomic and reasonable solution for initial item development to capture a more comprehensive assessment of ER, we are
and refinement. However, this strategy might have resulted currently working on development of an other-rated version
in limitations such as retention of items that do not reflect of the HFERST, together with an online version of the
the full breadth of the constructs or a full range of difficulty HFERST that can be used as an application on a daily basis
and high correlations among scales. Regarding the first con- for ecological momentary assessments providing further
cern, according to Kline (1979), the mean interitem correla- information on the context in which ER is applied. This
tions of HFERST scales showed an ideal balance between multimethod assessment will allow researchers to compare
item specificity and scale homogeneity. However, other different assessment methods, to evaluate how the data col-
authors suggest that interitem correlations greater than .50 lected by these methods are correlated with one another,
might result in the attenuation paradox, that is, an increase and to investigate which method is more beneficial in pre-
in internal consistency at the expense of a reduction in dicting various mental health outcomes.
validity and breadth of the measured construct (Clark & Despite these limitations, the results of this study indi-
Watson, 1995). Notably, our findings concerning the crite- cate that the HFERST is a valuable addition to the existing
rion validity of the HFERST scales with psychopathology measures of ER. The HFERST incorporates important ER
and the low to moderate correlations among the HFERST strategies into one single questionnaire and shows good
subscales increase our confidence that these higher inter- psychometric characteristics. Furthermore, results of this
item correlations are not detrimental to the utility of the study have shown that the HFERST is a valid questionnaire
HFERST. Nevertheless, replication of our results in more for applying within the context of psychopathology and
heterogeneous samples, as well as analyses in an item psychotherapeutic research. In addition, including three
response theory framework (Edelen & Reeve, 2007) are scales of experience suppression, social support, and avoid-
important steps to further investigate if the HFERST can ance in the HFERST can contribute to a better understand-
capture the full breadth of ER strategy use. ing of these understudied ER strategies in future research.
Future research should investigate psychometric charac-
teristics of HFERST in clinical samples and should also
examine the utility of the HFERST in capturing individual Appendix A
change in ER during therapy. Since ER has proven to be a
protective and predictive factor for psychopathology An Overview of the Questionnaires That Were
(Potthoff et al., 2016; Sheppes, Suri, & Gross, 2015), future Reviewed During Item Development Process
research should compare the effectiveness of various ER and Details on the Similarity of Current
questionnaires in the prediction of long-term psychopathol- HFERST Items to Items on Those Reviewed
ogy symptoms in order to provide evidence for the unique
Questionnaires
power of each questionnaire. Furthermore, investigation of
the HFERST simultaneously in multiple groups of healthy Most of the included items are original (items with no fur-
participants and individuals with various psychopatholo- ther details) or have been significantly reformulated and
gies will provide an opportunity to answer the question to only represent a general similarity in the content. Following
what extent each strategy can be considered functional or questionnaires have been reviewed during item develop-
dysfunctional. For example, it remains an interesting ques- ment procedure:
tion whether some functional strategies (e.g., problem solv-
ing) can be dysfunctional in the context of specific disorders •• CERQ (Garnefski & Kraaij, 2007)
(e.g., obsessive compulsive disorder). Furthermore, no ER •• DERS (Gratz & Roemer, 2004)
20 Assessment 00(0)
•• Emotionserleben und Emotionsregulation (emo- •• Item 10 (general similarity): I could not tolerate
tional experience and emotional regulation; Benecke, unpleasant emotions (EPS No. 26).
Vogt, Bock, Koschier, & Peham, 2008) •• I was sure that I could tolerate intensive, unpleasant
•• Emotion Control Questionnaire (ECQ; Roger & feelings (SEK No. 18).
Najarian, 1989) •• Item 11
•• ERQ (Gross & John, 2003)
•• Emotional Processing Scale (EPS; Baker, Thomas, 4. Problem solving
Thomas, & Owens, 2007)
•• Selbsteinschätzung emotionaler Kompetenzen (Self- •• Item 12
report measure forassessment of emotion regulation •• Item 13 (general similarity): I think about how to
skills; SEK-27; Berking & Znoj, 2008) change the situation (CERQ No. 23).
•• EMOtion REGulation Questionnaire (EMOREG; •• Item 14 (general similarity): I think about how I can
Znoj & Lude, 2002) best cope with the situation (CERQ No. 14).
•• Young/Rygh Avoidance Inventory (YRAI-1; Young, •• Item 15 (general similarity): I am a person who
Rygh, Berbalk, & Grutschpalk, 2003) weights carefully when a decision has to be made
(EMOREG No. 5).
Items
5. Suppression of emotional expression
1. Rumination
•• Item 16 (general similarity): When someone upsets
•• Item 1 (general similarity): When I’m upset, I take time me, I try to hide my feelings (ECQ-2 No. 1).
to figure out what I’m really feeling (DERS No. 18). •• Item 17
•• I want to understand why I feel the way I do about •• Item 18 (general similarity): People find it difficult to
what I have experienced (CERQ No. 21). tell whether I am excited about something or not
•• Item 2 (general similarity): I am preoccupied with (ECQ-2 No. 6).
what I think and feel about what I have experienced •• Item 19 from ECQ-2 (general similarity): I usually
(CERQ No. 12). manage to remain outwardly calm, even though I
•• Item 3 (general similarity): I often find myself think- may be churned up inside (No. 50).
ing over and over about things that have made me
angry (ECQ No. 22). 6. Suppression of emotional experience
•• Item 4
•• Item 20
2. Reappraisal •• Item 21
•• Item 22
•• Item 5 (general similarity): I think that the situation •• Item 23
also has its positive sides (CERQ No. 15).
•• Item 6 (general similarity): When I’m faced with 7. Avoidance
stressful situations, I make myself think about it in a
way that helps me stay calm (ERQ No. 5). •• Item 24 (general similarity): I try not to put myself in
•• Item 7 (general similarity): When I want to feel more the situations that are difficult or unpleasant to me
positive emotions, I change the way I am thinking (YRAI No. 37).
about the situation (ERQ No. 7). •• Item 26 (general similarity): I am a person who
•• Item 8 (general similarity): When I want to feel more avoids stressful thoughts or anticipation of threaten-
positive/ less negative emotions, I change the way I ing events (EMOREG No. 2).
am thinking about the situation (ERQ No. 7 and 10). •• Item 27 (general similarity): I try not to put myself in
the situations that are difficult or unpleasant to me
3. Acceptance (EPS, No. 22).
•• Item 9 (integrates the content of items No. 11 and 20
from CERQ): I think that I have to accept the situa- 8. Social support
tion (No. 11).
•• I think that I cannot change anything about it (No. •• Item 29
20). •• Item 31
Appendix B
Heidelberg Form for Emotion Regulation Strategies—German Version.
21
Appendix C
22
Heidelberg Form for Emotion Regulation Strategies—English Translation.
About half
Please rate the extent to which the following statements applied to you within the past 4 weeks Never Occasionally the time Usually Always
Strategy 1: Rumination
1. When I have negative feelings, I often brood about why I am feeling this way.
2. After emotional experiences/situations, I think a lot about what I did and said, so as to understand my
feelings better.
3. I realize over and over again, that I have to think about something that made me angry or sad.
4. I remember past conflicts often and think about what I could have done differently.
Strategy 2: Reappraisal
5. When I feel bad, I try to see the positive aspects of a situation.
6. When I find myself in a stressful situation, I change my thoughts about the situation in such a manner
that I become calm.
7. When I want to feel better, I concentrate on the good aspects of a situation.
8. I change my feelings by thinking differently about my current situation.
Strategy 3: Acceptance
9. When I cannot change something, I accept the situation as it is.
10. I am able to tolerate and endure uncomfortable situations.
11. I am able to accept things as they are.
Strategy 4: Problem solving
12. When looking for solutions for a problem, I do not only rely on my instincts but try to think as
rationally as possible.
13. I think about possible solutions for how to change the situation.
14. When I am confronted with problems, I think very carefully about how I can deal best with the situation.
15. When I have to make a decision, I carefully weigh the different alternatives against each other.
Strategy 5: Suppression of emotional expression
16. When something makes me angry or sad, I try to hide my feelings from others.
17. I hide physical expressions of my feelings.
18. It is very hard for other people to tell how I am feeling at the moment.
19. Even when I am very agitated, I am able to keep a calm exterior.
Strategy 6: Suppression of emotional experience
20. I try not to allow for negative emotions to begin with.
21. I rarely let my emotions run high, but keep them down.
22. Whenever possible, I avoid realizing my feelings.
23. When I have strong emotions, I immediately push them aside.
Strategy 7: Avoidance
24. I prefer to avoid situations that could cause negative emotions in me.
25. I try to avoid thoughts about things that weigh me down.
26. Whenever possible, I take care that I am not confronted with uncomfortable situations.
Strategy 8: Social support
27. I like to share negative and positive feelings by calling or meeting up with others and talking about them.
28. I often talk about my emotions with my partner or my close friends.
Appendix D
Factor Loadings for All HFERST Items From Study 1
Table A1. Factor Structure of Heidelberg Form for Emotion Regulation Strategies in Study 1.
23
24
Appendix D (continued)
Note. Loadings >.30 are printed in bold. The items are originally in German as presented in Appendix A.
Izadpanah et al. 25
Personality and Individual Differences, 30, 1311-1327. Schumaker (Eds.), Advanced structural equation modeling:
doi:10.1016/S0191-8869(00)00113-6 Issues and techniques (pp. 315-354). Mahawah, NJ: Erlbaum.
Gratz, K. L., & Roemer, L. (2004). Multidimensional assess- Meredith, W. (1993). Measurement invariance, factor analy-
ment of emotion regulation and dysregulation: Development, sis and factorial invariance. Psychometrika, 58, 525-543.
factor structure, and initial validation of the Difficulties in doi:10.1007/bf02294825
Emotion Regulation Scale. Journal of Psychopathology Nolen-Hoeksema, S., & Aldao, A. (2011). Gender and age dif-
and Behavioral Assessment, 26(1), 41-54. doi:10.1023/ ferences in emotion regulation strategies and their relation-
b:joba.0000007455.08539.94 ship to depressive symptoms. Personality and Individual
Gross, J. J. (2002). Emotion regulation: Affective, cognitive, Differences, 51, 704-708. doi:10.1016/j.paid.2011.06.012
and social consequences. Psychophysiology, 39, 281-291. Öngen, D. E. (2010). Cognitive emotion regulation in the pre-
doi:10.1017/S0048577201393198 diction of depression and submissive behavior: Gender and
Gross, J. J. (2003). Individual differences in two emotion regu- grade level differences in Turkish adolescents. Procedia—
lation processes: Implications for affect, relationships, and Social and Behavioral Sciences, 9, 1516-1523. doi:10.1016/j.
well-being. Journal of Personality and Social Psychology, sbspro.2010.12.358
85, 348-362. doi:10.1037/0022-3514.85.2.348 Papageorgiou, C., & Wells, A. (2001). Positive beliefs about
Hayes, S. C., Luoma, J. B., Bond, F. W., Masuda, A., & Lillis, depressive rumination: Development and preliminary vali-
J. (2006). Acceptance and commitment therapy: Model, pro- dation of a self-report scale. Behavior Therapy, 32, 13-26.
cesses and outcomes. Behaviour Research and Therapy, 44, doi:10.1016/S0005-7894(01)80041-1
1-25. doi:10.1016/j.brat.2005.06.006 Penedo, F. J., & Dahn, J. R. (2005). Exercise and well-being: A
Haynes, S. N., Richard, D. C. S., & Kubany, E. S. (1995). Content review of mental and physical health benefits associated with
validity in psychological assessment: A functional approach physical activity. Current Opinion in Psychiatry, 18, 189-
to concepts and methods. Psychological Assessment, 7, 238- 193. doi:10.1097/00001504-200503000-00013
247. doi:10.1037/1040-3590.7.3.238 Potthoff, S., Garnefski, N., Miklósi, M., Ubbiali, A., Domínguez-
Izadpanah, S., Schumacher, M., Bahr, A., Stopsack, M., Grabe, Sánchez, F. J., Martins, E. C., . . .Kraaij, V. (2016). Cognitive
H. J., & Barnow, S. (2016). A 5-year longitudinal study of the emotion regulation and psychopathology across cultures:
adolescent reinforcement sensitivity as a risk factor for anxi- A comparison between six European countries. Personality
ety symptoms in adulthood: Investigating the indirect effect and Individual Differences, 98, 218-224. doi:10.1016/j.
of cognitive emotion regulation. Personality and Individual paid.2016.04.022
Differences, 95, 68-73. doi:10.1016/j.paid.2016.02.021 Quoidbach, J., Mikolajczak, M., & Gross, J. J. (2015). Positive inter-
Jermann, F., Linden, M. V. d., d’Acremont, M., & Zermatten, ventions: An emotion regulation perspective. Psychological
A. (2006). Cognitive Emotion Regulation Questionnaire Bulletin, 141, 655-693. doi:10.1037/a0038648
(CERQ). European Journal of Psychological Assessment, 22, Reinecke, A., Filippini, N., Berna, C., Western, D. G., Hanson,
126-131. doi:10.1027/1015-5759.22.2.126 B., Cooper, M. J., . . .Harmer, C. J. (2015). Effective emo-
Kendall, M. G., & Stuart, A. (1958). The advanced theory of sta- tion regulation strategies improve fMRI and ECG markers of
tistics (Vol. 1). London, England: Charles Griffin. psychopathology in panic disorder: Implications for psycho-
Kline, P. (1979). Psychometrics and psychology. London, logical treatment action. Translational Psychiatry, 5, e673.
England: Academic Press. doi:10.1038/tp.2015.160
Leung, P. W. L., & Poon, M. W. L. (2001). Dysfunctional sche- Roger, D., & Najarian, B. (1989). The construction and validation
mas and cognitive distortions in psychopathology: A test of of a new scale for measuring emotion control. Personality and
the specificity hypothesis. Journal of Child Psychology and Individual Differences, 10, 845-853.
Psychiatry and Allied Disciplines, 42, 755-765. doi:10.1017/ Salters-Pedneault, K., Steenkamp, M., & Litz, B. T. (2010).
S0021963001007442 Suppression. In A. M. Kring & D. M. Sloan (Eds.),
Liverant, G. I., Brown, T. A., Barlow, D. H., & Roemer, L. (2008). Emotion regulation and psychopathology: A transdiagnos-
Emotion regulation in unipolar depression: The effects of tic approach to etiology (pp. 137-156). New York, NY:
acceptance and suppression of subjective emotional experi- Guilford Press.
ence on the intensity and duration of sadness and negative Schumacker, R. E., & Lomax, R. G. (2004). A beginner’s guide
affect. Behaviour Research and Therapy, 46, 1201-1209. to structural equation modeling (2nd ed.). Mahwah, NJ:
doi:10.1016/j.brat.2008.08.001 Lawrence Erlbaum.
Loch, N., Hiller, W., & Witthöft, M. (2011). Der Cognitive Sheppes, G., Suri, G., & Gross, J. J. (2015). Emotion regulation
Emotion Regulation Questionnaire (CERQ). Zeitschrift für and psychopathology. Annual Review of Clinical Psychology,
Klinische Psychologie und Psychotherapie, 40, 94-106. 11, 379-405. doi:10.1146/annurev-clinpsy-032814-112739
doi:10.1026/1616-3443/a000079 Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate sta-
Marroquin, B., & Nolen-Hoeksema, S. (2015). Emotion regula- tistics (5th ed.). Boston, MA: Allyn & Bacon.
tion and depressive symptoms: Close relationships as social Tuna, E., & Bozo, Ö. (2012). The Cognitive Emotion Regulation
context and influence. Journal of Personality and Social Questionnaire: Factor structure and psychometric properties
Psychology, 109, 836-855. doi:10.1037/pspi0000034 of the Turkish Version. Boston, MA: Springer.
Marsh, H. W., Balla, J. R., & Hau, K. T. (1996). An evaluation Weber, F., & Exner, C. (2013). Metacognitive beliefs and
of incremental fit indices: A clarification of mathemati- rumination: A longitudinal study. Cognitive Therapy and
cal and empirical properties. In G. A. Marcoulides & R. E. Research, 37, 1257-1261. doi:10.1007/s10608-013-9555-y
Izadpanah et al. 27
Wittchen, H.-U., Zaudig, M., & Fydrich, T. (1997). SKID others are diminished in remitted major depression. European
Strukturiertes Klinisches Interview für DSM-IV (Achse I und Psychiatry, 30, 448-453. doi:10.1016/j.eurpsy.2015.02.005
II)[SCID: Structured clinical interview for DSM-IV (Axis I Zlomke, K. R., & Hahn, K. S. (2010). Cognitive emotion regu-
and II)]. Göttingen, Germany: Hogrefe. lation strategies: Gender differences and associations to
Young, J., Rygh, J., Berbalk, H., & Grutschpalk, J. (2003). An worry. Personality and Individual Differences, 48, 408-413.
unpublished German translation of Young/Rygh Avoidance doi:10.1016/j.paid.2009.11.007
Inventory (YRAI-1). Unpublished manuscript. Znoj, H. J., & Lude, P. (2002). Regulation of emotion and psy-
Zahn, R., Lythe, K. E., Gethin, J. A., Green, S., Deakin, J. F. W., chological symptoms in people with spinal cord injury. Swiss
Workman, C., & Moll, J. (2015). Negative emotions towards Journal of Psychology, 61, 203-210.