Personality and Individual Differences 81 (2015) 195–200

Contents lists available at ScienceDirect

Personality and Individual Differences

journal homepage: www.elsevier.com/locate/paid

The Situational Test of Emotional Management – Brief (STEM-B):

Development and validation using item response theory and latent class
analysis q
Veleka Allen a,⇑, Nazia Rahman b, Alexander Weissman b, Carolyn MacCann c, Charles Lewis d,
Richard D. Roberts e
a
Otsuka Pharmaceutical, USA
b
Law School Admission Council, USA
c
The University of Sydney, School of Psychology, Australia
d
Fordham University, USA
e
Professional Examination Service, USA

a r t i c l e i n f o a b s t r a c t

Article history: This study uses a 3-Parameter Logistic item response theory (IRT) model to develop an 18-item short-
Received 27 April 2014 form of the Situational Test of Emotion Management (STEM). In a sample of 900 people, the short form
Received in revised form 24 January 2015 showed acceptable reliability (reliability index = .87; Cronbach’s alpha = .84) and a meaningful correla-
Accepted 30 January 2015
tion with another indicator of emotional intelligence (i.e., r = .30 with the Situational Test of Emotional
Available online 27 February 2015
Understanding). Latent class analysis of the short-form detected two classes. For all items, participants
in Class 2 had a higher probability of selecting the best option than Class 1. When response options were
Keywords:
coded to represent different emotion regulation strategies, Class 2 had a higher probability of endorsing
Emotional intelligence
Emotional regulation
‘‘situation modification’’ and Class 1 had a higher probability of endorsing ‘‘no regulation’’. These results
Situational judgment test provide validity evidence for the STEM-B as an assessment of emotion regulation.
Latent class analysis Ó 2015 Elsevier Ltd. All rights reserved.
Item response theory

1. Introduction
Emotional management is the fourth branch of the Mayer and
Salovey (1997) conceptual model of emotional intelligence (EI),
and involves the regulation of negative emotions and the enhance-
ment of positive ones (MacCann & Roberts, 2008; Mayer, Roberts, &
Barsade, 2008). There are two primary theoretical models of EI. The
first is an ability-based model, where EI is the capacity to process,
comprehend, and manipulate emotion-related information and is
usually assessed by items not unlike those found in traditional cog-
nitive tests. The second is a mixed-model approach, where EI is
considered a combination of character traits, motivation, and abil-
ity constructs and is usually assessed by self- or observer-report
rating scales (e.g., Mayer et al., 2008). The current research con-
cerns the ability-based conceptualization of emotion management.
We report the development of a new short-form of emotional man-
agement based on MacCann and Roberts (2008) Situational Test of
Emotional Management (STEM). A 3-Parameter Logistic (3-PL)
q nation); (d) cognitive change (modifying one’s evaluation of the
This article is a Special issue article – ‘‘Young researcher award 2014’’.
⇑ Corresponding author at: Otsuka Pharmaceutical, 506 Carnegie Dr., Princeton,
NJ 08550, USA.
E-mail address: vda0325@yahoo.com (V. Allen).
item response theory (IRT) analysis is used to select items for the
purpose of developing a brief instrument (i.e., the STEM-B). This
study also examines whether the STEM-B assesses emotion regula-
tion by comparing how frequently different latent classes endorse
response options representing different strategies for emotion
regulation.
1.1. Emotion management and emotion regulation
The emotion management branch of the Mayer–Salovey model
of EI is conceptually linked to Gross’ (1998) process model of emo-
tion regulation (e.g., Joseph & Newman, 2010). The process model
outlines several different techniques or strategies that can be used
to regulate emotions at different time points across the emotion
experience. These include: (a) situation selection (approaching or
avoiding particular people, places, or objects); (b) situation modifi-
cation (active efforts made to address the situation at hand; similar
to problem-solving or task-focused coping); (c) attentional deploy-
ment (a shift in focus through distraction, concentration, or rumi-
situation, including cognitive reframing and reappraisal); and (e)
response modulation (directly influencing the physiological impact
of emotions through the use of, for example, drugs, exercise, and

http://dx.doi.org/10.1016/j.paid.2015.01.053
0191-8869/Ó 2015 Elsevier Ltd. All rights reserved.
196 V. Allen et al. / Personality and Individual Differences 81 (2015) 195–200
relaxation techniques) (Gross, 1998). Individual differences in
emotion management may correspond to differences in the type
of emotion regulation strategies used. The current study tests this
conceptual link by classifying options on the STEM-B as represent-
ing one of these five emotion regulation strategies (or no regula-
tion at all). We then examine whether two latent classes
(determined from STEM-B responses) show different scores on
the STEM-B, and also show different rates of endorsing the various
types of emotion regulation strategies.
1.2. Measuring EI: the Situational Test of Emotion Management
(STEM)
Most of the research on ability-based EI uses the Mayer–Salo-
vey–Caruso Emotional Intelligence Test (MSCEIT; Mayer, Salovey,
& Caruso, 2002) or its predecessor, the Multifactor Emotional Intel-
ligence Scale (MEIS; Mayer, Caruso, & Salovey, 1999). MacCann and
Roberts (2008) developed alternative assessments of the higher
two branches of the four-branch model: (a) the Situational Test
of Emotional Understanding (STEU); and (b) the Situational Test
of Emotion Management (STEM).
The STEM consists of 44 multiple-choice items that are avail-
able in the American Psychological Association PsycTESTS database
and as Supplementary material to MacCann and Roberts (2008)
(see http://dx.doi.org/10.1037/a0012746.supp). Estimates of Cron-
bach’s alpha were .68 and .85 in a sample of 207 Australian under-
graduates and 850 Belgian medical students respectively
(Libbrecht & Lievens, 2012; MacCann & Roberts, 2008). The STEM
also shows some evidence of convergent and discriminant validity.
STEM scores correlate at .30 with MSCEIT management scores and
are also associated with other branches of EI such as emotion
understanding and emotion perception (Austin, 2010; Libbrecht
& Lievens, 2012; MacCann & Roberts, 2008).
1.3. Analysis of the STEM using item response theory
Typically, classical test theory approaches are used to assess the
psychometric properties of EI assessments. However, item respon-
se theory (IRT) has some advantages over classical test methods,
and emerging research has begun to use this measurement para-
digm in EI research (Allen et al., 2014; Anguiano-Carrasco,
MacCann, Geiger, Seybert, & Roberts, 2014). Recently, we success-
fully used IRT analysis to develop a short-form of emotional under-
standing (the STEU-B; Allen, Weissman, Hellwig, MacCann, &
Roberts, 2014), and the current study uses a similar approach to
developing the STEM-B.
Assumptions such as unidimensionality must be met when
employing most IRT models. Before undertaking IRT analysis, we
test the unidimensionality of the STEM by calculating the ratio of
the first to second eigenvalue in a one-factor exploratory factor
analysis (EFA) of STEM items (Morizot, Ainsworth, & Reise, 2007).
1.4. Analysis of the STEM using latent class analysis
Latent class modeling is another useful tool used to examine the
psychometric properties of assessments. Latent class models clas-
sify respondents into classes or categories based on their pattern
of responses and provide probabilities of class assignment for each
respondent (Skrondal & Rabe-Hesketh, 2004). Latent class analysis
can provide validity evidence in that if item responses are fit to a
two-class solution, the classes may show different types of pre-de-
fined responses. For example, one class may more frequently select
the best option. In the current study, we investigate whether dif-
ferent latent classes show: (a) high versus low scores on the
STEM-B; and (b) different endorsement rates for different types
of emotion regulation strategies.
1.5. Validity evidence for the short version of the STEM
To provide further validity evidence, we report correlations of
the STEM-B with another EI test (Situational Test of Emotion
Understanding, STEU). Emotion understanding and management
are conceptually related branches of EI in the four-branch model
of EI, together forming the strategic area of EI (Mayer et al.,
2008). A recent meta-analysis reported the correlation between
constructs to be .55 (Joseph & Newman, 2010). We thus expect sig-
nificant correlations between STEM and STEU scores. Furthermore,
we expect STEM and the STEM-B will show a similar magnitude of
relationship with the STEU, demonstrating that the short form has
similar validity evidence to the long form.
1.6. Aims and rationale of the present study
The primary goal of this study was to develop a short form of
the STEM using IRT analysis. We consider evidence for both the
reliability and validity of STEM-B scores. Validity evidence is based
on: (a) correlations with the STEU; (b) the use of latent classes to
validate the scoring key (i.e., predict high versus low scorers);
and (c) the use of latent classes to predict endorsement of qualita-
tively different forms of emotion management (e.g., situation
selection versus situation modification).
2. Method
2.1. Participants
Data was drawn from four samples where the 44-item STEM
was administered as part of the study protocol: (a) MacCann and
Roberts (2008) Study 1 (n = 112 Australian psychology under-
graduates; 77 female; mean age = 21.30 years; SD = 6.31); (b)
Aldao, Nolen-Hoeksema, and Schweizer (2010) (n = 152 Australian
psychology undergraduates; 89 female; mean age = 20.64 years,
SD = 4.25); (c) a sample of white-collar workers in the United
States (n = 507; 247 female; mean age = 36.40 years, SD = 13.27);
and (d) a sample of USA undergraduates (n = 129; 80 female; mean
age = 21.08 years; SD = 4.38). That is, the total sample size was 900
(55.9% female; mean age = 26.32 years, SD = 11.48), and was a mix
of university students and white collar workers from the United
States and Australia.
2.2. Measure
2.2.1. Situational Test of Emotional Management (STEM; MacCann &
Roberts, 2008)
The complete set of items comprising the STEM may be found in
the APA PSYCTests database at http://dx.doi.org/10.1037/
a0012746.supp and in Supplementary material for MacCann and
Roberts (2008). In all analyses in the current paper, item numbers
correspond to these documents. The STEM ordinarily uses partial
scoring, with scoring weights determined by the proportion of
experts who select each option as the best answer. In the current
study, dichotomous scoring is used in order to conduct 3-PL IRT
analyses, with the best option scored as ‘‘1’’ and the other options
as ‘‘0’’.
2.2.2. Data analysis
The unidimensionality assumption of the data was tested by
comparing the ratio of the first to second eigenvalue in a one-factor
EFA conducted in SPSS 18.0. Bilog-MG (Zimowski, Muraki, Mislevy,
& Bock, 1999) was used to fit a 3-PL IRT model to the scored data of
the STEM long form (44 items) in order to estimate item para-
meters. The item characteristic curves were then used to create a
 V. Allen et al. / Personality and Individual Differences 81 (2015) 195–200
short form by retaining items that provided the most information.
The 3-PL IRT model was fit again on the subset of responses to the
items that were retained on the short form. Latent class analysis of
raw (unscored) item responses was conducted on both the long
and newly created short forms using PROC LCA in SAS (freely esti-
mated parameters; maximum number of iterations = 5000; maxi-
mum absolute deviation convergence method; convergence
criterion = 0.000001). A two-class solution was fit to the data to
investigate if two classes would represent low-scoring and high-s-
coring individuals, and if there would be differences in emotion
regulation strategies selected by the groups.
3. Results
3.1. Descriptive statistics
The STEM mean proportion correct was 0.53 (SD = 0.16), and
Cronbach’s alpha = .83. The mean score for females (M = 0.55,
SD = 0.16, n = 493) was higher than that for males (M = 0.51,
SD = 0.15, n = 389). This difference was significant (t = 3.31;
p < 0.01), with a small effect size (Cohen’s d = 0.25).
3.2. Testing for unidimensionality
Unidimensionality was tested by comparing the ratio of first to
second eigenvalues obtained from an EFA of STEM responses.
Morizot et al. (2007) suggest that a ratio of above 3 between the
first and second eigenvalue indicates that an assessment measures
one dimension. The first five eigenvalues were 7.671, 1.625, 1.546,
1.470, and 1.304, respectively. The ratio between the first and sec-
ond eigenvalues was 4.72, indicating that the STEM can be consid-
ered as measuring one dimension, thus supporting the
unidimensionality assumption.
3.3. IRT analysis of the standard form of the STEM
The 3-PL IRT model (D = 1.7; ML estimation; Maximum EM
cycles = 25; convergence criterion = .001) was fit to item responses
on the long form. Five items had negative biserial correlations and
were subsequently omitted. The model was then fit to item
responses for the remaining 39 items and resulted in the inability
to calibrate item 24. The model was then fit to 38 items. The relia-
bility index (average standard error of latent trait scores) obtained
from IRT analysis was .91 and Cronbach’s alpha was .86 (based on
dichotomously scored items).
3.4. IRT analysis leading to a short form of the STEM
A STEM short form was developed using the following steps.
First, items where at least 75% of the experts endorsed a single
response were selected, leaving 22 items remaining. This was done
to ensure that there was little ambiguity in which response choice
was the best answer. These 22 items were fit to a 3-PL IRT model.
The aim was to produce a test information function for the short
form similar to that of the long form. Item parameters, item infor-
mation curves (amount of information an item contains along
the latent trait scale), and the test information curve (amount of
information the items collectively contain along the latent trait
scale) were reviewed. Items with low ‘‘maximum effectiveness’’
(maximum amount of item information <0.05) and providing infor-
mation for similar areas of the latent scale were omitted. The short
form (STEM-B) now included 18 items (reliability index = .87). The
item parameters are presented in Table 1. Figure 1 shows the test
information functions (TIF) for the short and long forms. The short
form TIF is lower as expected (it contains fewer items, capturing
197
less information). Although the STEM-B captures less information,
it is comparable to the STEM, especially at lower ability levels.
Cronbach’s alpha reliability for STEM-B was .84 (M = 0.59,
SD = 0.25). There were significant gender differences (males:
M = 0.56, SD = 0.24; females: M = 0.61, SD = 0.26; t = 2.897;
p < .01), with a small effect size (Cohen’s d = 0.20). These values
are all based on the dichotomously scored items.
3.5. Latent class analysis on the long and short forms of the STEM
Raw responses on the STEM-B (i.e., the observed response
options of A, B, C, or D) were fit to a two-class latent solution.
Results are presented in Table 2. For every item, people in Latent
Class 2 had a higher probability of endorsing the correct answer
than people in Latent Class 1. For example, on Item 5, Latent Class
2 had 0.60 probability of endorsing the correct response choice (C),
while Latent Class 1 had a probability of 0.13. Thus, Latent Class 1
represents lower scoring respondents (n = 218) and Latent Class 2
represents higher scoring respondents (n = 682).
To examine whether emotion regulation strategy choice dif-
fered across classes, each option for every STEM-B item was classi-
fied as one of: (a) no regulation, (b) situation selection, (c) situation
modification, (d) attentional deployment, (e) cognitive change, or
(f) response modulation. This classification was conducted inde-
pendently by two separate raters to assess reliability of coding.
Raters agreed 67% of the time, with a weighted Kappa of .664, indi-
cating an acceptable level of inter-rater consistency (Viera &
Garrett, 2005). A third rater classified item responses to resolve
the disagreements, and the reported classifications are from this
final reconciliation (see Table 3).
Table 3 shows the strategy represented by each option. For each
class, strategies with the highest probability of selection (most
popular choice) are shown in bold, and strategies with the lowest
probability of selection (least popular choice) are underlined. For
Class 2 (the high scorers), situation modification was the most pop-
ular choice for 14 of the 18 items, whereas no regulation, situation
selection, and attention deployment were not the most popular
choice for any item. In fact, situation selection was the least popu-
lar choice for 7 items. In contrast, for Class 1 (the low scorers)
situation modification was the most popular choice in only 3 items
(and the least popular choice for 7 items), no regulation as the
most popular choice for 6 items, and attention deployment was
the most popular choice for 3 items. In summary, high scorers fre-
quently endorse situation modification and do not endorse the ‘‘no
regulation’’ or ‘‘situation selection’’ strategies, whereas the reverse
pattern holds for low scorers. This finding provides some construct
validity evidence for the scoring key, as qualitative differences
between the two groups were identified.
Pearson correlations between a participant’s estimated prob-
ability of assignment to Latent Class 2 and their number-correct
scores (based on dichotomously scored responses) on the original
long form and STEM-B were conducted. The probability of assign-
ment to Latent Class 2 correlated highly with number-correct
scores on both the original long form and the STEM-B (r = .86 in
both instances). Number-correct scores from the long and short
STEM forms also correlated highly (r = .89; disattenuated
r = 1.00), as did the probability of assignment to Latent Class 2
for short and long forms (r = .98).
3.6. Correlation analyses between STEM and STEU
A subset of the sample (N = 559) completed both STEM and
STEU instruments. The STEU was significantly correlated with the
expert proportion scored STEM (r = .29; p < .01; disattenuated
r = .35) and STEM-B (r = .24; p < .01; disattenuated r = .29) as well
198 V. Allen et al. / Personality and Individual Differences 81 (2015) 195–200

Table 1
Item parameters for STEM-B.

Item Item parameter estimates (standard errors) Point of maximum item information on h scale
a b c
5 0.77 (0.08) 0.24 (0.09) 0.07 (0.03) 0.167
7 0.68 (0.07) 0.00 (0.10) 0.08 (0.03) 0.049
9 0.92 (0.08) 0.04 (0.08) 0.07 (0.03) 0.024
11 1.50 (0.13) 0.56 (0.06) 0.08 (0.03) 0.398
13 0.54 (0.07) 1.21 (0.30) 0.24 (0.09) 0.287
17 1.22 (0.14) 0.38 (0.10) 0.21 (0.05) 0.174
19 1.11 (0.09) 0.35 (0.07) 0.07 (0.03) 0.190
25 0.81 (0.07) 0.00 (0.08) 0.06 (0.02) 0.039
29 0.63 (0.09) 0.86 (0.12) 0.10 (0.03) 0.405
31 1.12 (0.21) 1.11 (0.10) 0.29 (0.03) 0.914
32 1.29 (0.14) 0.14 (0.08) 0.20 (0.04) 0.016
33 0.58 (0.06) 0.62 (0.11) 0.06 (0.03) 0.251
34 1.83 (0.15) 0.50 (0.05) 0.05 (0.02) 0.397
36 2.75 (0.28) 0.61 (0.04) 0.09 (0.02) 0.534
38 0.51 (0.08) 0.75 (0.18) 0.14 (0.05) 0.309
41 0.66 (0.11) 0.99 (0.14) 0.15 (0.04) 0.514
42 0.68 (0.11) 0.21 (0.25) 0.34 (0.07) 0.045
44 1.26 (0.12) 0.67 (0.09) 0.14 (0.04) 0.415

Notes: STEM-B = Situational Test of Emotional Management-Brief; a = discrimination parameter; b = difficulty parameter; c = guessing parameter. The item numbers given in
this table correspond directly to those given in Appendix B of MacCann and Roberts (2008); see http://dx.doi.org/10.1037/a0012746.supp).

as the dichotomously scored STEM (r = .28; p < .01; disattenuated

Test Informaon Funcons for Revised Long and r = .35) and STEM-B (r = .24; p < .01; disattenuated r = .30).
Short STEM forms
20 4. Discussion
18
16
The two broad goals of the research were met. First, an 18-item
14 Long Form
(STEM) version of the STEM was developed that had acceptable psycho-
12
Test Informaon

metric properties, and a much shorter testing time (i.e., under

10 10 min, compared to 20 min for the full STEM). Second, STEM-B
Short
8 Form scores predicted both STEU scores and latent class membership,
6 (STEM-B) supporting the scoring key, and also predicted different choices
4 of emotion regulation strategies. Taken together, these results sup-
2 port the reliability and construct validity of the STEM-B as an indi-
0 cator of emotion regulation.
-2 In terms of reliability, results suggest that the STEM-B is compa-
-5 -4 -3 -2 -1 0 1 2 3 4 5
rable to (and possibly better than) the STEM. Despite a reduction in
Theta
length of more than 50%, reliability of the STEM-B is almost as high
Fig. 1. STEM and STEM-B test information functions. as the STEM: Cronbach’s alpha increased from .83 to .84, and the
reliability index decreased slightly, from .91 to .87. This indicates
that the STEM-B is a more efficient assessment of emotional
Table 2
management.
STEM-B latent class assignment: probability of endorsing each item response by
latent class assignment. The STEM-B was significantly correlated with the STEU,
although the association was smaller than expected. However, a
Item Latent Class 1 Latent Class 2
lower correlation than reported in Joseph and Newman’s (2010)
A B C D A B C D meta-analysis might be expected, as their estimate was based on
5 0.42 0.39 0.13 0.06 0.01 0.24 0.60 0.15 multiple tests of emotion understanding and management (rather
7 0.15 0.20 0.61 0.04 0.67 0.31 0.02 0.00 than a single test of each, as in the current study). The strength of
9 0.13 0.67 0.14 0.06 0.01 0.29 0.69 0.02
association declines very little from long to short form, suggesting
11 0.10 0.18 0.64 0.08 0.08 0.89 0.02 0.01
13 0.10 0.10 0.54 0.26 0.01 0.07 0.87 0.05 that the STEM-B has similar evidence of validity to the STEM. Fur-
17 0.13 0.31 0.51 0.06 0.01 0.83 0.15 0.00 thermore, the latent class analysis results provided some internal
19 0.57 0.16 0.14 0.13 0.02 0.80 0.00 0.18 evidence of validity of the STEM-B. The two latent classes repre-
25 0.08 0.19 0.12 0.62 0.09 0.23 0.67 0.01 sented respondents scoring high versus low on the STEM-B. More-
29 0.13 0.16 0.51 0.19 0.47 0.03 0.28 0.21
31 0.30 0.18 0.44 0.07 0.48 0.07 0.30 0.15
over, the two latent classes showed very different rates of
32 0.23 0.13 0.31 0.33 0.78 0.02 0.13 0.07 endorsement of different emotion regulation strategies. In essence,
33 0.23 0.52 0.18 0.08 0.10 0.15 0.23 0.52 situation modification was the emotion regulation strategy of
34 0.54 0.13 0.24 0.09 0.08 0.02 0.00 0.90 choice for high scorers (with situation selection the least likely
36 0.26 0.21 0.41 0.13 0.00 0.04 0.00 0.96
strategy to be chosen), whereas low scorers were most likely to
38 0.13 0.59 0.22 0.06 0.38 0.09 0.52 0.01
41 0.06 0.57 0.17 0.20 0.02 0.00 0.52 0.46 endorse responses involving no attempts at emotion regulation.
42 0.05 0.47 0.41 0.08 0.01 0.79 0.18 0.03 The representation of STEM-B options as different types of
44 0.10 0.55 0.27 0.09 0.01 0.03 0.90 0.07 emotion regulation is important for two reasons. First, explaining
Notes: STEM-B = Situational Test of Emotional Management-Brief; A, B, C, D refer to
differences in emotion management in terms of regulation
the item response choices; Probabilities of endorsing A, B, C, D by latent class sum to strategies outlines the processes that may underlie emotion
approximately 1 due to rounding error; Bolded probabilities are the key. management. Much of the research on EI has focused on
 V. Allen et al. / Personality and Individual Differences 81 (2015) 195–200 199

Table 3
Emotion regulation classification of response choices for STEM-B.

Item Latent Class 1 Latent Class 2

A B C D A B C D
5 No reg Modify Modify Deploy No reg Modify Modify Deploy
7 Modify Modify No reg Select Modify Modify No reg Select
9 Select Deploy Modify Select Select Deploy Modify Select
11 Modify Modify No reg Modify Modify Modify No reg. Modify
13 No reg Modify Modify Change No reg Modify Modify Change
17 Change Modify Change Modify Change Modify Change Modify
19 No reg Modify No reg Modify No reg Modify No reg Modify
25 Change Modify Change No reg Change Modify Change No reg
29 Modify Select Change Modify Modify Select Change Modify
31 Modify Deploy Select Change Modify Deploy Select Change
32 Modify Select No reg Deploy Modify Select No reg Deploy
33 Select Deploy Modulate Change Select Deploy Modulate Change
34 Select No reg Select Modify Select No reg Select Modify
36 Select Deploy Change Modify Select Deploy Change Modify
38 Modify Change Modify Deploy Modify Change Modify Deploy
41 Select Modify Change Modify Select Modify Change Modify
42 Modify Modify Modify No reg Modify Modify Modify No reg
44 Select No reg Change Deploy Select No reg Change Deploy

# Highest probability # Lowest probability # Highest probability # Lowest probability

No reg 6 0 0 4
Select 2 4 0 7
Modify 3 8 14 4
Deploy 3 3 0 2
Change 4 3 4 1

Notes: No reg = no regulation strategy; select = situation selection; deploy = attentional deployment; change = cognitive change; modify = situation modification;
modulate = response modulation; options with the highest probability of endorsement are in bold; options with the lowest probability of endorsement are underlined.

descriptive or structural models rather than explanatory models
(i.e., a focus on describing what EI consists of and what it may pre-
dict, rather focusing on the processes and qualities that distinguish
high and low levels of EI). The current results suggest that indi-
vidual differences in emotion management are underpinned by
qualitatively different processes for emotion regulation. These
results are supported by a recent meta-analysis linking psy-
chopathology to differences in emotion regulation strategies. Psy-
chopathology was related to greater use of situation selection
(e.g., avoidance) and attentional deployment (e.g., rumination)
but less use of situation modification and cognitive change (Aldao
et al., 2010). It is now possible to understand why some people
are good at managing their emotions – they tend to use better
strategies. If the processes that underpin high versus low emotion
management can be understood, it will be much easier to design
theoretically driven programs to train emotion management (i.e.,
interventions that disrupt or change the processes underlying poor
emotion management). In the last few years there have been sever-
al research articles suggesting that it is possible to develop or influ-
ence EI (e.g., Nelis et al., 2011; Nelis, Quoidbach, Mikolajczak, &
Hansenne, 2009; Schutte & Malouff, 2012). The current research
suggests that interventions targeted toward teaching effective
regulation strategies (or discouraging the less effective strategies)
would be a fruitful way for intervention research to proceed.
Second, the coding of STEM-B options as representing different
emotion regulation strategies provides a useful resource for
analyzing the types of errors that people make. For example, one
test taker may consistently select options representing the ‘‘do
nothing’’ approach of no regulation whereas another test-taker
may consistently endorse situation selection (and hence may pos-
sibly point toward a tendency to avoid aversive situations). The
prospect of this type of qualitative feedback allows the STEM-B
to be used in training or coaching context to identify particular
tendencies underlying ineffective emotion management.
4.1. Limitations and future directions
One possible limitation with the current study is that STEM had
to be scored dichotomously to allow 3-PL IRT parameters to be
estimated. The final items selected were the ones where there
was the greatest degree of expert agreement. On one hand, this
may indicate that the STEM-B items have the most evidence of a
valid and reliable scoring key. On the other hand, this is an indica-
tion that easier items were selected (STEM-B score range, mean
score, and standard deviation were higher than STEM (R = 0–1.0;
M (SD) = 0.59 (.25) versus R = .14–.84; M (SD) = 0.53 (.16)), such
that ceiling effects may prove an issue in populations where rea-
sonably high levels of emotion management can be assumed. Addi-
tionally, STEM-B items were selected based on test information
curves. This resulted in a decline in measurement precision for
respondents with ability levels further away from the mean.
Another limitation is that the psychometric properties of the
STEM-B were not investigated on another independent sample.
Therefore, future research could test the psychometric properties
of the STEM-B from a different sample, and provide evidence that
its scores are associated with other measures of emotion manage-
ment (e.g., the MSCEIT Management scores) and predict emotion-
related outcomes. Moreover, the link between STEM-B scores and
the use of particular emotion regulation strategies could be
assessed using many different methods (e.g., experience sampling
as a way of detecting the regulation strategies that people use in
their real lives).
4.2. Conclusion
This study succeeded in its four aims: (1) the STEM demonstrat-
ed unidimensionality; (2) IRT was successfully used to develop a
short form (the STEM-B) with acceptable psychometric properties;
(3) the latent class analysis and coding of the STEM-B response
200 V. Allen et al. / Personality and Individual Differences 81 (2015) 195–200
options were successfully used in tandem to measure the construct
of emotion regulation; and (4) correlation analysis results of the
STEU with both the STEM and STEM-B provided external validity
evidence for the STEM-B. This paper provides two resources for
EI researchers and practitioners: first, a short form measure of
emotion management; and second, the codes for response options
as representing different processes of emotion management.
References
Aldao, A., Nolen-Hoeksema, S., & Schweizer, S. (2010). Emotion-regulation
strategies across psychopathology: A meta-analytic review. Clinical Psychology
Review, 30, 217–237.
Allen, V., Mason, A., & MacCann, C. (2014). Being willing and able to help: Evidence
for the role of emotional intelligence in the empathy/altruism hypothesis.
Submitted to Emotion.
Allen, V. D., Weissman, A., Hellwig, S., MacCann, C., & Roberts, R. D. (2014b).
Development of the situational test of emotional understanding – Brief (STEU-
B) using item response theory. Personality and Individual Differences, 65, 3–7.
Anguiano-Carrasco, C. MacCann, C., Geiger, M., Seybert, J. M., & Roberts, R. D. (2014).
Development of a forced-choice measure of typical-performance emotional
intelligence. Journal of Psychoeducational Assessment.
Austin, E. J. (2010). Measurement of ability emotional intelligence: Results for two
new tests. British Journal of Psychology, 101, 563–578.
Gross, J. J. (1998). The emerging field of emotion regulation: An integrative review.
Review of General Psychology, 2, 271–299.
Joseph, D. L., & Newman, D. A. (2010). Emotional intelligence: An integrative meta-
analysis and cascading model. Journal of Applied Psychology, 95, 54–78.
Libbrecht, N., & Lievens, F. (2012). Validity evidence for the situational judgment
test paradigm in emotional intelligence measurement. International Journal of
Psychology, 47, 438–447.
MacCann, C., & Roberts, R. D. (2008). New paradigms for assessing emotional
intelligence: Theory and data. Emotion, 8, 540–551.
Mayer, J. D., Caruso, D., & Salovey, P. (1999). Emotional intelligence meets
traditional standards for an intelligence. Intelligence, 27, 267–298.
Mayer, J. D., Roberts, R. D., & Barsade, S. G. (2008). Human abilities: Emotional
intelligence. Annual Review of Psychology, 59, 507–536.
Mayer, J. D., & Salovey, P. (1997). What is emotional intelligence? In P. Salovey & D.
Sluyter (Eds.), Emotional development and emotional intelligence: Implications for
educators (pp. 3–31). New York: Basic Books.
Mayer, J. D., Salovey, P., & Caruso, D. R. (2002). Mayer-Salovey-Caruso Emotional
Intelligence Test MSCEIT User’s Manual. North Tonawanda, New York: Multi-
Health Systems Inc.
Morizot, J., Ainsworth, A. T., & Reise, S. (2007). Toward modern psychometrics:
Application of item response theory models in personality research. In R. W.
Robins, R. C. Fraley, & R. F. Krueger (Eds.), Handbook of research methods in
personality psychology (pp. 407–423). New York: Guildford.
Nelis, D., Kotsou, I., Quoidbach, J., Hansenne, M., Weytens, F., Dupuis, P., et al.
(2011). Increasing emotional competence improves psychological and physical
well-being, social relationships, and employability. Emotion, 11, 354–366.
Nelis, D., Quoidbach, J., Mikolajczak, M., & Hansenne, M. (2009). Increasing
emotional intelligence: (How) is it possible? Personality and Individual
Differences, 47, 36–41.
Schutte, N. S., & Malouff, J. M. (2012). Priming ability emotional intelligence.
Intelligence, 40, 614–621.
Skrondal, A., & Rabe-Hesketh, S. (2004). Interdisciplinary statistics generalized latent
variable modeling: Multilevel, longitudinal, and structural equation models. Boca
Raton, Florida: Chapman & Hall/CRC.
Viera, A. J., & Garrett, J. M. (2005). Understanding interobserver agreement: The
kappa statistic. Family Medicine, 37(5), 360–363.
Zimowski, M. F., Muraki, E., Mislevy, R. J., & Bock, R. D. (1999). BILOG-MG: Multiple-
group IRT analysis and test maintenance for binary items. Chicago: Scientific
Software International.