Psychology in the Schools, Vol.

33(3), 1996
0 1996 John Wiley & Sons, Inc. CCC 0033-3085/96/030185-07

INTERNALIZING SYMPTOMS OF GIFTED AND NON-GIFTED

ELEMENTARY-AGE STUDENTS: A COMPARATIVE VALIDITY STUDY
USING THE INTERNALIZING SYMPTOMS SCALE FOR CHILDREN
KENNETH W. MERRELL
Utah State Universig

STEVEN J. GILL
Tacoma, WashingtonPublic Schools

HOLLY MCFARLAND AND TODD McFARLAND

Utah State Universig

As part of the validation research process for a new self-reportsocial-emotionaltest forchil-

dren, internalizing social-emotional symptoms (e.g., depression, anxiety, social withdraw-
al, somatic complaints, positive and negative affectivity) of a group of elementary-age gift-
ed students (n = 65) were contrasted with those of a carefully matched (by gender and age)
comparison group of non-gifted students (n = 65). Subjects completed the Internalizing
Symptoms Scale for Children (ISSC) (Merrell & Walters, 1996). a self-report measure of
internalizing symptoms, affect, and cognition. The gifted students reported significantly
fewer internalizing symptoms than did the comparison group. An analysis of critical items
separating the two groups indicated that the gifted students differed most substantially from
their non-gifted peers on ISSC items that relate to self-efficacy and perceived self-impor-
tance. Although these types of self-perceptions are considered to be a peripheral rather than
a central component of specific internalizing disorders, it is hypothesized that their positive
presence in children may act as a “buffering” factor, possibly insulating children from in-
sults to their social-emotional functioning that may lead to the development of internaliz-
ing forms of psychopathology. The results of this investigation are discussed in terms of
their relationship to conflicting previous research in this area, to future research needs in
the study of social-emotional symptoms and development of gifted children, and in terms
of the construct validity evidence for the ISSC. 0 1996 John Wiley & Sons, Inc.

During the past few decades, there has been a growing interest in the social and emotional
aspects of giftedness. This interest has centered on varied aspects of social-emotional adjust-
ment, ranging from social skills to depression and suicide. Recently, there have been numer-
ous studies focusing on gifted and accelerated students which have attempted to identify the
key associations between the social and emotional aspects of adjustment and giftedness (Bland,
Sowa, & Callahan, 1994; Freeman, 1994; Hayes & Sloat, 1990; Kline & Short, 1991; Sayler
& Brookshire, 1993; Sowa, McIntire, May, & Bland, 1994).
Much of the research in this area has indicated that gifted children are at heightened risk
for developing social-emotional problems. For example, Lajoie and Shore (1981) argued that
giftedness is positively correlated with suicide. Roedell (1 986) hypothesized that difficulties
in social and emotional adjustment are proportional to the degree of the child’s giftedness; in
other words, the more gifted the individual, the more likely it may be that social-emotional
problems will occur. Some studies have noted that giftedness tends to have a negative effect
on the development of social competence and emotional adjustment (Hayes & Sloat, 1989).A
study by Hayes & Sloat (1990) indicated that concern over gifted children committing suicide
is warranted and justified because they have a heightened sensitivity due to their giftedness,
and are often affected by problems that their non-gifted peers are not even aware of; they may
not have the ability or the maturity to cope with these problems, and are therefore at risk for

Correspondenceand requests for reprints should be sent to Kenneth W. Merrell, Departmentof Psychology,Utah
State University,Logan, UT 84322-1448.

185
186 Merrell, Gill, McFarland, and McFarland

depression and, ultimately, possible suicide attempts. Beer (1991) found higher levels of gen-
eral anxiety as well as test anxiety in gifted children. These findings may indicate a certain
sensitivity or vulnerability to depression and other internalizing disorders which have been de-
scribed by other researchers (Hayes & Sloat, 1989; Kline & Short, 1991). Some researchers
have reported that gifted children tend to be more socially introverted (Berndt, Kaiser, & Van
Aalst, 1982) and that they often prefer to be left alone or joined by a single friend rather than
participate in larger and more social groups of peers (Terman, 1925). Additionally, the possi-
ble heightened sensitivity gifted children may experience may leave them much more aware
of their peers’ reactions to errors in a social setting, which may increase anxiety and hinder so-
cial development (Hayes & Sloat, 1990).
Although much of the empirical evidence on social-emotional aspects of giftedness in
children focuses on the negative aspects of giftedness, there is also a growing body of evidence
to the contrary, indicating that gifted children may have advantages and strengths in their so-
cial-emotional adjustment as compared to their non-gifted peers. Sayler and Brookshire (1993)
reported gifted students as having a higher internal locus of control and stronger global self-
concept than non-gifted students. Other findings of the same study indicated that gifted stu-
dents were more likely to report that others see them as socially and academically competent,
relatively popular, and athletic. These findings may indicate that gifted students have a higher
social self-concept than their non-gifted peers. Merrell and Gill (1992) reported significantly
higher levels of teacher-rated social competence and lower levels of antisocial behavior among
gifted students as compared to a matched sample of non-gifted children. Other studies have in-
dicated that gifted students develop problem solving skills as well as social skills commensu-
rate with their accelerated academic development (Sowa, et al., 1994). Additionally, Bland, et
al. (1994) argued that gifted children share many characteristics common to resilient adults;
e.g., self-reliance, risk-taking, and high internal locus of control. Gifted children are often ver-
bally proficient, which may enable them to discuss their problems with adults in a mature man-
ner. This characteristic, in turn, may reduce feelings of isolation and anxiety. Ultimately, Bland
et al.3 study found that the problems and difficulties that arise in a gifted child’s life are trig-
gered by the same events and feelings as those of other non-gifted children and that giftedness
does not adversely impact a child. In fact, gifted children may be better able to cope with and
surmount problems common to all children because of their giftedness (Sowa et al.).
The conflicting research regarding giftedness as it relates to social and emotional adjust-
ment suggests that a general consensus has not been reached in this area. Each study provides
a valuable piece of information; however, further research on giftedness, its related social-
emotional characteristics, and the implications for gifted children is warranted. To this end,
the present investigation was initiated, with the intent of focusing specifically on interna-
lizing characteristics and symptoms of social-emotional adjustment, which include the symp-
toms of depression, anxiety, social withdrawal, somatic complaints, and general negative af-
fect (Reynolds, 1992). Thus, the purpose of this investigation was to examine the symptoms
of self-reported internalizing symptoms of mid-elementary-age gifted students in comparison
with those of a carefully matched control sample of non-gifted students. An additional prima-
ry purpose of this investigation was to examine the construct validity of a new self-report as-
sessment instrument for detecting internalizing symptoms in children, the ISSC, particularly
regarding its potential sensitivity to group differences.
METHOD
Subjects
Subjects included 130 students in grades 3-6, divided into two groups. The first group
consisted of 65 students (33 boys and 32 girls) who participated in special acceleration or en-
 Internalizing Symptoms of Gifted Students 187

richment programs because they had been identified as talented and gifted based on superior
performance on standardized intellectual ability and academic achievement tests. This group
of students is referred to hereafter as the gifted group. All of the subjects in the gifted group
were selected from a large urbanlsuburban school district in the Pacific Northwest based on
their similar identification and participation in the gifted program. The second group consist-
ed of 65 students (33 boys and 32 girls) who had not been identified as talented and gifted, did
not have identified disabilities or receive special education services, and were carefully selected
from the ISSC national standardization group data base using a randomized block procedure
(Campbell & Stanley, 1968) to match the gifted subjects based on the demographic variables
of gender and age. This group is referred to hereafter as the non-gifted group. The racialleth-
nic composition of both groups was essentially similar, with approximately 80% of subjects
White or Caucasian, and 20% of the subjects members of various ethnic minority groups.

Procedure
All subjects were administered ISSC. These administrations followed a common set of
directions, were group-based and took place in classroom settings. Appropriate informed con-
sent procedures were followed prior to the test administrations.

Instrument
The ISSC is a self-report measure for use in screening for internalizing symptoms of chil-
dren in grades 3 4 . The initial development of the ISSC was driven by the high degree of over-
lap among various internalizing symptoms and disorders (e.g., Brady & Kendall, 1992;
Reynolds, 1992), coupled with the fact that most of the self-report measures for internalizing
symptoms in children tend to be aimed at a specific syndrome (e.g., depression or anxiety)
rather than broadly aimed at the larger construct of interrelated internalizing symptoms (Mer-
rell, 1994). Thus, the purpose for which the ISSC was developed was to provide a compre-
hensive self-report instrument for intermediate elementary-age children to assess the symp-
toms of various internalizing symptoms (e.g., depression, anxiety, social withdrawal, somatic
complaints) in a combined manner.
The ISSC includes 48 items that reflect either the presence or absence of a wide variety
of internalizing symptoms. Children respond to each item by selecting one of the following
statements that most accurately reflect how true they consider the statement to be for them:
Never True, Hardly Ever True, Sometimes True, and Often True. Completion of the ISSC pro-
tocol takes most children an average of 15-18 minutes. About two-thirds of the items are word-
ed in a manner that suggest the presence of a particular internalizing characteristic, while about
one-third of the items are presented in a manner that indicates either the absence of a specific
internalizing symptom, or the presence of positive affect or cognitions incompatible with spe-
cific internalization symptoms. Item responses are keyed to a four point scale (ranging from 0
to 3), where higher values always indicate greater internalizing symptoms.
Extensive and methodologically rigorous procedures were utilized during the develop-
ment of the ISSC items, to ensure a strong representation of the domain of children’s interna-
lizing symptoms, and thus, content validity of the instrument. For example, item development
and selection procedures were based on current theories and empirical evidence regarding the
internalizing dimension of child psychopathology (e.g., Cicchetti & Toth, 1991; Quay, 1986;
Quay & LaGreca, 1986), involved exhaustive literature search procedures, and utilized a pan-
el of experts to determine appropriateness and content validity of items. Development proce-
dures and content validity evidence of the ISSC are described in more detail in the ISSC tech-
nical manual (Merrell & Walters, 1996) and in other sources (Merrell & Crowley, in press;
Merrell & Dobmeyer, in press; Walters & Merrell, 1995).
188 Merrell, Gill, McFarland, and McFarland

Factor analytic studies have indicated that the ISSC includes two factors (Merrell &
Crowley, in press). The first factor, Negative AffecttGeneraL Distress, contains items that indi-
cate the presence of specific internalizing symptoms or emotional distress. The second factor,
Positive AfSect, contains items that indicate the absence of internalizing symptoms/emotional
distress, or the presence of positive affect and cognitions incompatible with emotional distress.
These factor analytic findings are compatible with those of previous researchers (e.g., Clark,
Beck, & Stewart, 1990; Watson, Clark, & Carey, 1988) who have found that positive and neg-
ative expressions of affectivity are separate components that make unique contributions to the
development or prevention of internalizing disorders such as depression and anxiety; they are
not merely polar opposites along a continuum.
Based on analyses of the initial standardization sample of over 1,800 cases, the ISSC has
an internal consistency reliability (coefficient alpha) of 9 2 , with the two subscales having in-
ternal consistency reliabilities of .90 and .86, respectively. The ISSC has been shown to dis-
play sensitivity towards gender differences in internalizing symptoms (Merrell & Dobmeyer,
in press). Convergent construct validity of the ISSC as a measure of internalizing symptoms
has been demonstrated through strong correlations (.7 1 to 3 6 ) with the Internalizing broad
band score of the Youth Self-Report, and through moderate to strong correlations between the
ISSC and measures of specific internalizing syndromes, including the Children’s Depression
Inventory (CDI) (.60 to .76 correlations with CDI total score) and the Revised Children’s Man-
ifest Anxiety Scale (RCMAS) (.56 to .79 correlations with RCMAS total score) (Merrell &
Walters, 1996). Data collection for the ISSC national standardization and various other relia-
bility and validity studies for the measure are ongoing. Sample ISSC items are presented in
Table 1.

RESULTS
ISSC scores of the gifted and non-gifted students were compared and contrasted various
ways. To obtain an estimate of the statistical significance of ISSC score differences between
the two groups, mean total and factor scores were contrasted using I-tests for independent
means. To obtain an estimate of the practical significance of group differences on the ISSC
scores, standard Effect Size (ES)estimates were calculated. The ES is considered to be an in-
dex of the degree of overlap among differing score distributions. The standard ES calculation
procedure was utilized, wherein the difference between the mean ISSC total scores of the gift-
ed and non-gifted groups was divided by the harmonic standard deviation of the two groups,
and the resulting ES estimate is interpreted in terms of a small, medium, or large effect (Co-
hen, 1988).

Table 1
Sample ISSC Items, Listed b y Factor
Factor 1: Negative AffectlGeneral Distress Factor 2: Positive Affect

I am shy I feel cheerful

I worry about things I feel important
I have bad dreams I have lots of energy
I worry that I will hurt someone I do things as well as other kids
I have trouble sleeping I like the way I look
Lots of things scare me I do well in school
When there is a problem, it is my fault I feel happy
It is hard for me 10breathe I like myself
 Internalizing Symptoms of Gifted Students 189

The ISSC total scores of the gifted group ( M = 46.31, SD = 20.84) were significantly
lower (t(128) = 2.20, p = .03; ES = .47, medium effect) than those of the non-gifted group
( M = 53.34, SD = 15.1l), indicating the gifted students had significantly lower levels of self-
reported internalizing symptoms. On ISSC Factor 1, Negative AffectlGeneral Distress, the
scores of the gifted students ( M = 41.01, SD = 17.81) were lower than those of the non-gift-
ed students ( M = 44.70. SD = 13.28), although this difference did not reach statistical signif-
icance (t(128) = 1.34, p = .18) and the practical significance of the difference could be con-
sidered modest (ES = .28, small effect). On ISSC Factor 2, Positive Affect, the scores of the
gifted students ( M = 8.43, SD = 7.30) were lower than those of the non-gifted students ( M =
12.14, SD = 7.1 1) by a substantial and significant margin (t(128) = 2 . 9 3 , ~= .004; ES = .52,
medium effect).
To test the properties of the ISSC in terms of its ability to sort or classify the subjects cor-
rectly based on membership in the two groups, a two-group linear discriminant function analy-
sis was conducted using the 48 ISSC items as classification variables, and the group member-
ship of the subjects (gifted vs. non-gifted) as the grouping variable. The resulting discriminant
function was significant: Wilks’ Lambda F = .54, ~ ~ ( 4 = 8 6) 4 . 6 0 , ~= .05. The classification
procedure in the discriminant analysis resulted in 80.77% of the subjects being classified cor-
rectly into their respective groups based on self-reported internalizing symptoms, including
87.7% of the gifted students, and 73.8% of the non-gifted students. The resulting structure ma-
trix indicated that pooled-within-groups correlations between the discriminating variables (the
48 ISSC scores) and the canonical discriminant function (the separation of gifted and non-gift-
ed students) ranged from .l 1 to S7. Many of the 48 items were correlated quite weakly with
the discriminant function, but 5 items were identified that correlated with this function higher
than .25, and could thus be considered more important or “critical” items in separating the two
groups. A qualitative analysis of the content of these items (presented in Table 2) indicates a
strong degree of similarity or commonality among them: they all appear to relate to feelings
or cognitions regarding perceived self-efficacy.
DISCUSSION
To summarize the main results of this investigation, students in the gifted group reported
lower levels of internalizing symptoms on the ISSC, both in terms of the two subscales and
the total score. The practical significance of these group differences ranged from small on the
ISSC Negative Affect/General Distress subscale, to moderate or moderately large on the Pos-
itive Affect subscale and on the ISSC total score. These results support the notion that gifted
children as a group exhibit better social-emotional adjustment than their non-gifted peers

Table 2
Five Items from the Internalizing Symptoms Scale for
Children Considered Critical in the Separation of Gifted
and Non-Gifted Subjects, With Pooled- Within-Groups
Correlations from the Discriminant Function Analysis,
Listed in Order of Descending Power

I do things as well as other kids .52

I do well in school .42
It is hard for me to think .29
I feel important .27
I feel like I have made too many mistakes .26
190 Merrell, Gill, McFarland, and McFarland

(Bland, et al., 1994; Merrell & Gill, 1992; Sayler & Brookshire, 1993; Sowa, et al., 1994), at
least within the limits of the social-emotional construct that was investigated. This investiga-
tion did not yield any results indicative of heightened social-emotional risk for internalizing
problems in the gifted group. Because of the carefully controlled matching procedure used to
make the gifted and non-gifted groups as similar as possible on the demographic variables of
age and gender, there appear to be no reasonable competing explanations for the differences in
internalizing symptoms between the two groups other than their gifted and non-gifted status.
It is particularly interesting to note that the ISSC domain in which the differences between
gifted and non-gifted students was most pronounced was in subscale 2, Positive Affect. It ap-
pears that in addition to reporting generally lower levels of emotional distress and negative af-
fect than their non-gifted peers, gifted students are particularly likely to present stronger lev-
els of positive affect and self-perceptions. A qualitative analysis of the five “critical” items that
separated the gifted from non-gifted students most strongly, namely, I do things as well as oth-
er kids, I do well in school, I t is hard for me to think, I feel important, and I feel like I have
made too many mistakes, indicates that these items have certain similarities. First, none of these
items are central symptoms of internalizing disorders, such as sad affect, somatic complaints,
social withdrawal. or anxious cognitive/physiological states. Rather, these five items all appear
to be peripheral emotional or cognitive characteristics that tend to accompany internalizing
disorders when they are manifest in the negative manner. Second, these five items all appear
to be measures of self-efficacy or self-perception. Thus the gifted students showed a clear
propensity to report substantially stronger feelings and thoughts of self-efficacy and self-im-
portance. It is intriguing to speculate on the clinical implications of the types of self-reported
statements that most clearly separated the gifted and non-gifted students. Perhaps the positive
presence of these self-efficacy/self-importance items in many gifted children may act as a
buffer against the development of serious internalizing disorders such as depression and anx-
iety. The results of this investigation tend to support this assertion, but clearly, more support-
ing evidence in this regard is needed before such an assertion can be regarded as a common
principle.
Some limitations inherent in this investigation may prevent optimum generalizability of
the results, and should be considered. The present sample (65 in each group, 130 total), though
carefully selected and matched along important demographic variables, is modest in size. A
substantially larger nationwide sample that yielded similar results would warrant more confi-
dence in their generalizability. The fact that only one instrument, the ISSC, was used as a mea-
sure of internalizing symptoms, further limits the generalizability of these findings. Although
the ISSC appears to have much promise as a research (and ultimately, clinical) tool because of
its solid technical characteristics and sensitivity to group differences, more research is needed
before it should be considered validated for specific uses. A future investigation of this nature
that used multiple measures of internalizing social-emotional status would also increase the
level of confidence with which these results could be generalized.
In view of the present findings supporting the notion that gifted students have significantly
more positive social-emotional characteristics than their non-gifted peers in terms of internal-
izing symptoms, it is difficult to explain why several previous investigations have produced
the opposite finding (e.g., Berndt, et al., 1982; Hayes & Sloat, 1989, 1990; Kline & Short,
1991). Perhaps findings in many of the studies in this general area are sample-dependent. It
may also be true that studies aimed at investigating specific internalizing syndromes or disor-
ders may produce somewhat differing results than studies such as the present one, where the
general construct of internalizing symptoms was targeted. To answer questions in this domain
more completely, future research regarding social-emotional adjustment of gifted children may
 Internalizing Symptoms of Gifted Students 191

ultimately need to explore specific developmental pathways leading to or away from social-
emotional problems, and how the characteristics of giftedness may impact these pathways.
REFERENCES
BEER,J. (1991). Depression, general anxiety, test anxiety, and rigidity of gifted junior high and high school chil-
dren. Psychological Reports, 69, 1128-1130.
BERNDT,D. J., KAISER, C. F., &VANAALST,F. (1982). Depression and self actualization in gifted adolescents. Jour-
nal of Clinical Psychology, 38, 142-150.
BLAND, L. C., SOWA,C. J., & CALLAHAN, C. M. (1994). An overview of resilience in gifted children. Roeper Re-
view, 17, 77-80.
BRADY, E. U., & KENDALL, P. C. (1992). Comorbidity of anxiety and depression in children and adolescents. Psy-
chological Bulletin, l I I , 24&255.
CAMPBELL, D. T., & STANLEY, J. C. (1968). Experimental and quasi-experimental research design. Chicago:
Rand-McNall y.
CICCHETTI, D., & TOTH,S. L. (1991). A developmental perspective on internalizing and externalizing disorders. In
D. Cicchetti & S. L. Toth (Eds.), Internalizing and externalizing expressions of dysfunction Hillsdale, NJ: Erl-
baum.
CLARK, D., BECK,A,, & STEWART, B. (1990). Cognitive specificity and positive-negative affectivity: Complemen-
tary or contradictory views of anxiety and depression? Journal ofAbnorma1 Psychology, 99, 148-1 55.
COHEN,J. (1988). Statistical power analysisfor the behavioral sciences (6th ed.). Hillsdale, NJ: Erlbaum.
FREEMAN, J. (1994). Some emotional aspects of being gifted. Journal for the Education of the Gifted, 17, 180-197.
HAYES,M. L., & SLOAT,R. S. (1989). Gifted students at risk for suicide. Roeper Review,12, 102-106.
HAYES, M. L., & SLOAT,R. S. (1990). Suicide and the gifted adolescent. Journalfor the Education of the Gifted, 13,
229-244.
KLINE,B. E., & SHORT,E. B. (1991). Changes in resilience: Gifted adolescent boys. RoeperReview, 13. 18&187.
LAJOIE,S. P., & SHORE,B. M. (1981). Three myths: The overrepresentation of the gifted among dropouts, delin-
quents and suicides. Gifted Child Quarterly, 25, 138-142.
MERRELL, K. W. (1994). Assessment of behavioral, social, and emotional problems: Direct and objective methods
for use with children and adolescents. New York: Longman.
MERRELL, K. W., & CROWLEY, S. L. (in press). Development and factor structure of a self-report measure for as-
sessing internalizing symptoms of elementary-age children. Journal of Personality Assessment.
MERRELL, K. W., & DOBMEYER, A. C. (in press). An evaluation of gender differences in self-reported internalizing
symptoms of elementary-age children. Journal of Psychoeducarional Assessment.
MERRELL, K.W., &GILL,S. J. (1992). Using teacher ratings of social behavior to differentiate gifted from non-gift-
ed students. Roeper Review, 16,286289.
MERRELL, K. W., & WALTERS, A. S. (1996). The Internalizing Symptoms Scale for Children: Users guide and tech-
nical manual. Logan, UT:Department of Psychology, Utah State University.
QUAY,H. C. (1986). Classification. In H. C. Quay & J. S. Weny (Eds.), Psychopathological disorders of childhood
(3rd ed.). New York: Wiley.
QUAY,H. C., & LAGRECA, A. M. (1986). Disorders of anxiety, withdrawal, and dysphoria. In H. C. Quay & J. S.
Weny, (Eds.). Psychopathological disorders of childhood (3rd ed.) New York: Wiley.
REYNOLDS, W. M. (1992). The study of internalizing disorders in children and adolescents. In W. M. Reynolds (Ed.),
Internalizing disorders in children and adolescents. New York: Wiley.
ROEDELL, W. C. (1986). Socioemotional vulnerabilities of young gifted children. Journal of Children in Conrem-
porary Society, 18, 17-29.
SAYLER, M. F., & BROOKSHIRE, W. K. (1993). Social, emotional, and behavioral adjustment of accelerated students,
students in gifted classes, and regular students in eighth grade. Gifed Child Quarterly, 37, 150-154.
SOWA,C. J., MCINTIRE,J., MAY,K. M., & BLAND,L. (1994). Social and emotional adjustment themes across gift-
ed children. Roeper Review, 17,95598.
TERMAN, L. M. (1925). Genetic studies of genius: Vol. 1: Mental and physical traits of a thousand gifted children.
Stanford, CA: Stanford University Press.
WALTERS, A. S., & MERRELL, K. W. (1995). Written versus oral administration of social-emotional self-report tests
for children: Does method of execution make a difference? Psychology in the Schools, 32 186-189.
WATSON,D., CLARK, L. A,, & CAREY, G. (1988). Positive and negative affectivity and their relation to anxiety and
depressive disorders. Journal of Abnormal Psychology, 97, 346-353.