The Relationship of Sex and Academic Performance

to Quality of
Recommendations for Graduate School
Jayne E. Stake, Elaine F. Walker, and Mary V. Speno
University of Missouri-St. Louis

To determine the relationship of sex and academic performance to

recommendations to graduate school, 41 0 letters of recommendation were
examined by two raters who were blind to the sex of the applicants and
referees. Letters were rated for eleven personal qualities and for indications
of weaknesses. Grade point average was found to be related to ratings of
intellectual potential ( p < .OOl), academic performance ( p < .OOl), re-
search ability ( p < .02), and motivation ( p < .01). The main effect of sex of
applicant was not significant for any variable; however, referees tended to
describe sarne-sex applicants as having more motivation and fewer weak-
nesses than opposite-sex applicants.

Women are underrepresented in colleges and universities in the

United States (Feldman, 1973) and in other countries (Rendel, 1975;
Zimmer, 1975). Some writers believe that this difference in participa-
tion is due, in part, to a bias against women students and faculty
(Blackstone & Fulton, 1975; Clark, 1977; Zimmer, 1975). The dif-
ference in participation rates does suggest such a bias; furthermore,
there is some direct evidence that women are less likely to be ac-
cepted to advanced programs (Heiss, 1970; Maxwell & Jones, 1976)
or to be received favorably by faculty members (Holmstrom &
Holmstrom, 1974).

~ ~ ~

Requests for reprints should be sent to Jayne E Stake, Psychology Department, Un\vers\ty
of Missouri-St. LOUIS, St. Louis, Missouri 6321 1.

Psychology of Women Quarterly, Vol. 5(4) Summer, 198 I 51 5

0361-684318111400-0515$00.95 @ 198 I Human Sciences Press
51 6

PSYCHOLOGY OF WOMEN QUARTERLY

Academic departments of psychology appear to be no exception

to this pattern of bias against women students and faculty. National
studies of psychology departments provide evidence of barriers to the
entrance and to the achievement of women students and faculty
(Brodsky, 1974; Fidell, 1970; O’Connell, 1977). Due to concerns
regarding the role of women in psychology, the APA Task Force on
the Status of Women in Psychology conducted a survey of 100 large,
APA-accredited departments (Astin, 1973). Within these departments
90 percent of the faculty members were men. Male faculty members
tended to hold higher rank, and, even within rank, they received
higher salaries. A bias toward men students was evident in graduate
program acceptance rates and in the distribution of fellowships and
assistantships.
The Task Force also found that over half of the graduate depart-
ments surveyed relied heavily on letters of recommendation in mak-
ing graduate admissions decisions. In light of the other evidence
gathered, the Task Force suggested that letters of recommendation
may contain sexist or discriminatory remarks about women. A survey
of letters written for students admitted to one psychology graduate
program provides some evidence that sexist comments do occur fre-
quently in letters of recommendation written for women applicants
(Lunneborg & Lillie, 1973). if, in fact, less favorable letters are written
for women, then letters of recommendation may be one means by
which sex biases limit the acceptance of women into graduate pro-
grams, and, thereby, perpetuate the lower status of women in psy-
c hoIogy .
That letters of recommendation written for women may contain
biased communications is consistent with some experimental social
research of judgments of women’s performance. Some studies have
reported that subjects asked to judge equal performances or qualifica-
tions of men and women have tended to give lower ratings to women
(e.g., Goldberg, 1968; Rosen & Jerdee, 1974). The presence and
direction of bias seems to be affected by the sex of the rater and the
ratee. Several studies suggest that raters tend to have a same-sex bias:
Females have sometimes rated the performance of women more fa-
vorably than the performance of men (Levenson, Burford, Bonno, &
Davis, 1975; Mackie, 1976; Muchinsky & Harris, 19771, and males
have sometimes rated the performance of men more favorably than
the performance of women (Etaugh & Rose, 1975; Taynor & Deaux,
1975). However, this pattern has not been consistent; male same-sex
favoritism and female same-sex favoritism has not been found to
occur together in any one of the studies cited above. Also, opposite-
 51 7

JAYNE E. STAKE, ELAINE F. WALKER, AND M A R Y V. SPENO

sex favoritism has occurred in a few studies (Bigoness, 1976; Etaugh

& Sanders, 1974). Hence, results from experimental studies do not
show a clear and consistent bias against women’s performance in all
situations or by all raters.
One variable that has been shown to have a consistent relation-
ship to ratings is ratee performance. When performance has been
systematically varied in studies of sex bias, strong performance effects
have been found (Bigoness, 1976; Muchinsky & Harris, 1977; Brief &
Wallace, 1976). These effects have been larger than the effects of
rater or ratee sex.
The Task Force suggested that letters of recommendation may
contain biases against women applicants to psychology graduate
programs. The purpose of this study was to examine objectively pos-
sible biases in letters of recommendation for male and female appli-
cants for graduate study in psychology. O n the basis of the evidence
presented above, it was predicted that applicants’ academic perfor-
mance would be highly related to referees’ ratings. It was also ex-
pected that the sex of referee and applicant would be related to
ratings. Due to inconsistencies in past research findings, the exact
nature of these relationships could not be predicted.

METHOD

Sample
Included in the study were letters of recommendation written for
233 men and 177 women who applied for admission to a midwestern
doctorate program in clinical psychology during the years 1972 -
1977. The letters were written by 184 women and 226 men referees.
The applicants were randomly selected from the group of all appli-
cants who had applied to the program during those years. One letter
of recommendation was chosen from each applicant’s file. Because
of the general scarcity of female referees, it was necessary to choose
letters written by females whenever possible. Choices were made on
a random basis when more than one letter by a female was available
and when all letters were written by males.
The mean age of the women applicants was 24.8 years and the
mean age of the men applicants was 25.5 years. The educational
background of the applicants represented a cross section of universi-
ties and colleges located in the Northeast (27%), South (1 5%), North
Central Region (8%), Midwest (42%), and West (8%).
518

PSYCHOLOGY OF WOMEN QUARTERLY

Procedures
A copy of each letter was independently judged by a male and a female rater
who were blind to the purposes of the study. Indications of sex of the
applicant and the referee were deleted from each letter before the raters
evaluated them; hence, the raters were unaware of the sexual identity of all
applicants and referees. Prior to evaluating the letters, the raters received
several hours of training to insure that they understood the content of the
letters and that they used consistent criteria in making judgments. During the
training the raters judged sample letters; when differences between raters
occurred, these differences were discussed and corrections in methods for
rating were made. At no time did raters view letters containing sex identity
information.
The authors determined from a sample of letters of recommendation
that indications of the following characteristics recurred in recommen-
dations for graduate school: interpersonal skills, self-confidence, inde-
pendence, leadership ability, intelligence, research ability, academic per-
formance, motivation, appearance, communicative ability, and ability to use
criticism constructively. The raters were instructed to consider each of these
traits as they judged each letter. When reference to a characteristic was
present in a letter, the raters assigned a value for that characteristic. The
values ranged from one to seven: a value of seven was given when descrip-
tions were most favorable and a value of one when descriptions were least
favorable. No value was assigned when no mention of a characteristic was
included in a letter. The mean ratings of the two judges were 4.94 and 4.98
and the standard deviations were 1.08 and 1.07. The judges were also
instructed to watch for references to applicants' weaknesses. When weak-
nesses were mentioned, they were assigned a value ranging from one (slight
weakness) to five (severe weakness). The interjudge reliability, based on all
pairs of ratings, was +.67.
Each applicant's file contained a grade point average (G.P.A.) based on
all undergraduate work. This G.P.A. was taken as a measure of academic
competence. Applicants with a G.P.A. of 3.3 or above were designated high
G.P.A. applicants; applicants with a G.P.A. of less than 3 . 3 were designated
low G.P.A. applicants. The variables of grade, sex of applicant, and sex of
referee formed a 2 x 2 x 2 factorial design.

RESULTS

Ratings were included in the analyses only when both judges

had determined that enough information was given to justify a rating.
In those instances in which one judge rated a characteristic and one
did not, the single rating was excluded from analysis. When both
judges had given an identical rating, that rating was used in the
 519

JAYNE E. STAKE, ELAINE F. WALKER, A N D M A R Y V. SPENO

analysis. When the judges had disagreed in their ratings, an average

of the two ratings was calculated.
Four of the characteristics considered, reaction to criticism, self-
confidence, leadership ability, and appearance, were rated by both
judges in only a small number of cases (4% or fewer). Apparently,
direct references to these characteristics do not recur often in letters
of recommendation for graduate school. The remaining characteris-
tics appeared in the letters in sufficient numbers to allow meaningful
comparisons, and the means for each of these characteristics is
shown in Table 1. An analysis of variance was performed for each of
these variables; all significant effects are described below.

Applicants' grades. Applicants' grades were related to ratings

of intellectual potential, F(1, 326) = 11.61, p < .001, academic
performance, F ( 1 , 290) = 12.57, p < ,001, research ability, F(1, 97)
= 5.92, p < .02, motivation, F(1, 305) = 8.01, p < .01, and com-
municative ability, F(1, 70) = 3.35, p < .08. As predicted, applicants
with stronger records of academic performance received higher
ratings. Mean ratings of applicants with high grades and applicants

.. ~~~~ - ... ~~~ ~ ~ _ _ _

High G P A. Low G P. A .
F c m i i l c Kctercc !+ale Referee Female Reieree Male R e f e r e e
~ - ~- .-
Appli~ant Applicant App 1i can t Applicant

Fcmalt Mali. Female Male Female Male Female Hale

~ -. ~

5.3H 5.35 5.32 5.45 5.16 5.10 5.08 4.94

(.82) (.76) (.78) (.74) (.64) i.80) (.9h) (.81)

5.72 5.h5 5.57 1.80 5.41 5.45 5.19 5.25

(.ill (.7L) (.78) (.If,) ( . h l ) (1.09) (1.03) (-88)

4.46 4.89 4.89 4.94 5.81 4.39 3.90 4.29

i.(>J) (.9f,) (.89) C.90) (1.04) (1.02) (1.43) (.83)

5.60 5.41 5.27 5.46 4.50 5.46 5.18 4.83

(.57) (.69) (1.03) (1.06) (.Ill L.84) (.78) (.73)

1.24 5.51) 5.25 5.66 5.62 5.25 5.10 5.01

f.h4) (67) f.66) (.69) (.h7) (.71) (.Xi) i.93)

4.75 4.811 4.72 4.67 4.97 4.74 h.88 4.h9

i.83) C.91) (.EL) C.73) (.81) (.82) (.76) (.hi)

-...-.. - - , ~ -._- . __
ier v . i l u e s i n d i r . a l c ' mtim posil . a t LrIE'.
520

PSYCHOLOGY OF WOMEN QUARTERLY

with low grades were, respectively, 5.37 and 5.05 for intellectual
potential, 5.68 and 5.31 for academic performance, 5.42 and 5.03 for
research ability, 5.47 and 5.20 for motivation, and 4.76 and 4.32 for
communicative abi Iity.

Sex of applicant and referee. No main effects were found for

sex of applicant. One main effect was found for sex of referee:
Motivation ratings obtained from female referees ( M = 5.45) were
higher than motivation ratings received by male referees ( M = 5.22),
f(1, 305) = 6.76, p < -01.

Interactions. Analysis of motivation ratings revealed two

significant second order interactions. The appl icant-sex by referee-
sex interaction indicated a tendency for referees to view same-sex
applicants as more highly motivated, F(1, 305) = 4.45, p < .05.
Ratings of letters by female writers were higher for female applicants
( M = 5.58) than for male applicants ( M = 5.35) whereas ratings of
letters by male writers were higher for male applicants ( M = 5.25)
than for female applicants ( M = 5.18). Also, an applicant-sex by
grade interaction revealed that males with high grades received
higher motivation ratings (A4 = 5.58) than males with low grades ( M
= 5.21), but that the ratings for high G.P.A. females ( M = 5.38) were
similar to the ratings of low G.P.A. females ( M = 5.331, F(1, 305) =
5.60, p < -02.
A three-way interaction was found for ratings of research ability,
F(1, 104) = 5.49, p < .03. As can be seen in Table 1 , this interaction
indicated a tendency toward higher ratings for same-sex high G.P.A.
applicants and for opposite-sex low G.P.A. applicants.

lndications of Weaknesses. In addition to ratings for specified

characteristics, judges also examined letters for instances of negative
comments. Only 38 letters (9.3%) contained such references, and this
number was too small to justify a three-way analysis of variance of
the values of the weakness ratings. However, it was possible to exam-
ine the frequency of occurrence of negative comments. Overall, no
difference occurred in the number of male and female applicants
receiving negative comments nor in the number of male and female
referees writing negative comments. However, within the group of 38
negative letters, a relationship was found between referee sex and
applicant sex: Referees were more likely to include indications of
weaknesses in letters written for opposite-sex applicants, x2(1) =
4.11, p < .05.
 52 1

JAYNE E. STAKE, ELAINE F. WALKER, A N D M A R Y V. SPENO

DISCUSSlON

The results of this study provide evidence that past academic

performance is closely related to quality of recommendations.G.P.A.
was positively related to ratings of intellectual potential, academic
performance, communicative ability, research ability, and motiva-
tion. In contrast, the main effect of sex of applicant was not significant
for any of the characteristics considered.
The Task Force on the Status of Women in Psychology was
concerned that letters of recommendation might reflect biases against
women applicants. Yet, the results of this study indicate that an appli-
cant’s previous academic work is far more important than the sex of
the applicant in determining the quality of letters of recommendation.
Thus, this study has provided some encouraging evidence of fairness
and objectivity in letters written for graduate school applicants.
Motivation ratings were related to grades for male applicants, but
not for female applicants, and motivation ratings associated with
female referees were higher. These findings are difficult to interpret.
Because these relationships did not occur for any other characteristic,
these findings may be due to chance factors.
However, interactions between referee sex and applicant sex
were more consistent. The ratings of letters indicated that referees
tended to describe same-sex applicants as more highly motivated and
to describe same-sex applicants with high grades as having somewhat
higher research ability. Also, indications of weaknesses occurred less
often in letters written for same-sex applicants.
These results are consistent with other findings of same-sex fa-
voritism cited earlier, since they suggest that referees tend to write
slightly more positive letters for same-sex applicants. If the number of
male and female referees were equal, these findings would not
suggest a particular disadvantage for female applicants. However,
most referees are college faculty members, and only a small percent-
age of college faculty members are women. Given that somewhat
more positive letters are written for same-sex applicants, this scarcity
of potential women referees may be a disadvantage to women appli-
cants.
What may account for the more positive letters written for
same-sex applicants? One possible explanation is that referees may
identify more with the same-sex applicant. For example, the female
referee may be more likely than the male referee to view other
women as truly motivated to enter a profession, since the female
referee is herself highly motivated in her career.
522

PSYCHOLOGY OF WOMEN QUARTERLY

Regardless of the reason for the relationship between sex of ref-

eree and sex of applicant, this study’s results suggest that referees
should take particular care to fairly evaluate opposite-sex applicants
when writing letters of recommendation,

REFERENCES
Astin, H. S. Report of the task force on the status of women in psychology. American Psycholo-
gist, 1973, 28, 61 1-616.
Bigoness, W. J. Effect of applicants’ sex, race, and performance on employers performance
ratings: Some additional findings. journal of Applied Psychology, 1976, 6 I , 80-84.
Blackstone, T., & Fulton, 0. Sex discrimination among university teachers; A British-American
comparison. British journal of Sociology, 1975, 26, 261 -275.
Brief, A. P., & Wallace, M. J. The impact of employee sex and performance on the allocation of
organizational rewards. journal of Psychology, 1976, 92, 25-34.
Brodsky, A. Women as graduate students. American Psychologist, 1974, 29, 523 -529.
Clark, L. Fact and fantasy: A recent profile of women in academia. Peabody journal of Educa-
tion, 1977, 54, 103 -1 09.
Etaugh, C., & Sanders, S. Evaluation of performance as a function of status and sex variables.
journal of Social Psychology, 1974, 94, 237-241.
Etaugh, C., & Rose, S. Adolescents’ sex bias in the evaluation of performance. Developmental
Psychology, 1975, I I , 663 -664.
Feldman, S. D. Escape from the doll’s house. St. Louis: McGraw-Hill, 1973.
Fidell, L. 5. Empirical verification of sex discrimination in hiring practices in psychology.
American Psychologist, 1970, 25, 1094-1 097.
Goldberg, P. Are women prejudiced against women? Transaction, 1968, 5, 28-30.
Heiss, A. M. Challenges to graduate schools. San Francisco: Jossey-Bass, Inc., 1970.
Holmstrom, E. I., & Holmstrom, R. The plight of the woman doctoral student. American
Educational Research journal, 1974, 1 I , 1 -1 7.
Levenson, H.; Burford, B.; Bonno, B. & Davis, L. Are women still prejudiced against women?A
replication and extension of Goldberg’s study. journal of Psychology, 1975, 89, 67-71.
Lunneborg, P. W., & Lillie, C. Sexism in graduate admissions. American Psychologist, 1973,
28, 187-189.
Mackie, M. Students’ perceptions of female professors. journal of Vocational Behavior, 1 976,
8, 337-348.
Maxwell, S. E., & Jones, L. V. Female and male admission to graduate school: An illustrative
inquiry. journal of Educational Statistics, 1976, 1 , 1-37.
Muchinsky, P. M., & Harris, S. L. The effect of applicant sex and scholastic standing on the
evaluation of job applicant resumes in sex-typed occupations. lournal of Vocational 5e-
havior, 1977, I I, 95-108.
O’Connell, A. N. Barriers to research in psychology. Report of the task force on women doing
research. Division 35 Newsletter, 1977, 4, 7-9.
Rendel, M. Men and women in higher education. Educational Review, 1975, 17, 192 -201.
Rosen, B., & Jerdee, T. H. Effects of applicant’s sex and difficulty of job on evaluations of
candidates for managerial positions. lournal of Applied Psychology, 1974, 59, 51 1 -512.
Taynor, I., & Deaux, K. Equity and perceived sex differences: Role behavior as defined by the
task, the mode, and the actor. journal of Personality and Social Psychology, 1975, 32,
381 -390.
Zimmer, T. A. Sexism in higher education. A cross-national analysis. Pacific Sociological
Review, 1975, 18, 55-67.