See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/238677287

Teaching for social justice: Using vignettes to assess attitudes and beliefs

Article · January 2007

CITATIONS READS
4 865

5 authors, including:

Joan Barnatt Sarah Enterline

Elon University 11 PUBLICATIONS   225 CITATIONS   
13 PUBLICATIONS   643 CITATIONS   
SEE PROFILE
SEE PROFILE

Marilyn cochran-smith Larry Ludlow

Boston College Boston College
150 PUBLICATIONS   18,159 CITATIONS    136 PUBLICATIONS   5,215 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

BC-LAMP Portfolio View project

teacher testing View project

All content following this page was uploaded by Sarah Enterline on 12 February 2014.

The user has requested enhancement of the downloaded file.

 Barnatt, J., Shakman, K., Enterline, S., Cochran-Smith, M., Ludlow, L.

Teaching for social justice:

Using vignettes to assess attitudes and beliefs

Joan Barnatt, Karen Shakman, Sarah Enterline, Marilyn Cochran-Smith, & Larry Ludlow
Boston College

Paper presented at the American Educational Research Association Annual Meeting

April 2007

Page 1 of 7
 Barnatt, J., Shakman, K., Enterline, S., Cochran-Smith, M., Ludlow, L.

Teaching for social justice: using vignettes to assess attitudes and beliefs

Abstract
Despite rich conceptual work about teacher education and social justice, there are
currently very few ways to measure differences among teacher candidates or to document
change over time. This study reports on the development of a social justice vignettes
(SJV) assessment tool for use in pre-service teacher education. The tool contains five
scenarios that relate to social justice issues in schools and classrooms, which were
constructed and piloted with different groups of teachers/teacher candidates and in
different settings. Data from interviews, field notes, and questionnaires, and quantitative
analysis of response range, as well as factor analysis of response items, offer support for
the use of vignettes as a measurement tool. Social justice vignettes also offer promise as
a pre-service or professional development tool for addressing issues of social justice in
the classroom.

Objectives & Research Question

This study is part of a larger university-based initiative in teacher education
reform, Teachers for a New Era (TNE), funded primarily by the Carnegie Corporation,
which focuses on the outcomes of teacher education, including teaching for social justice.
At this university, researchers have taken a ‘portfolio’ perspective to the problem of
assessing the impact of teacher education, wherein a collection of complementary studies
gather evidence, from multiple perspectives, to explore the impact of teacher education
on teachers and their pupils, and the larger purposes of education.

This study addresses the question: Can vignettes be developed as a reliable and
valid measure of social justice attitudes, beliefs, and dispositions of teacher candidates/
teachers? This question is posed recognizing that, despite rich conceptual work on social
justice, there are few measurement tools for capturing aspects of teaching for social
justice, identifying differences in issues of social justice, and documenting change in
teacher candidates’ beliefs about social justice through the teacher education experience.

Theoretical Frameworks

A number of teacher education programs in the US and abroad include social

justice as one of their goals. (See programs and frameworks described in Cochran-Smith,
1991, 1999; Ladson-Billings, 1999, 2001; Quartz & TEP Research Group, 2003;
Zeichner, 1993; Zeichner & Liston, 1987.) Multicultural education, culturally relevant
pedagogy, and high expectations for all learners in the constructivist classroom are
central tenets of teaching for social justice (Banks, 1995; Ladson-Billings, 1995).
Another aspect, often overlooked in programs, is the role of activism and civic
participation (Greene, 1998 in Ayres; Kahne & Westheimer, 2003). Implicit, and
sometimes explicit, is acknowledgement of the political position of the teacher (Cochran-

Page 2 of 7
 Barnatt, J., Shakman, K., Enterline, S., Cochran-Smith, M., Ludlow, L.

Smith, 1998; El-Haj, 2003; Kohl, 1998). Experts also acknowledge the challenge of
working for equity in inequitable school contexts (El-Haj, 2003).

The vignettes, developed for this assessment, draw on this body of work,
especially Cochran-Smith’s (1998) “theory of teacher education for social change,” and
“principles of practice” in learning to teach for social justice (1999). Gauging social
justice beliefs poses particular difficulties in instrument development. Social justice
issues appear in extraordinarily complex situations that are compounded by interpersonal
relationships and varying school contexts. Teacher’s beliefs about social justice are
highly sensitive, reflecting moral values and professional images. In addition, differences
may exist between the beliefs expressed and actions taken. Finally, social justice has
different meanings to different individuals, creating a serious challenge of measurement
error when utilizing conventional survey methods.

Two formats, case studies and surveys, represent the current research formats
currently in use in the field to gauge understanding of teacher candidates’ beliefs
regarding social justice issues (Cochran-Smith, Davis, and Fries, 2004). The TNE
portfolio does approach social justice through these methods, but recognizes their
limitations. Case studies offer rich description and insight into particular contexts, but
limit generalizability due to small sample size, and cannot provide program-wide data or
comparison over time. Standard surveys offer potential for efficiently targeting greater
numbers of respondents, but are troubled by limitations of addressing the complexity of
real events and detailed insight into patterns among responses.

Recent research utilizing vignettes to measure instructional practices in

mathematics classrooms, conducted by the RAND Corporation (Stecher, et al., 2006)
offers cautious support for this format as a valid means of measuring behaviors that are
consistent with what is found in classroom observations, and on surveys and logs.
Additional evidence in favor of vignettes as a format for surveying complex and sensitive
social issues comes from the fields of psychology, sociology, medicine, and mental
health. There is evidence that vignettes offer a way of determining the cognitive
processes that are utilized in making decisions, and suggest which elements of a situation
are important in solving a dilemma (Morrison, Stettler, and Anderson, 2004).
Furthermore, vignettes have been shown to address complex issues effectively and
economically with a large number of respondents, thus offering the efficiency of
quantitative data with a richness that is closer to qualitative research (Finch, 1987). The
concrete and realistic nature of the vignette contextualizes the scenario so that it makes
the situation familiar and interesting to the respondent, prompting reflective responses
(Morrison, Stettler, and Anderson, 2004; Schoenberg and Ravdal, 2000). In ascribing the
situations to another person, vignettes are found to be less threatening to respondents as
they make judgments (Barter and Renold, 2000; Schoenberg and Ravdal, 1999). While
differences in understanding and defining social justice may remain, the vignette grounds
the respondent in a framework so that responses reflect the participant’s personal beliefs
and attitudes in response to complex, difficult, and realistic situations.

Methods, techniques and modes of inquiry in Vignette Development

Page 3 of 7
 Barnatt, J., Shakman, K., Enterline, S., Cochran-Smith, M., Ludlow, L.

The vignettes for the assessment tool described here were developed drawing on
formats found in the literature, including classic moral dilemmas by Kohlberg (1976),
and the subsequent development of the Defining Issues Test by Rest, et al. (1986), which
streamlined Kohlberg’s open-ended questions with multiple choice responses. The
following formats were also considered: the Social Work Values Inventory (Pike, 1996),
which provides brief scenarios, requiring respondents to indicate the action they would
take based on anchoring statements; and the Moral Justification Scale (Gump, et al.,
2000), with extended vignettes and embedded statements that reflect caring and/or justice
responses. Respondents are asked to rate the statements, pulled directly from the vignette,
on a scale of 1 to 10, basing judgments on the importance of the statement in making a
decision about the dilemma.

Drawing from these resources, five social justice vignettes were developed, based
on the social justice theoretical frameworks above. Respondents were asked to read each
dilemma, respond to a series of statements by rating them on a five-point Likert scale,
and indicate actions they would be likely to take in response to the scenario. Finally,
respondents were asked to explain, in an open-ended response, why they would take
these actions. Four vignettes were short written dilemmas reminiscent of Kohlberg’s
work (1976), and a fifth followed the longer rendition of Gump, et al. (2000).

Each vignette is drawn from real-world examples in the classroom, either from
the researchers own background or from qualitative case studies of learning to teach,
which are part of the TNE portfolio mentioned above. For example, one of the vignettes,
“Speak English, Please!” takes place in a high school classroom in which students are
reading a novel by a Mexican American author. During the discussion of the text, two
Latina girls begin speaking excitedly to one another in Spanish, and an African-American
girl responds, “You’re in America now. Speak English, please!” The questions that
follow this scenario ask respondents to consider implications of Spanish language use in
the English classroom, the role of linguistic and cultural diversity in the classroom, and
the responsibility of the teacher in this situation. Other vignettes focus on topics such as
differentiated instruction and evaluation, racial diversity, and mandated curriculum.

Data sources and Analysis

In piloting and refining the vignettes, four different analyses were conducted.
Analysis 1 involved the use of a think-aloud protocol with two veteran teachers to
explore the authenticity of scenarios, the effectiveness of the response formats, and
clarity of information and directions. This initial analysis provided feedback for how
vignettes and questions should be edited, what additions should be made to action
statements, and which of the two different formats was more viable. Additional scale
adjustment was also indicated. In general, the think aloud protocol endorsed the
vignettes’ realism and indicated that they captured the tensions between personal beliefs
and realistic possibilities for action in the school setting.

Analysis 2 took place after the draft vignettes instrument was revised and used as
part of a professional development program for a middle school staff of 25. Participants
individually completed all vignettes, and then took part in self-selected small group

Page 4 of 7
 Barnatt, J., Shakman, K., Enterline, S., Cochran-Smith, M., Ludlow, L.

assessments of a single vignette. A protocol for review and discussion of the individual
vignettes in small groups was provided, and participants completed written comments.
Analysis of responses, field notes of discussion, and small group reports indicated further
revisions to the vignettes, and rating and action statements. Based on participant
feedback, the vignette with embedded statements was rewritten to conform to the format
used in the other vignettes. The Likert scale items were analyzed for range of responses
and for consistency on responses highlighting an individual principle. Again, the
response was affirming: the vignettes are realistic, reflect the complexity of teaching for
social justice, and initiate rich discussion and reflection.

Analysis 3 occurred after the vignettes were piloted in three university classrooms,
with a total of 59 respondents. Here, the vignettes were presented in a variety of ways to
determine whether the format impacted the responses. In one set of vignette
questionnaires, the action choices followed directly from the scenarios and preceded the
Likert scale questions, whereas in the other sets, the Likert scale items followed the
scenario. In another set of questionnaires, respondents were asked to describe actions
they would take in response to the scenarios, rather than offered suggestions. This
allowed us to check the actions we had generated, and make additions or modifications,
based on their responses. Analysis of these data indicated that fatigue may have had an
impact on responses, suggesting that the vignettes should be offered in a staggered
sequence across the sample. Open-ended action responses did not provide new actions,
and a range of responses was noted, as was consistency among items addressing the same
principle across vignettes.

Analysis 4 was similar to the second round in which the tool was used for
professional development. The revised vignettes were presented to university clinical
faculty who supervise student teachers as part of a two-day workshop. The vignettes
were used as a tool for generating discussion.

Based on the four analyses described above, a revised Social Justice Vignettes
instrument has been developed. This will be administered to approximately 200 entering
teacher candidates, at undergraduate and graduate levels, and will provide substantial
quantitative data. Analysis 5, which will be completed in December, will provide data to
determine reliability of the Likert scale items and construct validity through factor
analysis. Additional analysis will include exploring differences between and among
curriculum areas and grade levels represented in the population.

The intention is to use the SJV as an instrument to assess teacher candidates at the
beginning and end of the teacher education program. This instrument has the potential to
measure changes in the beliefs, dispositions and attitudes of teacher candidates over time.
These findings will also be analyzed in comparison to other elements of the portfolio of
TNE research, including twelve items on entering, exit, and one year out surveys being
conducted on all teacher candidates; qualitative responses on social justice from a study
following 22 teacher candidates through their master’s program and into the first year of
teaching; and a comparison study of our teacher candidates with novice teachers from
other programs.

Page 5 of 7
 Barnatt, J., Shakman, K., Enterline, S., Cochran-Smith, M., Ludlow, L.

Results and Educational Significance

Preliminary results indicate that the instrument is useful. That is, we have found
that a range of responses exists across participants, suggesting that differences in attitudes
and beliefs about social justice issues are reflected in the ratings. In addition, consistency
across individual participant ratings for responses intended to measure the same
principles was also demonstrated in the pilot study. These findings suggest that vignettes
may provide a useful tool that responds to the limitations presented by standard survey
methods, while maintaining efficiency in handling large participant numbers.

Reports by participants suggest that the vignettes genuinely reflect the complex
and tension-filled events that define social justice issues in the classroom. Additionally,
the vignette format is reported to serve as a potential means of gaining access to
respondents’ beliefs, attitudes, and dispositions in a non-threatening manner, encouraging
reflection on practice. As such, the vignettes offer promise both as a measurement tool
and as a pre-service or professional development tool for addressing issues of social
justice in the classroom.

This instrument may be relevant to other institutions as there is little empirical

evidence of the impact or effectiveness of preparing teachers to teach for social justice,
despite the fact that more and more teacher education programs highlight a commitment
to diversity and equity, and even explicitly to preparing teachers for social justice. In an
era of accountability for K-12 schools and teacher preparation institutions, teacher
educators have a responsibility to challenge what counts as success in a K-12 classroom,
and be able to measure that success. To remain true to the goal of promoting social
justice in and through teacher preparation, educational research must establish valid and
reliable ways to measure success in this area. This tool is a step in capturing this
educational outcome.

References
Banks, J.A. (1995). Multicultural education: Historical development, dimensions, and
practice. In J.A. Banks (Ed.), Handbook of research on multicultural education
(p. 3-19). New York: Macmillan.
Barter, C., & Renold, E. (1999). The use of vignettes in qualitative research. Social
Justice Research, 25, Department of Sociology, University of Surrey: UK.
Cochran-Smith, M. (1991). Learning to teach against the grain. Harvard Educational
Review, 61(3), 279-310.
Cochran-Smith, M. (1998). Teaching for social change: Toward a grounded theory of
teacher education. In A. Hargreaves, A. Lieberman, M. Fullan & D. Hopkins
(Eds.), The international handbook of educational change (pp. 916-951). The
Netherlands: Kluwer Academic Publications.
Cochran-Smith, M. (1999). Learning to teach for social justice. In G. Griffin (Ed.),
The education of teachers: Ninety-eighth yearbook of the National Society for the
Study of Education (pp. 114-144). Chicago: University of Chicago Press.
Cochran-Smith, M. (2004). Walking the Road: Race, Diversity and Social Justice in
Teacher Education. New York: Teacher College Press, pp. 102-119.

Page 6 of 7
 Barnatt, J., Shakman, K., Enterline, S., Cochran-Smith, M., Ludlow, L.

Cochran-Smith, M., Davis, D., & Fries, K. (2004). Multicultural teacher education:
Research, practice and policy. In J. Banks (Ed.), Handbook of research on multicultural
education (3rd ed., pp. 931-975). San Francisco: Jossey Bass
El-Haj, T.R.A.(2003). Practicing for equity from the standpoint of the particular:
Exploring the work of one urban teacher network. Teachers College Record. 105
(5), 817-845.
Finch, J. (1987). The vignette technique in survey research. Sociology. 21, 105-114.
Green, M. (1998). Teaching for Social Justice (Introduction), W. Ayers, J.A. Hunt, and
T. Quinn (Eds.), Teaching for Social Justice. New York: Teacher College Press,
pp. xxvii-xlvi.
Gump, L.S., Baker, R.C., Roll, S. (2000). The moral justification scale: Reliability and
validity of a new measure of care and justice orientations. Adolescence. 2000
Spring; 35 (137):
67-76.
Kahne, J. & Westheimer, J. (2003). Democracy and civic engagement - what schools
need to do. Phi Delta Kappan, 85(1) 34-46.
Kohl, H. (1998). The Discipline of Hope: Learning from a Lifetime of Teaching. Simon
& Schuster: New York.
Kohlberg, L. (1976). Moral stages and moralization: The cognitive-developmental
approach. In T. Lickona (ed.), Moral Development and Behavior: Theory,
Research and Social Issues. New York: Holt, Rinehart and Winston.
Ladson-Billings, G. (1995) Toward a Theory of Culturally Relevant Pedagogy,
American Educational Research Journal, Vol. 32 (3), pp. p. 465-491.
Ladson-Billings, G. (1999). Preparing teachers for diverse student populations: A
critical race theory perspective. In A. IIan-Nejad & D. Pearson (Eds.), Review of
Research in Education (24, 211-248). Washington DC: American Educational
Research Association.
Morrison, R.L., Stettler, K., & Anderson, A.E. (2004). Using vignettes in cognitive
research on establishment surveys. Journal of Official Statistics. 20 (2), 319-340.
Pike, C. K. (1996). Development and initial validation of the Social Work Values
Inventory. Research on Social Work Practice, 6, 337-352.
Quartz, K.H. & TEP Research Group (2003). “Too angry to leave:” Supporting new
teachers’ commitment to transform urban schools. Journal of Teacher Education.
54(2), 99-111.
Rest, J. R. (1986). Moral Development: Advances In Research and Theory. New York:
Praeger.
Schoenberg, N.E., Raveal, H. (2000). Using vignettes in awareness and attitudinal
research. Social Research Methodology, 3 (1), 63-74.
Stecher, B., Le, V., Hamilton, L., Ryan, G., Robyn, A., & Lockwood, J.R. (2006). Using
structured classroom vignettes to measure instructional practices in mathematics.
Educational Evaluation and Policy Analysis, 28(2), 101-130.
Zeichner, K.M.(1993). Educating teachers for cultural diversity. E. Lansing, MI:
Michigan State University, National Center for Research on Teacher Learning.
Zeichner, K., & Liston, D. (1987). Teaching student teachers to reflect. Harvard
Educational Review, 57, p. 1-22.

Page 7 of 7

View publication stats