M3 RESEARCH DESIGN

M3 Research Design Critique Report

Texas A&M University-Commerce
Cathleen Hernandez

M3 RESEARCH DESIGN

M3 Research Design Critique Report

Article Summary
Causal-Comparative
The purpose of the study conducted by Yvel Crevecoeur of The City College of
New York, Michael D. Coyne, and D. Betsy McCoach of The University of Connecticut
is to reanalyze data from a previous study which attempted to identify, to what degree,
the teaching and implementation of direct specific vocabulary instruction has an effect on
the reading abilities of ELL and EOL students within a 18-week kindergarten classroom
study. The reanalysis is including the language status of each student which the original
study did not include.
The research questions addressed were: What is the relationship between a direct
vocabulary intervention on that of treatment ELLs and treatment EOLs and whether there
are differential effects between each group? What is the relationship between the
language status of each student and the initial general receptive vocabulary knowledge on
the student response to intervention? (Crevecoeur, Coyne, McCoach, 2014). Since the
original study did not control for the language status (ELL/EOL) of the students, the
authors reevaluated the data to determine if language status affected the internal validity
of the study.

Summation of Research Literature

The research that the authors used as the basis for their specific study focused on
a reanalysis of a previous study (Coyne, McCoach, Loftus, Zipoli, Ruby, Crevecoeur, and
Kapp 2010) which the authors claim did not take into effect the English Language

M3 RESEARCH DESIGN

Learner status. The authors also included research conducted by Silverman (2007) which
focused on the vocabulary growth rates of 72 kindergarten ELL and EOL students. This
was a quasi-experimental study that did not include a control or comparison group. The
students were introduced to 50 target words over a 14-week period of time. The results of
this study showed that the vocabulary intervention was similarly a benefit to both the
ELLs and the EOLs.
Silverman and Hines (2009) was the second study reference used and its focus
was to analyze the results of utilizing multimedia and non-multimedia storybook readalouds with pre-kindergarten through second-grade ELLs and EOLs to see if their
vocabulary and reading levels improved. This study showed that ELLs benefited from
read-alouds though there was no significant effect on the EOL students.

Method for Collecting and Analyzing Data

In order to collect the data needed for the reanalysis study, researchers first asked
two specific questions of the original researchers. Question 1: What is the primary
language spoken in the students home? Question 2: What is students dominant language
in school? After these answers were documented, researchers verified language status
with school records, teachers, administrators, or other staff that had direct contact with
the students, students families, or students school records.
A pretest and a posttest were given to three groups of students in schools A, B,
and C. Groups in schools A and B were instructed by a kindergarten classroom teacher
and the group in school C was instructed by graduate student interventionists.
(Crevecoeur, Coyne, McCoach, 2014). The pretest and the posttest measured the level of

M3 RESEARCH DESIGN

vocabulary knowledge that each student possessed in regards to the 54 target vocabulary
words. Each student was to verbally define the word as well as explain the word being
used in context. Researchers included a listening comprehension assessment along with
the posttest to control for each students listening abilities. During the intervention, words
were introduced through the use of specifically chosen storybooks and several questions
were asked of the students in regards to each word to gauge understanding and usage
capabilities.

Results and Conclusions

The results of the study showed that the intervention of direct vocabulary
instruction is advantageous to all stakeholders and is strongly indicative of the need to
carry out similar programs in other elementary classrooms. ELL and EOL students within
the study had a similar response to intervention of direct vocabulary instruction, however,
in regards to general receptive knowledge of ELLs, EOLs had a significantly higher
general receptive vocabulary knowledge which is a important factor in regards to
response to intervention and affects the internal validity of the study. As well, controlling
for the category of ELL student will be necessary in future studies, such as if the ELL
student is a full bilingual student (learned both languages simultaneously) or a second
language learner (native language followed by English). Future studies on the
implementation of direct vocabulary instructional programs should take into account the
above factors in order to gain valid results that prove direct vocabulary instruction for
ELLs is of significant value to all involved.

M3 RESEARCH DESIGN

References
Coyne, M.D., McCoach, D.B., .Loftus, S., Zipoli, R.P., Ruby, M., Crevecoeur, Y., and
Kapp, S. 2010, Direct and extended vocabulary instruction in kindergarten:
Investigating transfer effects, Journal of Research on Educational Effectiveness,
3, 93-120
Crevecoeur, Y.C., Coyne, M.D., McCoach, D.B. (2014) English Language
Learners and English-Only Learners' Response to Direct Vocabulary Instruction,
Reading & Writing Quarterly, 30:1, 51-78, DOI: 10.1080/10573569.2013.758943
Silverman, R. D. (2007). Vocabulary development of English-language and English
only learners in kindergarten. The Elementary School Journal, 107, 365383.
Silverman, R., & Hines, S. (2009). The effects of multimedia-enhanced instruction
on the vocabulary of English-language learners and non-English-language
learners in pre-kindergarten through second grade. Journal of Educational
Psychology, 101, 305314.

M3 RESEARCH DESIGN

Point
s

EDAD 595 Module (3, 4, 5) Grading Rubric

CLAQWA (modified), Flateby & Metzger - University of South Florida
Assignment Requirements

Addresses and develops each aspect of the assignment and goes beyond the
assignment prompt to address additional related material.

Addresses each aspect of the assignment.

Addresses the appropriate topic and partially fulfills assignment requirements.

Addresses the appropriate topic but omits most or all of the assignment requirements.

Is off topic or vaguely addresses the topic.

Quality of Details

Provides details that help develop each element of the text and provides supporting
statements, evidence, or examples necessary to explain or persuade effectively.

Provides details that support the elements of the text with sufficient clarity, depth, and
accuracy.

Provides details that are related to the elements of the text but does not support those
elements with sufficient clarity, depth, or accuracy.

Provides details that are loosely related to the elements of the text but are lacking clarity,
depth, and accuracy.

Provides details that do not develop the elements of the text.

Quantity of Details

All points are supported by a sufficient number of details.

All points are developed, but some may need additional details.

Additional details are needed to develop some points.

Additional details are needed to develop most points.

Virtually, no details are present.

Grammar and Mechanics

Sentences are grammatically and mechanically correct.

Rare grammatical and mechanical errors exist but do not affect readability.

A limited variety of grammatical errors exist.

A variety of grammatical errors appear throughout the text, possibly affecting readability.

Most sentences exhibit multiple grammatical and mechanical errors, obstructing

meaning.

M3 RESEARCH DESIGN

7
Earned Point Total:

/20