CHECKLIST FOR EVALUATION OF RESEARCH

Author:

...................................................................................................................................................

[Last Name] [First Name] [Initial]

..................................................................................................................................................
[Title of research report/thesis/dissertation]
Appearing in:

................................................................................................................................................
[Journal (title) [Volume] [Month] [Year] [Pages]

Directions: Place a check mark in the appropriate column after carefully inspecting the research
report to see if it contains the item designated.

Factor Yes No Comment

1. Is the central problem for research clearly stated?
2. Are the subproblems clearly stated?
3. Does the research evidence plan and organization?
4. Has the researcher stated his hypotheses?
5. Are the hypotheses related to the principal problem or
the subproblems of the research?
6. Is the research methodology which has been employed
clearly stated?
7. Did d the researcher interpret the data, i.e., tell what the
facts mean?
8. Are the conclusions which the researcher presents
justified by the facts presented?
9. Is there any indication whether the hypotheses are
supported or rejected?
10.Is there any reference to or discussion of related literature
or studies by other researchers?
Number of tlly marks in each column (Enter these here)-->

Multiply the total of each column as follows ................ X 10 X0

Total Score (Column 1 only) .........................................

1
One Problem statement
1 Cite problem statement as stated in study.

Does it appear on the fist page? Yes (5) No (1)

2 Rewrite problem statement in your words.

Was the authors’ version clear enough to do this easily?

5 4 3 2 1 NA
3 Evaluate problem in terms of each of the criteria below. Give both a verbal description,
particularly of shortcomings, and numerical rating.

a. Workability

5 4 3 2 1 NA
b. Critical mass

5 4 3 2 1 NA
c. Interest**

d. Theoretical value

5 4 3 2 1 NA
e. Practical value

5 4 3 2 1 NA
*In the case of typed manuscripts (in contrast to printed ones), the placement of the problem statement is acceptable if it appeals
somewhere within the first two pages.
** Not rated because interest reflects personal values.

2
Two Variables
1 What is (are) the independent variable(s)?

2 What is (are) the moderator variable(s), if any?

3 What is (are) the dependent variable(s)?

4 What are the important control variables?

5 What are two possible intervening variables?

6 Are there any important variables that might have been included but were not? If so, list
them and label them as to type.

3
Three Hpotheses
1 Were hypotheses cited in the study?
If yes, cite them. Yes (5) No (1)

2 Rewrite hypotheses in your own words.

Was the authors’ version clear enough to do this easily? 5 4 3 2 1 NA

3 For each hypothesis answer the following:

a. Is it specific-directional (5), general-directional (4), or null (3)*.

b. Is it supported by sufficient literature or argument? (If 3 or 5 4 3 2 1 NA

lower, offer specific critism)

c. Is it clear and understandable? (If you rated 3 or lower, 5 4 3 2 1 NA

indicate how you think it might have been improved).

d. Is it consistent with the methodology and design? (If you rated 5 4 3 2 1 NA

3 or lower, indicate in what ways they are inconsistent).

 A specific directional hypothesis calls for differences and specifies their directions: a general-directional hypothesis calls for
differences but does not specify their direction: a null hypothesis call for no differences.

4
Four Supportive material
1 What, in a sentence, is the context of the problem?

Is the context clear and sufficiently established? 5 4 3 2 1 NA

2 a. What is the magnitude of the literature review? 5 4 3 2 1 NA

b. How empirical and up-to-date is it? 5 4 3 2 1 NA

c. Does it bear on the variables and the hypotheses? 5 4 3 2 1 NA

d. Is it well organized (introduction, subheadings, summary)? 5 4 3 2 1 NA

e. Try to summarize it in a centence or two

3 a. Do the researchers attempt to establish the significance of the Yes (5) No (1)
study using argument and/or supportive literature?

b. Is the attempt convincing? 5 4 3 2 1 NA

c. Summarize it in a sentence or two.

d. Suggest a point of significance that might be added.

5
Five Operational definitions
1 How did the researchers operationalyze each of the independent variables?

Does each O D seem sufficiently operational and unique? 5 4 3 2 1 NA

(If 3 or lower, give specific criticism).

2 How did the researchers operationalize each of the moderator variables, if any?

Does each OD seem sufficiently operational and unique? 5 4 3 2 1 NA

(If 3 or lower, give specific criticism).

3 How did the researchers operationalize each of the dependent variables?

Does each OD seem sufficiently operational and unique? 5 4 3 2 1 NA

(If 3 or lower, give specific criticism).

4 How did the researchers operationalize each of the important control variables?

Does each OD seem sufficiently operational and unique? 5 4 3 2 1 NA

(If 3 or lower, give specific criticism).

*Where OD’s are laking try to write them in ourself based on the methods used.

6
Six Measurement and manipulation of independent and dependent variables
1 How each independent variable answer each of the following:
a. Was it measured or manipulated?

b. Describe briefly how this was done.

c. How suitable (or valid and reliable) can you tell the 5 4 3 2 1 NA
manipulation or measurement was, on the basis of both face
validity and documentation?
(If you rated 3 or lower, make a sugestion for improvement).

2 For each dependent variable answer each of the following:

a. Describe briefly how it was measured.

b. How valid and reliable was its measurement as documented? 5 4 3 2 1 NA

(If 3 or lower, give a specific criticism or suggestion for
improvement).

7
Seven Measurement and manipulation of moderator and control variables
1 For each moderator variable answer each of the following:
a. Was it measured or manipulated? (If no moderator variable was used, write none and
omit this section)

b. Describe briefly how this was done.

c. How suitable (or valid and reliable) can you tell the 5 4 3 2 1 NA
manipulation or measurement was, on the basis of validity
and documentation?
(If you rated 3 or lower, make a suggeston for improvement).

2 For each of the major control variables (up to three) answer each of the following:
a. Was it measured or manipulated?

b. Describe briefly how this was done.

c. How suitable (or valid and reliable) can you tell the 5 4 3 2 1 NA
manipulation or measurement was, on the basis of both face
validity and documentation? (If 3 or lower, make a suggestion
for improvement).

8
Eight Research design
1 Was the disign a true experimental one, quasi experimental, ex post facto, or a non design?
Diagram the design used.

2 How did the researchers control for selection bias? Be specific.

3 How did the researchers control for history bias? Be specific.

4 Were there any other sources of bias particularly uncontrolled for?

If yes, describe the prosedure(s) or lack of any.

5 How internally valid was the design? 5 4 3 2 1 NA

(If 3 or lower, describe how it might have been improved)

6 Describe the population to which the results can be generalized.

7 How externally valid was the design? 5 4 3 2 1 NA

(If 3 or lower, suggest and improvement to increase)

9
Nine
1 Were statistical tests used? Yes No
If yes, name them.

2 Were the statistical tests used the proper ones or the most 5 4 3 2 1 NA
suitable ones to the design?
(If 3 or lower, suggest another).

3 Was their use clearly described? 5 4 3 2 1 NA

(If 3 or lower, describe the source of confusion).

4 Should the researchers have done additional statistical tests or Yes No

presented additional statistical results?
If yes, name them.

5 Was there a presentation of statistical results in graphic or tabular Yes No

form?
If yes, could it (they) be understood and interpreted? 5 4 3 2 1 NA
(If 3 or lower, describe the source of confusion and suggest how it
might have been overcome).

10
Ten Presentation and discussion of results
1 a. Briefly describe the findings of the study.

b. Does the article inculde an adequate and clear summary of the 5 4 3 2 1 NA

authors’ findings?
(If 3 or lower, offer a specific criticism).

c. Were the findings clearly related to the study’s problem and hypotheses?
(If 3 or lower, offer a critiscism).

2 a. Give a brief interpretation of the findings to indicate why they came out as they did or
what they mean.

b. Did the authors offer and support meaningful and convincing 5 4 3 2 1 NA

interpretation of the finding that helped you understand the
results?
(If 3 or lower, offer specific criticism or a useful
interpretation).

3 a. Briefly state a conclusion that could be drawn from the study and a recommendation
based on that conclusion.

b. Were meaningful conclusions and recommendations offered? 5 4 3 2 1 NA

(If 3 or lower, offer a suggestion).

11