Reading Critically: Things to Notice and Questions to Ask
Reference:
The Five Basic Questions
Basic Question Component of Research Report Critical Evaluation (Questions to Ask)
What Is the Report Title.- clear, concise, accurate and
About? appealing.
Abstract.- contains a general
statement of the research topic, a brief
description of the study design and the
methods involved, and the most
important results obtained.
Purpose.- clear indication of exactly
why the study was performed.
How Does the Study Introduction, Research Purpose,
Fit Into What Already and Related Literature.- the
introduction provides context for the
is Known?
purpose of the study, a good
introduction not only accomplishes the
end of inserting the study into what is
already known, it also leads you
forward toward a discussion of
methods.
References.- learn if are old or recent,
type of journals published the reports
being cited, contribute to a judgment
about the quality of the study.
How Was the Study Method
Done? Subject.- when humans are studied,
individuals be described not as generic
“subjects” but as particular people –
adults, males, children, players, etc.
Use the term participant instead of
subject. How many were examined
also is a vital part of method. How
carefully the number of subjects has
been considered is of greater
importance than its precise size.
Research Plan or Design.- if the study
is quantitative, the primary variables
will be named and placed in their
respective positions. A rationale for
why that particular plan constitutes the
best choice from among alternatives.
Instrumentation.- artifacts ( tests,
measuring instruments, and protocols
for obtaining data) that were used in
the study. Values for reliability and
validity will be noted. Explain why each
instrument was selected and that is the
most accurate means of data collection
Action
Does the title indicate the important constructs and
relationships in the study?
Does the abstract provide enough information to make a
decision about reading the full report?
Does the abstract suggest the importance of the study?
Is the purpose of the study clearly stated, and it is framed within
introductory material that makes it easy to understand?
Having read the title, abstract, and statement of purpose, do
you find yourself interested in reading the next part of the
study?
Is the topic of the study introduced in terms of previous
investigations?
Did the author explain what is known and not known in
sufficient detail for you to understand how the study fits into the
present structure of knowledge?
Were decisions about the design and procedures of the study
explained in terms of what has been found effective or
ineffective in previous investigations?
If there have been conflicting findings in the relevant literature,
were they discussed and given consideration in the present
study?
Do the references include recent reports from investigators who
have established track records for research in the area?
Is there evidence that the subjects were treated with due
considerations for ethical standards?
Is there a clear explanation for why these particular subjects
were selected?
Are the selection criteria clearly specified?
Is the selection process thoroughly explained?
Has the author explained why the number of subjects is
adequate to the procedures and purposes of the study?
Is the quality of the subject sample high (do most of them
closely match the selection criteria)?
Does the author address any limitations produced by selection
procedures?
Did the author name the design and introduce the primary
variables and their relationships?
Was the general design of the study explained in clear terms?
Was a rationale given for the selection?
Was the subsection dealing with design placed where it was
helpful in understanding the remainder of the report?
Are all of the instruments used in the study named and
described?
In each case for which it is appropriate, are validity and
reliability values provided?
Have researcher-constructed instruments been pilot tested?
Has a rationale been given for selection when there were other
available options?
 How Was the Study Procedures.- explains how the
Done? instruments were used. It is not
sufficient to know that a questionnaire
(Continued)
was used, even if its validity and
reliability are given and a full copy of
the document provided in an appendix.
You need to know how it was used.
Analysis.- describes how the data were
organized and then subjected to
analysis.
What Was Found? Results
Description of the findings.- should
provide a reflection of the problems,
questions, or hypotheses laid out in the
opening section on the purpose(s) of
the study.
Figures.- use them when significant
improvements in clarity and economy
(by substantial reduction in text) can
be achieved. Used when it is important
to draw readers’ attention to general
comparisons and to interaction among
variables. Described by captions that
present the number of the figures as
cited in the main text. A caption must
allow the figure to stand alone.
Tables.- serve for condensing large
volumes of raw data into a economical
space and preserving the precise
characteristics of those data. Also use
them when it is more important to the
reader to know specific numeric values
rather than, for example, to know how
various values relate to each other or
to the whole. Their components are:
title, column headings, row headings,
data field, and, in some cases,
footnotes.
What Do the Results Discussion and Conclusions.- is the
Mean? point at which the author considers all that
has happened, decides what has been
learned, and communicates it to readers.
Researcher matches findings with the
original research purpose, indicates which
aspect of the results is the most important,
and gives an answer to the ever-present (if
unspoken) question, “So what?”
Researchers ought to make a clear
statement of what the findings mean to
them, three points that characterize good
quality discussion and sound conclusions:
1) if there were unanticipated difficulties in
the conduct of the study, 2) effort to place
findings into the wider frame of existing
knowledge, and 3) conclusions must match
the findings.
From: Locke, L. F., Silverman, S. J., and Spirduso. W. W. (1998). Reading and Understanding Research, SAGE Publications, Inc.
C:\Doctorado\Literature Review\Formats and Templates\5 Basic Questions – Template.doc
Were you informed of the environmental conditions under which
each instrument was used?
Does the report indicate the exact protocol to use for each
instrument?
Are you told how data from each instrument were recorded?
Did the author explain the exact manner in which each
intervention was applied to each subject?
Does the caption clearly identify the variables displayed?
Do the legends on the figure clearly identify the scales assigned
to each axis?
Does the caption provide essential information about the source
of data at each interval, as well as significant differences or
interactions?
Was everything you needed to understand the findings easily
available, or did you have to retreat to the text to retrieve
explanatory information?
Does the title clearly identify all the variables or categories of
variables displayed in the data field?
Do the row and column headings clearly identify the exact
nature of the data contained?
If appropriate, is information about sample or treatment group
sizes provided?
If appropriate, are statistically significant numbers clearly
marked and referenced in explanatory footnotes?
Is the degree of numeric precision no grater than necessary for
sustaining the meaning of the findings?
Was everything you needed to understand the table easily
available, or did, you have to retreat to the text to retrieve
explanatory information?
Do the conclusions stay within the bounds of the findings, or do
they range into unsupported speculation?
Are the conclusions presented in such manner that it is easy to
connect them with the purposes of the study and the specific
research problems that shaped it?
Does the author help you understand how the results fit into the
fund of existing knowledge?
Is the author frank and thorough in presenting the limitations to
which the conclusions are subject?
In retrospect, it is your sense that the author has made an
adequate response to the question, “So what?”
2