REVIEW 1

CHAPTER 6: METHODS OF MEASURING BEHAVIOR (=INSTRUMENTS/MEASURES)

Q1. Read the following, and identify the variables and an instrument (tests, observations,
questionnaires…) used in each research.
a. Males and female will have different attitudes towards English.
b. Fourth graders trained with 4 different programs will have different writing skills.
c. Undergraduate female students have higher motivation to study than male students.
d. Sixth graders who are learning with A and B methods will have different academic
achievements
e. A company would like to know if their customers like the new product.
Answers:
Item Variables Instruments
a Gender & attitudes towards English Attitude tests
b (4) programs & writing skills (Achievement) writing tests
c Gender & motivation to study Attitude tests
d (2) methods & academic achievements Achievement tests
e None Questionnaires and/or interviews

Q2. Read the abstracts below, and identify the measures in each research.
Abstract 1
Paraphrasing is an important skill in academic writing – it is a cognitive skill that requires higher order thinking and high
proficiency in both reading and writing. This research explored the paraphrasing ability of EFL learners at a tertiary
institution in Vietnam and examined the challenges they face when paraphrasing. Data were collected from ten second-year
English major students through (i) paraphrasing a given text and (ii) individual interviews. The findings revealed that
participants frequently paraphrased using synonyms, but rarely changed syntactic structures. The interviews showed that
the learners encountered several language-related difficulties when attempting to paraphrase, such as inadequate
understanding of the source text and the lack of vocabulary to use when paraphrasing. This paper provides insights into EFL
learners’ paraphrasing abilities and suggests some implications as to what EFL teachers should focus on to help less
experienced student writers improve their paraphrasing skills.

a. Answer: a writing task/test and (individual) interviews

Abstract 2
The present study represented a preliminary effort to empirically examine the efficacy of subtitled movie on listening
comprehension of intermediate English as a Foreign Language students. To achieve this purpose, out of a total of 200
intermediate students, 90 were picked based on a proficiency test. The material consisted of six episodes (approximately 5
minutes each) of a DVD entitled ‘Wild Weather’. The students viewed only one of the three treatment conditions: English
subtitles, Persian subtitles, no subtitles. After each viewing session, six sets of multiple-choice tests were administered to
examine listening comprehension rates. The results revealed that the English subtitles group performed at a considerably
higher level than the Persian subtitles group, which in turn performed at a substantially higher level than the no subtitle
group on the listening test.

b. Answer: a proficiency (listening) test [before the experiment, to choose the qualified participants], 6
episodes of a DVD entitled ‘Wild Weather’ [during the experiment], six sets of multiple choice
tests [after the experiment, to assess the experiment]
CHAPTER 5: LEVELS OF MEASUREMENT, RELIABILITY AND VALIDITY
Q3. Read the following questionnaire and identify the measurement levels (nominal, ordinal,
interval and ratio) of items 1-5
Background information
a. Gender:  Male  Female
b. Academic level:  High school  College  Undergraduate
 Graduate  Others
c. Average income per month:  ≤ 10.000.000VND  > 10.000.000VND
d. Age:  From 18-25  From 26 to 35  From 36 to above
Behavior
e. (SD= Strongly disagree, D= Disagree, N= Neutral, A= Agree, SA= Strongly Agree)
Items SD D N A SA
Taking selfies makes me happy.
I take selfies whenever I have a chance.
Everybody likes to hear my stories.
I feel alone when I am with my family.
I like to display behaviors that make other people pay attention to
me.

Answers: a. Nominal b. Ordinal c. Ratio d. (>=18)

Ratio Interval e. Ordinal

Q4. Read the following and identify the type of reliability (internal consistency, inter-rater,
parallel-forms, test-retest) tested in each research
Methodology
As part of his research evaluating a short-term intervention to develop reading skills, the teacher administers an
achievement test to participants, and 2.5 weeks later administers another version of the test, with different but
comparable items, to the same participants.

a. Answer: parallel-forms

Methodology
Data were collected using a self-administered questionnaire survey. The reliability and validity of all reflective
latent variables were checked. Construct reliability was assessed using Cronbach’s alpha (α) coefficient and the
composite reliability coefficient. All values exceeded the widely suggested value of .7, indicating that the level of
reliability of all the constructs was satisfactory (Fornell & Larcker, 1981).

b. Answer: internal consistency [Cronbach’s alpha (α) coefficient is used to measure internal consistency
 see Chapter 5, column 3 of the table ‘Types of reliability’]

Q5. Read the following and decide whether the self-designed test is reliable
Prof. Lee gave the same version of the Calculus test to participants at both time points, with a greater period of
time in between the tests. The reliability coefficient (r) was 0.8.

Answer: The test is reliable [-1 ≤ r ≤ 0: no reliability. r = 1: perfect reliability. See Chapter 5 - ‘How to
measure reliability’ for more information]

Q6. Read the following and identify the sources of error (trait error and method error)
a. Not enough sleep on the night before the test
b. Poor test items
c. The test room that was in a poor condition
d. Instructions that are blurred
e. Health of test taker
Answers: a. trait error b. method error c. method error d. method error e. trait error
CHAPTER 7: DESCRIPTIVE STATISTICS
ID GENDER AGE PRETEST POSTTEST GPAX OPINION
1 1 19 76 92 2.08 5
2 1 21 72 85 3.96 4
3 1 20 63 84 2.56 5
4 2 19 65 79 2.89 3
5 2 22 56 67 3.15 2
6 2 21 69 86 1.98 1
7 2 18 48 82 2.74 1
8 1 18 82 93 2.46 3
9 1 20 69 89 3.21 5
10 1 23 54 67 3.68 4

Q7. Identify mean, mode, median and standard deviation of the following variables.
No. Variable Mean Median Mode SD
1. AGE 20.1 20 18, 19, 20, 21 1.66
2. PRETEST 65.4 67 69 10.46
3. POSTTEST 82.4 85 67 9.17
4. GPAX 2.87 3 None 0.64
5. OPINION 3.3 4 5 1.57

CHAPTER 8: QUALITATIVE METHODS

Q8. Read the following and identify a research source (documentation, archival records,
physical artifacts, direct observation, participant observation, focus group) needed in each
research
a. the use of information technology policy in a school system company
b. being a gang member
c. a new policy on requirements for child-care workers
d. parents could be called together to find out about their feelings and perceptions regarding the
implementation of a new school day schedule
Answers:
Item a: physical artifacts
Item b: participant observation
Item c: documentation
Item d: focus group