ENG 2001

ENGLISH FOR ACADEMIC PURPOSES II

Week 4 Session 1
Instructor: Dr. Pauline Liang
Office: TB 506
Office Hour: Wed. 10-11am
Email: lianghuiling@cuhk.edu.cn
 TODAY’S AGENDA

• Quantitative research vs. qualitative research

• Selecting research designs

• A quantitative research method: questionnaires

• How to construct questionnaires

• Writing titles
2
METHODOLOGY VS. METHODS

• Research methodology: the philosophy or the general principle which

will guide your research

• Research methods: tools you use to collect your data

From: Dawson, C. (2002). Practical Research Methods: A User-friendly Guide to Mastering

Research Techniques and Projects. Oxford: How To Books.
3
METHODOLOGY: QUANTITATIVE VS. QUALITATIVE

• Quantitative research generates statistics through the use of large-scale

survey research, using methods such as questionnaires or structured
interviews.

• Qualitative research explores attitudes, behaviours and experiences

through such methods as interviews or focus groups. It attempts to get an
in-depth opinion from participants.
4
METHODOLOGY: QUANTITATIVE VS. QUALITATIVE

• Quantitative research is empirical research where the data are in the form
of numbers.

• Qualitative research is empirical research where the data are not in the
form of numbers.

Punch’s definition (1998:4)

5
 QUANTITATIVE RESEARCH
• A survey study: A survey study usually involves a relatively large sample and is
descriptive in nature. It either describes the features of a few variables or the
relation between two or more variables.
• “A quantitative investigation of stakeholder perceptions of LPATE”

• An experimental study: An experimental study is a study in which the researcher

manipulates one or more independent variables and measures their effect(s) on
one or more dependent variables while controlling the effect of extraneous
variables. (Test some hypothesis regarding causation)
• “Sad music induces pleasant emotion”
• P. 46 Essential of Research Design and Methodology (Course Reserve) 6
RESEARCH DESIGNS

• Which methodology or method(s) you choose depends on

o the purpose of your study, as well as the research questions
o your underlying philosophy of research
o your preferences and skills

7
WHICH METHOD(S) WILL YOU CHOOSE?

• This research aims to find

out what primary school
teachers think about the
educational value of “The
Teletubbies” television
programme.

8
WHICH METHOD(S) WILL YOU CHOOSE?

• What is the relationship

between students’
motivation and their L2
learning outcome? (Or, to
what extent can the
students’ motivation
predict their L2 learning
outcomes?)
9
PHILOSOPHY OF RESEARCH

• The positivist view: external reality awaits our discovery through a

series of increasingly good approximations to the truth

• The constructivist view: reality is constructed uniquely by each person

10
DATA COLLECTION - QUESTIONNAIRES

11
QUESTIONNAIRES

• Questionnaires are defined as “any written instruments that present

respondents with a series of questions or statements to which they are to
react, either by writing out their answers or selecting from among the
answers” (J.D. Brown, 2001, p.6)

12
WHAT CAN QUESTIONNAIRE MEASURE?

• Factual questions:
o Age, gender …
o e.g. Time spent on mobile phones, time to go to bed, Gaokao English scores, single or
in a relationship
• Behavioral questions (what people did or usually do):
o Peoples’ actions, life-styles and habits

• Attitudinal Questions (what people think)

o Attitudes, opinions, beliefs, interests, and values
13
WHERE DO THE ITEMS COME FROM?

What to ask?
1. Get data from exploratory data (interviews, informal chat with informants,
brainstorming with your team).
2. Borrow questions from established questionnaires (You need to
acknowledge the sources).
3. Get data from the literature.

14
 TYPES OF QUESTIONS: OPEN
• Fill-in questions:
My hometown is _________.
I started to learn English at the age of _________.
• Short-answer questions:
What are your best memories of studying at CUHK(SZ)?_________________
__________________________________.
What are disadvantages of using English as a medium of instruction at CUHK(SZ)? ______
____________________________________________________________
• Longer answer questions:
Please describe what are the most important things in learning English vocabulary?
15
TYPES OF QUESTIONS: CLOSED

• 1. List
• 2. Category
• 3. Ranking
• 4. Scale
Handout (on
• 5. Quantity/frequency Blackboard): Closed
question types in
• 6. Grid survey questionnaires
16
 CLOSED OR CLOSED-ENDED QUESTIONS

• Closed-ended questions provide respondents with preselected answers

from which to choose.
o one needs to know the possible answers, or choices in advance.
o produce standardised data that can be analysed statistically.
LIKERT SCALES

• One of the most commonly used technique in questionnaire design is

the Likert Scales.

• Likert scales consist of a series of statements. Respondents are asked to

indicate the extent they agree or disagree with these items.

18
QUESTIONS ON LIKERT SCALE

19
WRITING UP QUESTIONNAIRE ITEMS

• Questionnaire items rarely take the form of the actual questions that end
with a question mark. It is more effective to turn it into a statement.

20
RULES FOR ITEM WORDING

• Rule No.1 Aim for short and simple questions

• Rule No.2 Avoid ambiguous words
• It’s good to use English as a medium of instruction at CUHK(SZ).

• Try not to use words which may have a double meaning or be

misinterpreted.

21
RULES FOR ITEM WORDING

• Rule No. 3 Avoid two-edged (two-barreled) questions

• I like my English teacher and GE teacher.
• I watch English movies and read English books to practice my English.

22
 RULES FOR ITEM WORDING

Rule No. 4 Avoid negative questions

For example:
The United Nations should not have more authority to intervene in a nation's military
affairs."
I do not like my English class.
Negative questions are difficult for many respondents to answer because they require an
exercise in logical thinking. If you ask your respondents whether they agree or disagree
with the following statement, they may have difficulty answering it.
RULES FOR ITEM WORDING

Rule No. 5 Avoid biased or emotional words and phrases

• For example:
We should reward the hard-working CUHK(SZ) students who stayed up in the
library whole night.
• Biasing words and phrases elicit emotional responses that may have
little to do with the issues addressed by the survey.
RULES FOR ITEM WORDING

• Rule No. 6 Make sure that your questions don’t contain some type of
prestige bias.
• This phrase refers to questions which could embarrass or force
respondents into giving a false answer. They might do this if they do not
want to look “bad” in front of the researcher, or they might do it because
it is expected behaviour.
• Be careful about how you try to obtain this information

25
RULES FOR ITEM WORDING

• To sum up:
• Rule No.1 Aim for short and simple questions
• Rule No.2 Avoid ambiguous words
• Rule No. 3 Avoid two-edged (two-barreled) questions
• Rule No. 4 Avoid negative questions
• Rule No. 5 Avoid biased or emotional words and phrases
• Rule No. 6 Make sure that your questions don’t contain some type of prestige
bias.
26
PILOTING THE QUESTIONNAIRES

• Test it out
• Make sure your participants know it is a pilot test and ask them to
forward any comments they may have about the length, structure and
wording of the questionnaire.
• Amend accordingly and re-pilot
• Send out questionnaires

27
GROUP ACTIVITY

• Read the questionnaire items on the handout and decide what is wrong
with them. Think about how you might overcome the problems you have
identified.

28
SAMPLING

• How many subjects will be involved in the study, and why?

• How will the subjects be selected and to what extent can the
chosen subjects represent the population, and why?
• How many times will the subjects be involved in the study, and
why?

29
METHODS_PROPOSAL

• The methods section provides the technical details of how the study is
conducted or how the data are collected to answer the research questions
or test the hypothesis.

• Knowing how the data are collected helps the readers evaluate the
validity and reliability of your study.

30
METHODS_PROPOSAL

• Explain what kind of data you are going to use.

• Describe how you are going to collect the data and explain why.
• Explain how you are going to analyse the data.
• Relate your methods to your research questions.

31
A PROVISIONAL TITLE
• Alternative Sources of Feedback and Second Language Writing
Development in University Content Courses
• Formative assessment and self-regulated learning: How formative
assessment supports students' self-regulation in English language
learning
• Formative Assessment in a Test-Dominated Context: How Test
Practice Can Become More Productive
• Sad music induces pleasant emotion
• An analysis of Chinese EFL students’ use of first and second language
in peer feedback of L2 writing
• The relationship between loneliness and smartphone addiction of
students in the Chinese University of Hong Kong (Shenzhen)

33
HOMEWORK

• Start to draft your research proposal

34
ADDITIONAL MATERIALS: QUANTITATIVE DATA
ANALYSIS

35
ANALYSE NOMINAL DATA

• How can we analyse nominal data?

Y4
Nominal data Year 3
o I am studying at __________
o A. Year 1; B. Year 2; C. Year 3; D. Year 4 Year
1
We can calculate frequencies and percentages.
Year 2

36
ANALYSE NOMINAL DATA

Coniam, D., Falvey, P., & Xiao, Y. (2017). An Investigation of the

Impact on Hong Kong’s English Language Teaching Profession of the 37

Language Proficiency Assessment for Teachers of English (LPATE).

RELC Journal, 48(1), 115-133.
ANALYSE ORDINAL DATA

How can we analyse ordinal data such as Likert Scales?

You probably want to know

Example of
the average score. data set.
38
MEAN
• The mean is the numerical average of the data set.

• The mean is found by adding all the values in the set, then dividing the
sum by the number of values.

• Please check the video on the Blackboard on how to conduct analysis.

39
 What
information
can you get
from the
ANALYSE AND INTERPRET ORDINAL DATA mean data
here?

What can Mean tell us?

Q: Which do
you think
students agree
most? Which
least?

Source: 40

Xiao, Y. (2008). Washback of the NMET reading test on the third year students in a provincial
key school in Hubei province, China. Unpublished MA Dissertation. University of Warwick, UK.
STANDARD DEVIATIONS
Standard deviation tells about how measurements for a group are
spread out from the average.

3.
3.
5
5
41
STANDARD DEVIATION (SD)

Q: What can you

learn from the
SD of the two
items?

Source:
Xiao, Y. (2008). Washback of the NMET reading test on the third year
students in a provincial key school in Hubei province, China. 42

Unpublished MA Dissertation. University of Warwick, UK.

AVERAGE MEAN OF SEVERAL ITEMS
• If you have several items under one theme, you can calculate the average mean for a
group of items.
You can calculate the
mean of items in one
group to find out if
For example: CUHK SZ students think
• It’s good to use English as a medium of instruction at CUHK SZ. EMI is good.

• At CUHK SZ, using English as a medium of instruction helps me improve my English.

• At CUHK SZ, using English as a medium of instruction enables me to get access to a
wide range of resources.
• At CUHK SZ, using English as a medium of instruction offers me more chance for
studying abroad. 43
QUANTITATIVE APPROACHES

• Attempts to explain phenomena by collecting and analysing numerical data;

• Tells you if there is a “difference” but not necessarily why;
• Data collected are always numerical and analysed using statistical methods;
• Variables are controlled as much as possible so we can eliminate interference
and measure the effect of any change;
• Randomisation to reduce subjective bias;
• If there are no numbers involved, its not quantitative;
• Some types of research lend themselves better to quantitative approaches
than others.