OL14-WEEK 7

Chapter 8
Continue:
Response technique: rating and ranking scales
Steps in developing questionnaire
Goodness of measures: Reliability and Validity

Business Research Methods |

 Response Technique
(1) Rating Scales
 the respondent is required to
choose his preference value
toward a statement or item.
(2) Ranking Scales
 the respondent is required
to compare two or more
objects directly.
• Dichotomous scale
• Multiple-choice scale
• Likert scale
• Semantic differential scale
• Numerical scale
• Stapel scale
• Constant sum scale
• Graphic rating scale
• Paired comparison scale
• Forced ranking scale
• Comparative scale
Business Research Methods |
 B) RATING SCALES
Dichotomous Scale
Have you ever purchased online?
Yes No

Multiple-choice Scale
Which of the following products would you purchase online?
Air-ticket
Insurance
Groceries
Fashion products
Cosmetics
Business Research Methods |
Likert Scale

Online shopping is fun.

Strongly Disagree Disagree Neutral Agree Strongly Agree
1 2 3 4 5

Semantic Differential Scale

Business Research Methods |

Numerical Scale
Using the scale below, state how frequently you purchase following product online.
Very Frequently 5 4 3 2 1 Infrequently

Air-ticket ___
Groceries ___
Fashion products ___

Stapel Scale
State how satisfied you are with the following online company.
+3 +3 +3
+2 +2 +2
+1 +1 +1
Amazon eBay Alibaba.com
-1 -1 -1
-2 -2 -2
-3 -3 -3

Business Research Methods |

Constant Sum Scale
Security
In choosing an online shopping
Privacy
company, indicate the importance of
Price
the following five factors and allocate
Promotion
scores for each, to a total of 100.
Return Policy
Total 100

Graphic Rating Scale

Express your feeling when you receive a parcel from an online shopping company. (Mark ‘X’ at bottom of the graphic.)

  
|___|___|___|___|___|___|___|___|___|___|

Business Research Methods |

 B) RANKING SCALES

Paired Comparison Scale

Please select one of the following pairs in terms of quality mobile phones.

___ Samsung ____ Motorola

Comparative Scale
Compared to your previous mobile phone brand’s performance, how do you feel about the current brand?

Very Poor About the Same Excellence

1 2 3 4 5

Business Research Methods |

Forced Ranking Scale

Rank the following mobile phone brands in your order of preference. Assign 1 to the most preferred choice and 5 to the
least preferred.
___ Samsung
___ Apple
___ LG
___ Sony Ericsson
___ Motorola

Business Research Methods |

 Steps in developing Questionnaire
• Definition
A questionnaire is a pre-formulated, written set of questions to which the respondent
records his answers

• Steps
1. Determine the content of the questionnaire
2. Determine the form of response
3. Determine the wording of the questions
4. Determine the question sequence
5. Write cover letter

Business Research Methods |

 1. Questionnaire content
• Framework
Need information for all constructs in framework

• Measurement: Operationalizing
– Objective construct:
• 1 element/items
=> 1 question
– Subjective construct:
• multiple elements/items
=> multiple questions

Business Research Methods |

 2. Response format
• Closed vs. Open-ended questions
– Closed questions
• Helps respondents to make quick decisions
• Helps researchers to code
– Open-ended question
• First: unbiased point of view
• Final: additional insights
• Complementary to closed question: for interpretation purpose

• Cfr. Measurement: Response scales

Business Research Methods |

 3. Question wording
• Avoid double-barreled questions
– Ex. I like to swim and playing football in past time.

• Avoid ambiguous questions and words

– Ex. What do you think about parking?

• Avoid hyperbole/scientific - use of ordinary words

– Ex. It was the tastiest pie in the entire universe.
– Ex. Hibiscus rosa-sinensis is my favorite flower.

Business Research Methods |

 3. Question wording

• Avoid leading or biasing questions

– Ex. Don’t you think that UUM is the best university in Malaysia?

• Social desirability
– Ex. Everyday I brush my teeth before I go to sleep.

• Avoid recall depended questions

– Ex. How many books have you read last year?

Business Research Methods |

 3. Question wording
• Use positive and negative statements
– Ex. UUM delivers high quality service.
UUM has poor customer operational support

• Avoid double negatives

– Ex. I didn't steal nothing.

• Limit the length of the questions

Rules of thumb:
– < 20 words
– < one full line in print

Business Research Methods |

 4. Question sequence

Personal and sensitive data at the end

Business Research Methods |
 5. Cover letter
• The cover letter is the introductory page of the questionnaire

• It includes:
– Identification of the researcher
– Motivation for respondents to fill it in
– Confidentiality
– Thanking of the respondent

Business Research Methods |

 Goodness of Measures
• Ensure that the instruments we use in our research do indeed measure the variables
they are supposed to, and that they measure them accurately.

• In operationally defining perceptual and attitudinal variables, we have not

overlooked some important dimensions and elements or included some irrelevant
ones. The scales developed can often be imperfect, and errors are prone to occur in
the measurement of attitudinal variables.

• The use of better instruments will ensure more accuracy in results, which in turn will
enhance the scientific quality of the research.

Business Research Methods |

Goodness of Measures

Business Research Methods |

 Validity
• A test of how well an instrument that is developed measures the
particular concept it is intended to measure.

Business Research Methods |

 Validity of Variables
a) Content validity refers to the agreement among the professionals that
the items are designed accurately and adequately measure the
variable.

b) Construct validity refers to the degree to which the items have been
operationalised consistent with underlying theory.

c) Criterion validity refers to the degree to which the measurement is

adequate to differentiate individuals on an attribute.

Business Research Methods |

 Reliability
• A test of how consistently a measuring instrument measures whatever concept it is
measuring.

• Indicates extent to which it is without bias and hence ensures consistent

measurement across time (stability) and across the various items in the instrument
(internal consistency).

Business Research Methods |

 Stability
• Ability of a measure to remain the same over time, despite uncontrollable testing
conditions or the state of the respondents themselves.

– Test–Retest Reliability: The reliability coefficient obtained with a repetition of the

same measure on a second occasion.

– Parallel-Form Reliability: Responses on two comparable sets of measures tapping

the same construct are highly correlated.

Business Research Methods |

 Internal Consistency
• Indicative of the homogeneity of the items in the measure that tap the construct.

– Inter-item Consistency Reliability: This is a test of the consistency of respondents’

answers to all the items in a measure. The most popular test of inter-item
consistency reliability is the Cronbach’s coefficient alpha.

– Split-Half Reliability: Split-half reliability reflects the correlations between two

halves of an instrument.

Business Research Methods |

 Reliability, Validity and Evidence

Type What is Measured Evidence of Goodness

 Correlation (0.30 < r)
 Cronbach’s alpha (0.70 < α)
Reliability Item consistency
 Confirmatory factor analysis (Composite
Reliability > 0.70)
 Review past literature
Content validity Items developed are adequate
 Expert opinion
 Exploratory factor analysis (Loading above 0.50)
Construct validity Items developed are relevant  Confirmatory factor analysis (Average Variance
Extracted > 0.50)
An instrument developed can either be
Criterion validity  Correlation (0.50 < r < 0.70)
predictive or concurrent

Business Research Methods |

 Reliability and Validity

Reliable but Valid but Not reliable Reliable and

not valid not reliable and not valid valid

Business Research Methods |

 Exercise 1 (during class)

“A valid instrument is always reliable, but a reliable instrument is

not always valid”. Comment on this statement.

Business Research Methods | 26