Professional Documents
Culture Documents
Bus 332 Scientific Research Techniques ch9 10 11 12 13
Bus 332 Scientific Research Techniques ch9 10 11 12 13
Main Textbook:
William G. Zikmunds
Business Research Methods
WEEK DATE
TEACHING PLAN
1 2 3 4 5
20. 02 Introduction 27. 02 Ch. 9: Survey Research: An Overview 05.03 Ch. 10: Survey Research: Communication with Respondents 12.03 Ch. 11: Observation Methods 1 19.03 Ch. 11: Observation Methods 2
6
7 8 9 11 12 13
MIDTERM EXAM 1
09.04 Ch. 12: Experimental Research 1 16.04 Ch. 12: Experimental Research 2 30.04 Ch. 12: Experimental Research 3 07.05 Ch. 13: Measurement and Scaling 1 14. 05 Ch. 13: Measurement and Scaling 2
14
Selecting a Sample
Sample: Subset of a larger population
SAMPLE
Sampling: POPULATION Who is to be sampled? How large a sample? How will sample units be selected?
Random sampling: selecting a sample from a larger target population where each respondent is chosen entirely by chance and each member of the population has a known, but possibly nonequal, chance of being included in the sample.
2. Errors in Surveys
2.1. Random Sampling Error 2.2. Systematic Error (sample bias)
2.2.1. Respondent error * Nonresponse bias * Response bias 2.2.2. Administrative error * Data processing error * Sample selection error * Interviewer error * Interviewer cheating
Consider the hypothetic case in which a study sample could be increased until it was infinitely large; chance variation of the mean, or random error, would be reduced toward zero. These are random errors. Systematic errors would not be diminished by increasing sample size. (Bias in Research Studies, http://radiology.rsna.org/content/238/3/780.full)
Nonresponse Error
Nonrespondents: in almost every survey information from a small or large portion of the sample cannot be collected. These are those people who refuse to respond, or who can not be contacted (not-at-homes)
Self-selection bias: only those people who are interested strongly with topic of the survey may respond while those who are still within the same sample but indeferent or afraid avoid participating.
This leads to the over-representation of some extreme positions, but under-representation of others.
Response Bias
A bias that occurs when respondents tend to answer questions with a certain inclination or viewpoint that consciously (deliberate falsification) or unconsciously (unconscious misinterpretation) misrepresents the truth.
Longitudinal Study: data are collected at different times from the similar respondents to compare trends and identify changes.
Panel Study: A longitudinal survey of exactly the same respondents to record (in a diary) their attitudes, behaviors, or purchasing habits over time.
3. Telephone Surveys
Speed of Data Collection: Very fast
Telephone Surveys
Supervision of interviewers: High, especially with central location WATS (Wide Area Telecommunications Service) interviewing Anonymity of respondent: Moderate Ease of call back or follow-up: Easy Cost: Low to moderate Special features: Fieldwork and supervision of data collection are simplified; quite adaptable to computer technology (e.g. Central location interviewing, Computerassisted telephone interviewing, Computerized voiceactivated interviews)
Self-Administered Questionnaires
SELF-ADMINISTERED QUESTIONNAIRES PAPER QUESTIONNAIRES ELECTRONIC QUESTIONNAIRES
IN-PERSON DROP-OFF
INSERTS
FAX
KIOSK
4. Mail Surveys
Speed of data collection: Researcher has no control over return of questionnaire; slow Geographical flexibility: High Respondent cooperation: Moderate but, poorly designed questionnaire will have low response rate Versatility of questioning: Highly standardized format Questionnaire length: Varies depending on incentive Item non-response: High
Mail Surveys
Possibility of respondent misunderstanding: Highest--no interviewer present for clarification Degree of interviewer influence of answer: None - interviewer absent
E-mails are not secure and eavesdropping can possibly occur Respondent cooperation
Varies depending if e-mail is seen as spam
6. Internet Surveys
A self-administered questionnaire posted on a Web site. Respondents provide answers to questions displayed online by highlighting a phrase, clicking an icon, or keying in an answer.
Internet Surveys
Speed of data collection: Instantaneous Geographic flexibility: worldwide Cost effective, visual and interactive Respondent cooperation
Varies depending on web site Varies depending on type of sample When user does not opt-in or expect a voluntary survey cooperation is low. Self-selection problems in web site visitation surveys participants tend to be more deeply involved than the average person.
Internet Surveys
Versatility of questioning: Extremely versatile Questionnaire length: varying according to the answers of each respondent Item non-response: Software can assure none Possibility for respondent misunderstanding: High Interviewer influence of answers: None Supervision of interviewers: not required Anonymity of Respondent: Respondent can be anonymous or known Ease of Callback or Follow-up: difficult unless e-mail address is known Special Features: allows graphics and streaming media
Human behavior or physical Shoppers (buyers) movement action pattern in a store Verbal behavior Statements made by airline travelers who wait in line Facial expressions, tone of voice, and other form of body language
Expressive behavior
Temporal patterns
Physical objects
Certain data may be obtained more quickly Environmental conditions may be recorded May be combined with survey to provide supplemental evidence
Contrived Observation: Environment artificially set up by the researcher. Researchers are increasingly relying on computers to conduct simulated market testing. Offers a greater degree of control
Speedy Efficient Less expensive
However, it may be questionable as to whether or not the data collected does truly reflect a "real life" situation.
Disguised Observation Respondents are unaware they are being observed Allows for monitoring of the true reactions of individuals. Unethical if disguised observation monitors
Normally private behaviors Behaviors that may not be voluntarily revealed to researchers.
Mystery shopping popular disguised observational technique Mystery shopper Unknown to the retail establishment Visits the store Uses a structured script Observes and records the shopping experience.
Observation
Wear and tear of a book indicates how often it has been read garbology - looking for traces of purchase patterns in garbage detecting store traffic patterns by observing the wear in the floor (long term) or the dirt on the floor (short term)
Eye Tracking
Measures unconscious eye movements Records how the subject actually reads or views an advertisement, product packaging, promotional displays, websites, etc. Measures which sections attract customers' attention and how much time they spend looking at those sections Oculometers - what the subject is looking at Pupilometers - how interested is the viewer (This
device observes and records changes in the diameter of the subjects pupils)
Variations from normal voice pitch is considered a measure of emotional commitment to the question's answer.
Response Latency
It measures the speed with which a respondent gives a decision about a choice between alternatives It records the decision time necessary to make this choice. For instance: it can measure the effectiveness of an advertisement on brand preferences. It assumes that a quick expression of brand preference indicates a stronger preference.
People Meter
Electronic device to monitor television viewing behavior
who is watching what shows are being watched.
Psychogalvanometer
Measures galvanic skin response Involuntary changes in the electrical resistance of the skin Assumption: physiological changes accompany emotional reactions
1. Basics of Experiment & Causality 2. Advantages and disadvantages of the experimental method 3. Steps of a well-planned experiment 4. Validity in experiments
1.2. Causal Evidence There are three types of evidence necessary to support causality.
Agreement between Independent and Dependent Variables
First, there must be an agreement between independent and dependent variables. The presence or absence of one is associated with the presence or absence of the other.
The effect on the dependent variable should not precede the manipulation of the independent variable.
The effect and manipulation may occur simultaneously or the manipulation may occur before the effect.
3.2. Specify treatment levels: In an experiment, participants experience a manipulation of the independent variable, called the experimental treatment. The treatment levels are the arbitrary or natural groups the researcher makes within the independent variable. A control group is a group of participants that is measured but not exposed the independent variable being studied. A control group can provide a base level for comparison.
When neither the participant nor the researcher knows, the experiment is said to be double-blind.
3.4. Choose experimental design: The design is then selected. Several designs are discussed on the next several slides.
Matching is an equalizing process for assigning participants to experimental and control groups.
3.5.1. Random assignment : The sampling frame is often small for experiments and the participants may be self-selected. However, if randomization is used, those assigned to the experimental group are likely to be similar to those assigned to the control group. Random assignment allows one to make the groups as comparable as possible. It means that participants have an equal and known chance of being assigned to any of the groups in the experiment.
3.5.2. Matching : Matching is a control procedure to ensure that experimental and control groups are equated on one or more variables before the experiment.
The object of matching is to have each experimental and control participant matched on every characteristic used in the research. Matching employs a nonprobability quota sampling approach.
Quota matrix is a means of visualizing the matching process. If matching does not alleviate assignment problems, a combination of matching, randomization, and increasing the sample size may be useful.
Exhibit 10-3 presents an example of a quota matrix. One-third of the participants from each cell of the matrix would be assigned to each of the tree groups.
4. Validity in Experimentation
Internal validity exists when the conclusions drawn about a demonstrated experimental relationship truly implies cause. External validity exists when an observed causal relationship can be generalized across persons, settings, and times.
Experimental mortality: This occurs when the composition of the study groups changes during the test. Some participants may drop out the experiment.
Local history: The regular history effect already mentioned impacts both experimental and control groups alike. When one assigns all experimental persons to one group session and all control group people to another, there is a chance for some peculiar event to confound results.
Reactivity of testing on X
The reactive effect refers to sensitizing participants via a pretest so that they respond to the experimental stimulus (X) in a different way.
For instance, people who participate in a web survey may then be sensitized to store displays and organization.
3. Field experiments
X refers to the treatment or manipulation of the independent variable (more than one X refers to a different level of treatment).
O refers to the observation or measurement of the dependent variable. Experimental designs vary widely in their power to control contamination of the relationship between the independent and dependent variables. Experiments can be categorized as pre-experiments, true experiments, and field experiments based on the characteristic of control.
1. Pre-experiment
Pre-experimental research designs are research designs that are characterized by a lack of random selection and assignment.
Types of Pre-experiments: After-Only Case Study One Group Pretest-Posttest Design Static Group Comparison
In this type of experimental design only one treatment (X) or manipulation is done on the independent variable. Then, the dependent variable is measured.
An example is a media campaign about a products features without a prior measurement of consumer knowledge.
Results would reveal only how much target consumers know after the media campaign, but there is no way to judge the effectiveness of the campaign.
The lack of a pretest and control group makes this design inadequate for establishing causality.
O1
O2
This design meets the threats to internal validity better than the one-shot case study, but it is still a weak design. For example, a researcher examining the effect of a commercial on brand liking would begin by taking a pre-test to determine current levels of brand liking among the participants.
O1
O2
A comparison between the post-test and the pre-test shows the change in liking.
However, any changes in liking are not necessarily due to the commercial. The act of giving a pre-test could have influenced liking (testing effect).
O1
Control Group:
O2
This design provides for two groups, one of which receives the experimental stimulus while the other serves as a control.
For example, imagine that a new type of cheeseburger is being introduced, and an advertisement campaign is run.
After the ad airs, those who remember seeing it would be in the experimental group (X). Those who have no recall of the ad would be in the control group. The intent of each group to purchase the cheeseburger would be measured. The main weakness of this design is that there is no way to be certain that the two groups are equivalent or that the individuals are representative.
2. True experiment
A true experiment is a method of social research in which there are two kinds of variables. The independent variable is manipulated by the experimenter, and the dependent variable is measured. The signifying characteristic of a true experiment is that it randomly allocates the subjects in order to neutralize the potential to ensure equivalence. There is also a control group for comparison. Types of True experiments: Pretest-Posttest Control Group Design Posttest-Only Control Group Design
Control Group:
R R
O1 X O3
O2 O4
The symbol R means that the true experimental designs use randomly assigned groups to ensure equivalence. The effect of the experimental is: E = (O2-O1) (O4-O3). This design deals with many of the threats to internal validity, but local history, maturation, and communication among groups can still lead to problems. External validity is threatened because there is a chance for a reactive effect from testing.
R R
O1 O2
In this design, the pretest measurements are omitted. Pretests are well established in classical research design but are not really necessary when it is possible to randomize. The experimental effect is measured by the difference between O1 and O2. Internal validity threats from history, maturation, selection, and statistical regression are controlled adequately by the random assignment. Different mortality rates could cause a problem.
Example for Posttest-Only Control Group Design Buick dealerships wish to determine the effectiveness of a special test-drive incentive. Buick dealerships nationwide are randomly assigned to either the control group or the experimental group. Those in the experimental group use a promotion to encourage test drives. The control group does not use any such promotions. The number of test drives throughout are measured and compared to determine if the promotion resulted in significantly more test drives.
3. Field experiment
Experiment conducted in a natural setting (e.g. on a sports field during play). The conditions of field experiments are usually very difficult to replicate. Types of Field experiments: Nonequivalent Control Group Design Separate Sample Pretest-Posttest Design Group Time Series Design
O3
O2 O4
This is a strong and widely used quasiexperimental design. It differs from the pretest and posttest control group design because the test and control groups are not randomly assigned. There are two varieties: intact equivalent design and self-selected experimental group design.
Example for Nonequivalent Control Group Design For example, children from two different classes in school may be asked to test a toy. Participants are pre-tested on their interest in the toy. The experimental group spends time playing with the toy while the control group is not exposed to the toy. A post-test then measures interest in the toy.
R R
O1 (X) X
O2
This design is most applicable when we cannot know when and to whom to introduce the treatment but we can decide when and whom to measure. The parenthesized treatment (X) means that the experimenter cannot control exposure to the treatment. This is not a strong design because several threats to internal validity are not handled adequately. History can confound the results.
For example, an new advertising campaign for a prescription drug is introduced on television. Awareness of the brand name is measured prior to the campaign introduction. After the campaign ends, awareness is measured again.
R R
A time series design introduces repeated observations before and after treatment and allows participants to act as their own controls. The single treatment group design has before-after measurements as the only controls. There is also a multiple design with two or more comparison groups as well as the repeated measurements in each treatment group.
This format is especially useful where regularly kept records are a natural part of the environment and are unlikely to be reactive. The time series approach is also good way to study unplanned events in an ex post facto manner. The internal validity problem for this design is history. To reduce this risk, we keep a record of possible extraneous factors and attempt to adjust the result to reflect their influence. For example, if the government were to begin price controls, we could still study the effects of this action on gasoline prices later if we had regularly collected records for the period before and after the advent of price control.
Experimental Group:
Experimental Group: Experimental Group: Control Group: O1
X
X X
O
O2 O1 O2
O1 O3
O2 O4
O1 O2
Experimental Group: Control Group: Experimental Group: R Control Group: R Experimental Group: R Control Group: R
O1 O3 O1
O2 O4
(X) X
O2
1. MEASUREMENT
The process of assigning numbers or scores to attributes of people or objects. The process of describing some property of a phenomenon of interest by assigning numbers in a reliable and valid way Precise measurement requires: a) Careful conceptual definition i.e. careful definition of the concept (e.g. loyalty) to be measured b) Operational definition of the concept c) Assignment rules by which numbers or scores are assigned to different levels of the concept that an individual (or object) possesses.
Concept - A generalized idea about a class of objects, attributes, occurrences, or processes. Examples: Gender, Age, Education, brand loyalty, satisfaction, market orientation Construct - A concept that is measured with multiple variables. Examples: Brand loyalty, satisfaction, market orientation, socio-economic status Variable - Anything that varies or changes from one instance to another; can exhibit differences in value, usually in magnitude or strength, or in direction.
Concepts must be precisely defined for effective measurement. Consider the following definitions of brand loyalty: 1. The degree to which a consumer consistently purchases the same brand within a product class. (Peter & Olson) 2. A favorable attitude toward, and consistent purchases of, a particular brand. (Wilkie, p.276) The two definitions have different implications for measurement they imply different operationalizations of the concept of brand loyalty
Operational definition - A definition that gives meaning to a concept by specifying what the researcher must do (i.e. activities or operations that should be performed) in order to measure the concept under investigation.
Operationalization - The process of identifying scales that correspond to variance in a concept.
Operationalization should be in line with the conceptual definition. For example: Conceptual definition # 1 for brand loyalty:degree to which a consumer consistently purchases the same brand within a product class. Operational definition # 1 for brand loyalty: the percent of purchases going to brand A over a period of time. Operationalization: in order to measure loyalty for brand A, you will need to: Observe consumers brand purchases over a period of time, and Compute the percent of purchases going to brand A
Operationalization should be in line with the conceptual definition. For example: Conceptual definition # 2 for brand loyalty: A favorable attitude toward, and consistent purchases of, a particular brand Operational definition # 2 for brand loyalty: consumers attitude towards the brand A and the percent of purchases going to brand A over a period of time.
Operationalization: in order to measure loyalty for brand A, you will need to: Observe consumers brand purchases over a period of time, Compute the percent of purchases going to brand A, and Ask consumers questions to determine their attitudes toward brand A
Please tell me how true each statement is about the media. Is it very true, not very true, or not at all true? 1. The program was not very accurate in its portrayal of the problem. 2. Most of the story was staged for entertainment purposes. 3. The presentation was slanted and unfair.
2. SCALES
To effectively carry out any measurement we need to use some form of a scale. A scale is any series of items (numbers) arranged along a continuous spectrum of values for the purpose of quantification (i.e. for the purpose of placing objects based on how much of an attribute they possess) The thermometer for instance consists of numbers arranged in a continuous spectrum to indicate the magnitude of heat possessed by an object.
Four levels of scale measurement result from this mapping 1. Nominal Scale: a scale in which the numbers or letters assigned to an object serve only as labels for identification or classification, e.g. Gender (Male=1, Female=2) 2. Ordinal Scale: a scale that arranges objects or alternatives according to their magnitude in an ordered relationship, e.g. Academic status (Sophomore=1, Freshman=2, Junior=3, etc
3. Interval Scale: a scale that both arranges objects according to their magnitude, distinguishes this ordered arrangement in units of equal intervals, but does not have a natural zero representing absence of the given attribute, e.g. the temperature scale (40oC is not twice as hot as 20oC) 4. Ratio Scale: a scale that has absolute rather than relative quantities and an absolute (natural) zero where there is an absence of a given attribute, e.g. income, age.
Data Characteristics
Numerical Operation
Ordinal
Classification and Rank ordering order but no distance or unique origin Classification, order, and distance but no unique origin Arithmetic operations that preserve order and magnitude
Median Academic status Range (1=Freshman, Percentile ranking 2=Sophomore, 3=Junior, 4=Senior) Mean Standard deviation Variance Temperature in degrees Satisfaction on semantic differential scale Age in years Income in dollars
Interval
Ratio
Classification, Arithmetic Geometric mean order, distance operations on Coefficient of and unique origin actual quantities variation
However, for index measures, the variables need not be strongly correlated with each other, whilst for composite measures, the variables are typically strongly correlated as they are all assumed to be measuring the construct in the same way
Index Measures
Example 1: Index Measure Construct: Social class Measures: Linear combination (index) of occupation, education, income. Social class = 1Education + 2Occupation + 3Family Background
Composite Measures
Example 2: Composite Measure Construct: Attitude Toward Brand A Measures: Extent of agreement/disagreement with multiple statements: a) I like Brand A very much b) Brand A is the best in the market c) I always buy Brand A Statements a), b), c), constitute a scale to measure attitudes toward brand A
Reverse Coding
Means that the value assigned for a response is treated oppositely from the other items.
3.1. RELIABILITY
The degree to which a measure is free from random error and therefore gives consistent results. An indicator of the measures internal consistency
Test-Retest Stability
(Repeatability)
Reliability
Internal Consistency
Splitting halves
Equivalent forms
2. Equivalent forms
3.2. VALIDITY
The accuracy of a measure or the extent to which a score truthfully represents a concept. The ability of a measure (scale) to measure what it is intended measure. Establishing validity involves answers to the following:
Is there a consensus that the scale measures what it is supposed to measure? Does the measure correlate with other measures of the same concept? Does the behavior expected from the measure predict actual observed behavior?
Validity
Face or Content
Criterion Validity
Construct Validity
Concurrent
Predictive
TYPES of VALIDITY
1. Face or content validity: The subjective agreement among professionals that a scale logically appears to measure what it is intended to measure.
2. Criterion Validity: the degree of correlation of a measure with other standard measures of the same construct. Concurrent Validity: the new measure/scale is taken at same time as criterion measure. Predictive Validity: new measure is able to predict a future event / measure (the criterion measure).
TYPES of VALIDITY
3. Construct Validity: degree to which a measure/scale confirms a network of related hypotheses generated from theory based on the concepts. Convergent Validity. Discriminant Validity.
3.3. SENSITIVITY
The ability of a measure/scale to accurately measure variability in stimuli or responses;
The ability of a measure/scale to make fine distinctions among respondents with/objects with different levels of the attribute (construct). Example - A typical bathroom scale is not sensitive enough to be used to measure the weight of jewelry; it cannot make fine distinctions among objects with very small weights.
Composite measures allow for a greater range of possible scores, they are more sensitive than single-item scales. Sensitivity is generally increased by adding more response points or adding scale items.
References
Main Textbook: William G. Zikmunds Business Research Methods Other Texbook: Donald R. Cooper and Pamela S. Schindlers Business Research Methods Lecture Notes: Dr. Alhassan G. Abdul-Muhmin