Introduction to experimental designs

Lecture 1 (SEN1221 – part II)

9-12-2019
Eric Molin - TU Delft

Delft
University of
Technology

Challenge the future

Introduction

stated choice
Regret experiments
in Traveler Decision Making 2
Focus

• part I of this course: estimating choice models

• data were given

• part II of this course: collecting choice data

• mainly how to construct stated choice experiments?
• the model determines the construction of the experiment

• also: extensions of utility function

• interaction effects
• dummy & effects coding of categorical attributes
• non-linearity

Regret in Traveler Decision Making 3

3
organization

• 6 Lectures about theory

• 2 computer practical's

• Q&A & exam example – this part of the course

• Exam
• Standard written exam questions, thus without computer
• But write & understand Ngene syntax code!

Regret in Traveler Decision Making 4

4
Two data collection paradigms

• revealed preference: real-market alternatives

1. ‘By which mode did you travel to work today?’
2. ‘What was the travel time?’
3. ‘What were the total costs’?
focus
• stated preference: hypothetical alternatives
Which option would you choose for traveling to work’?
❑ car, costs €4, takes 20 minutes
❑ train, €5, takes 30 minutes

Regret in Traveler Decision Making 5

5
Assignment: defining terms

Which term is indicated?

alternative

choice set

attribute attribute-level

Regret in Traveler Decision Making 6

6
Answer to assignment

• attributes: characteristics of choice alternatives

• e.g. cost, time, distance, comfort

• attribute levels: particular values of attributes

• e.g. €10, 15 minutes, 2 km., second class

• choice alternative
• choice option with scores (levels) on attributes

• choice set
• the group of choice alternatives that can be chosen at a
particular moment in time

Regret in Traveler Decision Making 7

7
8

SC choice set example 1

Which electric car do you prefer?

0 0

Regret in Traveler Decision Making 8

8
9

SC choice set example 2

Regret in Traveler Decision Making 9

9
Overview course part II

• L1 - Constructing choice alternatives & experimental designs

• L2 - Constructing choice sets & orthogonal designs
• Practical 1: orthogonal designs

• L3 - Efficient designs
• L4 - Non-linearity and Hierarchical Experiments (complex variables)
• Practical 2: efficient designs & non-linearity

• L5 – Context-dependent experiments
• L6 – Reflection on SC methods

• Q&A and exam practice

Regret in Traveler Decision Making 10

10
main challenge of SC experiments

1. create sufficient variation in the choice situations in order that

the intended utility functions can be estimated…

2. …in such a way that estimated parameters are reliable

→ have small standard errors by:
• choosing proper experimental design
• creating choice tasks that do not exhaust respondents

3. …in such a way that estimated parameters are valid

→ resemble true parameters
• construct choice situations that resemble real world
choice situations as much as possible
Regret in Traveler Decision Making 11
11
reliable parameter

• reliable parameter
• if parameter has a small standard error
• is precise

more reliable

less reliable

Regret in Traveler Decision Making 12

12
unbiased parameters

• estimator is unbiased if:

• estimated parameter = true value of unknown parameter
• thus, not systematically too high or too low
• e.g. estimated VoT in experiment resembles the willingness
to pay to save time in reality

Regret in Traveler Decision Making 13

13
 validity and reliability

reliable parameters: efficient - small standard error

valid parameters: unbiased (resemble true parameters)
Regret in Traveler Decision Making 14
14
rating experiments

stated choice
Regret experiments
in Traveler Decision Making 15
6

history: rating experiments

How attractive is each of the following rooms to you?

You may express your response on a rating scale running from

(1) very unattractive to (10) very attractive

10 m2 room space
250 euro per month
share toilet and bathroom with 3 other persons
share kitchen with 6 other persons
2 km. from university
2 km. from city center

very unattractive 1 2 3 4 5 6 7 8 9 10 very attractive

Regret in Traveler Decision Making 16

16
regression analysis

• rating data are analyzed by linear regression analysis

• observed rating is assumed to be of interval level
• regression equation is estimated:
observed rating = ∑ bi * Xi + ei

• bi: regression coefficient to be estimated

• denotes the units change in the rating by one unit
change in attribute Xi

• ei: prediction error (residual)

• for a more elaborate introduction to regression analysis:

slides: multiple regression Trail 15-10-19.pptx by Eric
Regret in Traveler Decision Molin
Making 17
17
 estimated coefficients

est. stand.error
• constant 7.256 .430
• room space .146 .011 interpret estimate & constant
• price -.011 .002
• wc&bath -.148 .028
• kitchen -.094 .028
• dist center -.367 .042
• dist uni -.295 .042

• interpretation est. room space: every additional square meter

room space, increases attractiveness by 0.146 points

• interpretation constant: ‘the score on attractiveness if all

attributes have value 0
• nonsense!: thus interpret as constant
Regret in correction factor
Traveler Decision Making 18
18
‘reliability → standard error (s.e.)

• s.e. indicator of the reliability of an estimated parameter value

• should be as small as possible!

• 95% reliability interval

• parameter estimate +/- 1.96 * s.e.
• low s.e. → small interval → reliable parameter
• practical use:
• any value in the 95% interval does not differ
statistically from the estimate
• if 0 is in the interval: parameter is not significant

• t-value of parameter = estimated parameter / s.e.

• low s.e. → high t-value Regret
→ statistical significance
in Traveler Decision Making 19
19
reliability
est. +/- 1.96*stand.error
est. stand.error 95% reliability interval
• constant 7.256 .430 6.413 8.099
• room space .146 .011 0.124 0.168
• price -.011 .002 -0.015 -0.007
• wc&bath -.148 .028 -0.203 -0.093
• kitchen -.094 .028 -0.149 -0.039
• dist center -.367 .042 -0.449 -0.285
• dist uni -.295 .042 -0.377 -0.213

• standard error: imprecision of estimate

• 95% reliability interval: any number in the interval does not differ
in statistical sense from estimate
• needs to be as small as possible → small stand.errors needed!
Regret in Traveler Decision Making 20
20
calculating s.e. in multiple regression

𝑠𝑒
𝑠. 𝑒.𝑏𝑖 =
standard error of
coefficient bi (1 − 𝑅𝑋2𝑖 𝐺𝑘 ) ∗ 𝑠𝑋2𝑖 ∗ (𝑁 − 1)

• se = standard deviation of residuals (a model characteristic)

• sXi2 = variance of predictor Xi
• N – 1 = Number of observations – 1
• 𝑅𝑋2𝑖 𝐺𝑘 = explained variance of predictor by other predictor vars.
• 1- 𝑅𝑋2𝑖 𝐺𝑘 = tolerance = unexplained variance by other predictors

• Which magnitude should the correlations among the predictor

variables have, so s.e. has the smallest value?
• if all correlations are zero → smallest possible s.e.
Regret in Traveler Decision Making 21
21
The reliability of a coefficient is higher when…
Standard dev. of residuals
𝑠𝑒
𝑠. 𝑒.𝑏𝑖 =
(1 − 𝑅𝑋2𝑖 𝐺𝑘 ) ∗ 𝑠𝑋2𝑖 ∗ (𝑁 − 1) number of observations

tolerance variance of attribute Xi

- …the standard deviation of residuals is small (high R2)

- hence, small prediction errors / high explained variance

- …tolerance is large (low explained variance by other attributes)

- low correlations between attributes

- … the variance of the attribute is large

- hence, observe values over largest range of X values

- … N is large
- thus realize many observations!
Regret in Traveler Decision Making 22
22
Summary

• we aim to construct experiments that result in:

1. reliable estimates
• mainly: minimize standard errors
2. valid (unbiased) parameters
• a.o. avoid impossible attribute level combinations
• experiment resembles real choice environment

• minimizing standard errors by:

1. establishing zero correlations among attributes
• is nuanced later
2. choose wide ranges for attribute level values
• realize high variances of attributes
Regret in Traveler Decision Making 23
23
constructing choice alternatives

stated choice
Regret experiments
in Traveler Decision Making 24
Methodological decisions

this lecture
1. Which attributes to select?
2. Which attribute levels to include?
3. How to construct choice alternatives?
4. How to construct choice sets?
• orthogonal designs? (L2)
• efficient designs? (L3)
5. Should non-linearity be taken into account? (L4)
6. How to deal with complex variables? (L4)
7. Which choice context? Should it vary? (L5)
Regret in Traveler Decision Making 25
25
Selecting attributes and attribute levels

stated choice
Regret experiments
in Traveler Decision Making 26
which attributes to include?

1. most important attributes for respondents

• otherwise respondents make their own assumptions

2. relevant for policy or design

• include attributes that can be influenced by policy

Regret in Traveler Decision Making 27

27
 number of attribute levels

• number of levels per attribute

• typically (but not necessarily) limited to 2 to 4 levels

• for nominal & ordinal level attributes:

• all relevant categories

• for interval & ratio level attributes:

• depends on assumptions about the shape of the utility
function
• and the wish to estimate deviations from linearity

Regret in Traveler Decision Making 28

28
Two attribute levels

• can the linearity assumption be tested?

• no, this requires at least three levels!

1,5

0,5

-0,5 150 250

-1

-1,5

-2

Regret in Traveler Decision Making 29

29
three attribute levels

• by applying specific coding, utility contributions of the three levels

can be calculated
1.5

0.5

-0.5
150 200 250
-1

-1.5

-2

• Does this allow testing for linearity?

• yes, test whether a quadratic component is significant
• See lecture 4
Regret in Traveler Decision Making 30
30
number of attribute levels

1.5
2

1.5
1

1 0.5

0.5 0
0

-0.5
-0.5
150 200 250
-1
150 250 -1

-1.5 -1.5

-2 -2

two levels: straight line three levels: curved line

1,5
1
0,5
0
-0,5
0 c. 25 c. 50 c. 75 c.
Note: estimating smooth relation-
-1
-1,5
-2 ships are possible by including
four levels: S-curved line polynomials (X2, X3)
Regret in Traveler Decision Making 31
31
 range of attribute levels (interval & ratio)

• choose wide ranges

• all levels should fit in
• of existing alternatives
• of future alt. after implementing policy measures
• it increases validity:
• interpolation is more reliable than extrapolation!
• it increase reliability:
• smaller standard errors of parameters

• however, check:
• all combinations of all attribute values should make sense

Regret in Traveler Decision Making 32

32
equidistance in attribute levels

• whenever possible, preserve equidistance

• this assures orthogonality between attributes

• thus
• 10, 20, 30 minutes
• and not 10, 15, 30 minutes

Regret in Traveler Decision Making 33

33
Combining attribute levels into alternatives

stated choice
Regret experiments
in Traveler Decision Making 34
Full-factorial designs

stated choice
Regret experiments
in Traveler Decision Making 35
Full factorial designs

• All possible combinations of all selected attribute levels are

constructed
• Number of alternatives: LN
• L = number of levels
• N = number of attributes

• How many alternatives?

• 2 attributes in 2 levels each? 2*2 = 22 = 4
• 3 attributes in 2 levels each? 2*2*2=23 = 8
• 3 attributes in 3 levels each? 3*3*3=33 = 27
• 3 attributes in two levels and 3 attributes in four levels?
2*2*2*4*4*4 = 23 * 43 = 8 * 64 = 512

• full factorial designs allow to estimate

Regret interaction-effects
in Traveler Decision Making 36
36
intermezzo: interaction-effects

stated choice
Regret experiments
in Traveler Decision Making 37
What are interaction effects?

• main effects only

• Vj = βT • waiting time + βS • station size
• impact waiting time is independent of station size

• Would waiting time be less annoying at large stations with waiting

room, shops, café’s, or at to small stations without any facilities?

• extension with interaction effects

• Vj = βT • waiting time + βS • station + βTS • waiting time • station
• Vj = (βT +βTS • station) • waiting time + βS • station
• waiting time parameter becomes: (βT + βTS • station size)
• if βTS is statistically significant:
• impact waiting time is dependent on station size
Regret in Traveler Decision Making 38
38
Interaction effect in conceptual model

station
size Utility choice

waiting
time

station size moderates (modifies) the weight (of the parameter) of

waiting time

Regret in Traveler Decision Making 39

39
Assignment

V = -0.10 • WTIME + 0.10 • STATION + 0.05 • WTIME • STATION

coding station:
• large station, with facilities = 1
• small station, no facilities = 0
• see lecture 4 for coding

• Rewrite the equation to make clear that the parameter for

WTIME is dependent on STATION size

Regret in Traveler Decision Making 40

40
 answer to assignment

V = (-0.10 + 0.05 • STATION) • WTIME + 0.1 • STATION

• coding: large station = 1 ; small station = 0
• WTIME = waiting time in minutes

• parameter of waiting time per station size:

• βWT_large station: (-0.10 + 0.05 • 1) = -0.05
• βWT_small station: (-0.10 + 0.05 • 0) = -0.10

• thus as expected, a minute waiting time weighs less in large stations

end of intermezzo

Regret in Traveler Decision Making 41

41
Full factorial design

• advantage
• it is most simple
• all main and all interaction effects can be estimated

• disadvantage
• often too many alternatives
• happily, we usually do not need all combinations
• in transport interaction effects are usually found to
be zero
• so typically, we assume interaction effects to be
zero and can withstand with a smaller number of
alternatives
Regret in Traveler Decision Making 42
42
Fractional factorial designs

stated choice
Regret experiments
in Traveler Decision Making 43
Fractional factorial designs

• Fractional factorial designs

• fraction of full factorial design
• scheme’s constructed by mathematicians
• or by software Ngene

• Advantage:
• smaller number of alternatives

• Disadvantage:
• allow estimating main effects only, thus no interaction effects

Regret in Traveler Decision Making 44

44
experimental design scheme

• table with numbers that determines which attribute levels to

combine into alternatives: 1 0 0
1 1 1
0 0 1
0 1 0

• each attribute is assigned to a column (vertical)

• How many attributes does this design allows to vary?

• number in cell indicates an attribute level

• How many levels can be varied for each attribute?

• each row represents an alternative

• How many alternatives does this design construct?
Regret in Traveler Decision Making 45
45
constructing alternatives

experimental
design 4 alternatives
1 0 0 1) €3 , 10 min , low comfort
1 1 1 2) €3 , 20 min , high comfort
0 0 1
3) €1 , 10 min , high comfort
0 1 0
4) €1 , 20 min , low comfort

design coding of 3 attributes with 2 levels each:

1 price 2 time 3 comfort

0: € 1 0: 10 min. 0: low
1: € 3 1: 20 min 1: high

Regret in Traveler Decision Making 46

46
types of fractional factorial designs

1. random designs
• randomly selected from full factorial

2. orthogonal (fractional factorial) designs

• correlations between attributes are zero
• traditional
• lecture 1 & 2

3. efficient designs
• minimizes standard errors (of logit parameters)
• requires priors
• more recent
• lecture 3
Regret in Traveler Decision Making 47
47
Orthogonal designs

stated choice
Regret experiments
in Traveler Decision Making 48
Orthogonal

• recap multicollinearity
• high correlation among attributes
• resulting in high unreliable estimates or parameters than
cannot be estimated

• solution: orthogonal designs

• zero correlations between attributes
• results in low standard errors (reliable parameters)

• main question: which alternatives from all possible should we

select?

Regret in Traveler Decision Making 49

49
Assignment

1. What is the correlation between the attributes travel time and

travel cost in the following set of nine attributes?
1) 7 min - € 1.30
2) 7 min - € 2.60
3) 7 min - € 3.90
4) 21 min - € 1.30
5) 21 min - € 2.60
6) 21 min - € 3.90
7) 35 min - € 1.30
8) 35 min - € 2.60
9) 35 min - € 3.90

2. Is this an orthogonal design?

3. is this a fractional factorial design?
4. does this design allow estimating interaction-effects?
Regret in Traveler Decision Making 50
50
Answer to assignment

1. Correlation is zero
• all combination appear an equal number of times
• thus, low travel times are equally often paired with
high as with low costs

2. Yes, this an orthogonal design

3. No, this is full-factorial design

4. By definition, full-factorial designs allow estimating interaction

effects

Regret in Traveler Decision Making 51

51
 Check orthogonality of full factorial design
1 price 2 time 3 comfort
0: € 1 0: 10 min. 0: low
1: € 3 1: 20 min 1: high

full factorial

full factorial designs are always orthogonal!

Regret in Traveler Decision Making 52
52
 orthogonal fractional factorial designs

1 price 2 time 3 comfort

0: € 1 0: 10 min. 0: low
1: € 3 1: 20 min 1: high
the fractionals are orthogonal
full factorial two fractional factorials

1 0 0
1 0 0 1 1 1
1 1 1 0 0 1
0 0 1 0 1 0
0 1 0
1 1 0
1 0 1 1 1 0
0 1 1 1 0 1
0 0 0 0 1 1
0 0 0 Regret in Traveler Decision Making 53
53
 random fractional factorial design (1)

Random fraction 1

• not an orthogonal design

• will result in higher standard errors of parameters

Regret in Traveler Decision Making 54

54
 random fractional factorial design (2)

Random fraction 2

• perfect negative correlation

• extreme multicollinearity
• not possible to estimate separate effects for price and time
Regret in Traveler Decision Making 55
55
Attribute level balance

stated choice
Regret experiments
in Traveler Decision Making 56
attribute level balance
1 0 0
1 1 1
0 0 1
0 1 0
• attribute level balance:
• each attribute level appears an equal number of times
• this applies to every attribute (columns)

• why important?
• the same number of observations for every attribute level

• if dummy or effect coded (see lecture 4)

• parameters for each level have the same standard error
• equal probability of becoming statistically significant

Regret in Traveler Decision Making 57

57
Assignment – attribute level balance?

first orthogonal fractional second orthogonal fractional

random fraction 1 random fraction 2

Regret in Traveler Decision Making 58

58
Answer assignment

first orthogonal fractional second orthogonal fractional

balanced balanced

random fraction 1 random fraction 2

not balanced, balanced

(and not orthogonal) (but
Regret not orthogonal)
in Traveler Decision Making 59
59
Basic plans

stated choice
Regret experiments
in Traveler Decision Making 60
Basic plans

• published fractional factorial design schemes

• allows combining attribute levels into alternatives
• you do not need to create them, but know how to use them
• see document BB: ‘use of basic plans.doc’

• characteristics of designs
1. orthogonal
• if well applied
• basic rule: don’t select columns with same number
2. attribute level balance
• if columns properly combined
Regret in Traveler Decision Making 61
61
Some basic plans

62
• How to make a choice?
Regret in Traveler Decision Making
62
Selecting a basic plan

• starting point: attributes and levels are chosen

• select the smallest design that is able to accommodate all

selected attributes:
• establishing orthogonality
• preferably with attribute level balance

• assign each attribute to a free column:

• two-level attribute: find column with numbers (0,1)
• three-level attribute: … numbers (0,1,2)
• four-level attribute: … numbers (0,1,2,3)

• do not select columns with the same

Regretcolumn
in Traveler number
Decision Making 63
63
 Assignment

1. How many attributes can you vary with this design?

2. How many levels can these attributes have?

Basic Plan 2: 34; 24; 9 trials

Profiel 1 2 3 4 1 2 3 4

.
1 0 0 0 0 0 0 0 0
2 0 1 1 2 0 1 1 0
3 0 2 2 1 0 0 0 1
4 1 0 1 1 1 0 1 1
5 1 1 2 0 1 1 0 0
6 1 2 0 2 1 0 0 0
7 2 0 2 2 0 0 0 0
8 2 1 0 0 0 1 0 1
9 2 2 1 1 0 0 1 0
Regret in Traveler Decision Making 64
64
answer to assignment

• a maximum of 4 attributes
• 4 columns have the same number: are correlated

• the attributes can have either 3 or 2 levels

Basic Plan 2: 34; 24; 9 trials

Profiel 1 2 3 4 1 2 3 4

1 0 0 0 0 0 0 0 0
2 0 1 1 2 0 1 1 0
3 0 2 2 1 0 0 0 1
4 1 0 1 1 1 0 1 1
5 1 1 2 0 1 1 0 0
6 1 2 0 2 1 0 0 0
7 2 0 2 2 0 0 0 0
8 2 1 0 0 0 1 0 1
9 2 2 1 1 0 Regret
0 in Traveler
1 0
Decision Making 65
65
Assignment
Assume you construct flight tickets:
• price: 0=500; 1=600; 2=700 euro
• travel time: 0=6; 1=7; 2=8 hours
• comfort: 0=economy; 1=business
1. Which attribute do you assign to which column?
2. Is the design orthogonal?
3. Is the design attribute level balanced?
Basic Plan 2: 34; 24; 9 trials
.
Profiel 1 2 3 4 1 2 3 4

1 0 0 0 0 0 0 0 0
2 0 1 1 2 0 1 1 0
3 0 2 2 1 0 0 0 1
4 1 0 1 1 1 0 1 1
5 1 1 2 0 1 1 0 0
6 1 2 0 2 1 0 0 0
7 2 0 2 2 0 0 0 0
8 2 1 0 0 0 1 0 1
Regret in Traveler Decision Making 66
9 2 2 1 1 0 0 1 0 66
Answer to assignment
Basic Plan 2: 34; 24; 9 trials

Profie 1 2 3 4 1 2 3 4
l

1 0 0 0 0 0 0 0 0
2 0 1 1 2 0 1 1 0
3 0 2 2 1 0 0 0 1
4 1 0 1 1 1 0 1 1
5 1 1 2 0 1 1 0 0
6 1 2 0 2 1 0 0 0
7 2 0 2 2 0 0 0 0
.
8 2 1 0 0 0 1 0 1
9 2 2 1 1 0 0 1 0
• Assignment to columns (more options possible)
• price: 0=column 1
• travel time: column 2
• comfort: column 3 of second block
• note: you cannot use columns 1 & 2 (same column number)
• the design is orthogonal (different column numbers used)
67
• the design is not attribute level balanced (comfort is not)
Regret in Traveler Decision Making
67
 Assignment
• What are the levels of choice alternative 9?
1. price: 0=500; 1=600; 2=700 euro
2. travel time: 0=6; 1=7; 2=8 hours
3. comfort: 0=economy; 1=business
Basic Plan 2: 34; 24; 9 trials

Profie 1 2 3 4 1 2 3 4
l
.
1 0 0 0 0 0 0 0 0
2 0 1 1 2 0 1 1 0
3 0 2 2 1 0 0 0 1
4 1 0 1 1 1 0 1 1
5 1 1 2 0 1 1 0 0
6 1 2 0 2 1 0 0 0
7 2 0 2 2 0 0 0 0
8 2 1 0 0 0 1 0 1
9 2 2 1 1 0 0 1 0
Regret in Traveler Decision Making 68
68
Answer to assignment
Basic Plan 2: 34; 24; 9 trials

Profie 1 2 3 4 1 2 3 4
l

1 0 0 0 0 0 0 0 0
2 0 1 1 2 0 1 1 0
3 0 2 2 1 0 0 0 1
4 1 0 1 1 1 0 1 1
5 1 1 2 0 1 1 0 0
6 1 2 0 2 1 0 0 0
7 2 0 2 2 0 0 0 0
8 2 1 0 0 0 1 0 1
9 2 2 1 1 0 0 1 0 .

• What are the levels of choice alternative 9?

• price: column 1 → 0=500; 1=600; 2=700 euro
• travel time: column 2 → 0=6; 1=7; 2=8 hours
• comfort: column 3 → 0=economy; 1=business

price: 700 euro

travel time: 8 hours
Regret in Traveler Decision Making 69
comfort: business 69
Assignment
1. What is the best BP if you have 3
attributes with 2 levels each?

2. Why would you not use the other

BP?

3. Which BP if you selected the

following attributes:
• Price: €1, €2, €3,
• Time: 10, 15, 20 minutes
• Comfort: low, middle, high

4. Is this design suitable if you

selected 3 three-level & 2 two-level
attributes?
Regret in Traveler Decision Making 70
70
Answers to assignment
1. BP1

2. BP2 is not preferred:

1. 1 alternative more
2. no attribute level balance

3. BP2

4. No, a larger design is needed!

Regret in Traveler Decision Making 71

71
Assignment

Basic Plan 2: 34; 24; 9 trials

Profile 1 2 3 4 1 2 3 4

1 0 0 0 0 0 0 0 0
2 0 1 1 2 0 1 1 0
3 0 2 2 1 0 0 0 1
4 1 0 1 1 1 0 1 1
5 1 1 2 0 1 1 0 0
6 1 2 0 2 1 0 0 0
7 2 0 2 2 0 0 0 0
8 2 1 0 1 0 1 0 1
9 2 2 1 0 0 0 1 0

1. Is the design orthogonal if you use only 3 columns?

2. Is the design orthogonal if you use only 8 profiles?

Regret in Traveler Decision Making 72

72
Answers to assignment

Basic Plan 2: 34; 24; 9 trials

Profile 1 2 3 4 1 2 3 4

1 0 0 0 0 0 0 0 0
2 0 1 1 2 0 1 1 0
3 0 2 2 1 0 0 0 1
4 1 0 1 1 1 0 1 1
5 1 1 2 0 1 1 0 0
6 1 2 0 2 1 0 0 0
7 2 0 2 2 0 0 0 0
8 2 1 0 1 0 1 0 1
9 2 2 1 0 0 0 1 0

1. yes, if 4 attributes are orthogonal, 3 are also orthogonal

• thus you do not have to use all columns!

2. No! removing one profile will introduce correlations!

• so you must use all rows to obtain orthogonality!
Regret in Traveler Decision Making 73
73
proof

• column 4 is left out →

• attributes still uncorrelated

• row 9 is left out →

• attributes are correlated
Regret in Traveler Decision Making 74
74
Summary: characteristics of orthogonal designs

1. orthogonal
• no correlation between attributes
• low standard errors (= reliable parameters)

2. attribute level balance

• levels of an attribute appear an equal number of times
• the same standard errors

Regret in Traveler Decision Making 75

75
Summary: fractional & full factorial designs

• full factorial designs

• all possible combinations between attribute levels
• is orthogonal
+ allows estimating all main effects
+ allows estimating all interaction effects
- often too many choice situations

• orthogonal fractional factorial designs

• orthogonal selection from full factorial
+ result in much smaller number of choice sets
+ allows estimating all main effects
- no interaction effects!

Regret in Traveler Decision Making 76

76
next lecture

• constructing choice sets

• self study:
• study slides of this lecture
• read document ‘use of basic plans.doc’

Regret in Traveler Decision Making 77

77