Context-dependent experiments

Lecture 5 (SEN 1221, part II)

2-1-2020
Eric Molin – TU Delft

Delft
University of
Technology

Challenge the future

Context

stated choice
Regret experiments
in Traveler Decision Making 2
Context

• “Context is the physical, socioemotional, and mental setting in

which behavior takes place”
• description of the choice situation
• what respondents need to assume while making choices
• defines ‘the playing field’

• “Context is the entire framework and set of factors describing the

objective and subjective circumstances that surround and
influence action by an individual and/or a group”
• Goulias & Pendyala, handbook of choice modeling (p.101)

Regret in Traveler Decision Making 3

3
context is important

example of context elements

• in SC research, alternatives
are hypothetical

• therefore, you have to make

clear to respondents what is
the same and what differs
from normal circumstances

• note: the model is only valid

for the context you define

Regret in Traveler Decision Making 4

4
Examples

• e.g. mode choice:

• purpose
• “Assume you are traveling to your work location in the
morning…”

• e.g. holiday travel:

• destination?
• is the decision to fly already made?
• “Assume you are flying to a holiday destination outside
Europe…”

Regret in Traveler Decision Making 5

5
impact context variables

• if context= ‘Image you travel to work, you leave at your normal time’
• respondent assumes usual conditions:
• crowdedness s/he typically experiences
• (probably) traveling alone
• destination is clear
• respondents make all choices with this trip in mind

• standard approach: model impact ‘revealed’ context

• measure all personal trip characteristics / circumstances
• test whether mode choice differs among the context var.

Regret in Traveler Decision Making 6

6
limitations of measuring revealed context

• limitations:
• only possible if sufficient variation among respondents
• contexts that occur infrequently will be statistically
insignificant
• no intra-person variation observed
• would they make other choices in other contexts?

• solution
• vary context hypothetically by design
• observe choices for different contexts

Regret in Traveler Decision Making 7

7
Constructing context-dependent experiments

stated choice
Regret experiments
in Traveler Decision Making 8
including contexts in SC experiments

• Construct two different experiments:

1. contexts profiles
• e.g. weather, trip purpose, baggage, travel company
2. choice experiment
• regular alternatives in choice sets

• nest choice sets under the context profiles

• observe all choice sets for all context profiles
• in a balanced way

Regret in Traveler Decision Making 9

9
simplified example

• construct weather contexts profiles (4):

• temperature: warm or cold
• perspiration: dry or rain

• construct mode choice sets (4):

• suppose two labeled alternatives: car & train
• generic travel time: 20 or 30 minutes
• generic travel costs: 4 or 5 euro

• nesting
• 4 x 4 = 16 context-choice set combinations

Regret in Traveler Decision Making 10

10
all 16 context-mode choice sets

rainy & cold 1 mode set 1 car 20 min. $5 train 30 min. $4

(context 1) 2 mode set 2 car 30 min. $5 train 20 min. $5
3 mode set 3 car 20 min. $4 train 20 min. $4
4 mode set 4 car 30 min. $4 train 30 min. $5

rainy & warm 5 mode set 1 car 20 min. $5 train 30 min. $4

(context 2) 6 mode set 2 car 30 min. $5 train 20 min. $5
7 mode set 3 car 20 min. $4 train 20 min. $4
8 mode set 4 car 30 min. $4 train 30 min. $5

dry & cold 9 mode set 1 car 20 min. $5 train 30 min. $4

(context 3) 10 mode set 2 car 30 min. $5 train 20 min. $5
11 mode set 3 car 20 min. $4 train 20 min. $4
12 mode set 4 car 30 min. $4 train 30 min. $5

dry & warm 13 mode set 1 car 20 min. $5 train 30 min. $4

(context 4) 14 mode set 2 car 30 min. $5 train 20 min. $5
15 mode set 3 car 20 min. $4 train 20 min. $4
16 mode set 4 car 30 min. $4 train 30 min. $5
Regret in Traveler Decision Making 11
11
Examples of context-choice sets
• Imagine it is dry and warm weather

set 13
- context 4
- mode set 1

• Imagine it is rainy and cold weather

Regret in Traveler Decision Making 12

12
Modeling context effects

stated choice
Regret experiments
in Traveler Decision Making 13
Assignment

1. Do context variables vary within the same choice set?

2. Can they therefore be included in utility functions in the same

way as attributes?

Regret in Traveler Decision Making 14

14
Answer to assignment

1. no, context variables have the same level for all alternatives in a
choice set
• it would be nonsense to assume warm weather in one
alternative and freezing cold in another alternative

2. no, they cannot be included as main effects, but need to be

included as interaction effects:
• with constants
• with attribute parameters

Regret in Traveler Decision Making 15

15
effect context variables on mode choice

time & mode

choice
costs utility

temperature
perspiration

context variables temperature & perspiration may…

… change the mode constants:
• e.g. car may be more attractive in rainy, cold weather

… change the weight of time & costs on utility

• e.g. costs may be less important in rainy,
Regret in Traveler cold
Decision weather16
Making
16
interaction effects

• context variables may affect

• constants
• e.g. in rainy conditions car may be more preferred
• thus attributes not varied in the experiment are affected
• parameters
• e.g. car cost less important in rain
• thus more willing to pay to be sheltered from rain

• in utility function:
• context effects may modify constants and parameters
• are included as interaction effects

• typically contexts are effects coded (-1,in Traveler

Regret +1) Decision Making 17
17
context effects on constant
perspiration: rainy +1, dry -1
temperature: cold +1, warm -1

Vcar = βcar*C + βc*costs + βti*time

Actually not needed
+ βp*perspiration*C + βt* temperature*C
as C=1
• βcar constant: base preference for car (compared to train)
• C = coding of constant = 1

• βp change in car constant due to perspiration

• βt change in car constant due to temperature

Regret in Traveler Decision Making 18

18
Assignment
• If context interaction effects are statistically significant, we may
formulate a different utility function for each context
• the relevant part for the constant in the utility function:
• βcar + βp*perspiration + βt* temperature

• assume:
• estimates: βcar=2, βp =1, βt = 0.5
• coding context variables: rain +1, dry -1; cold +1, warm -1

1. What is the car constant in rainy, cold weather?

2. And in dry, warm weather?
3. And in dry, cold weather?
4. And in rainy, warm weather?
5. What is the average constant across all contexts?
Regret in Traveler Decision Making 19
19
Answer to assignment

• the relevant part for the constant in the utility function:

• βcar + βp*perspiration + βt * temperature
• 2 + 1* perspiration + 0.5* temperature
• coding: rain +1, dry -1; cold +1, warm -1
car constant
1. in rainy, cold weather βcar + 1*1 + 0.5*1 = 3.5
2. in dry, warm weather βcar + 1*-1 + 0.5*-1 = 0.5
3. in dry, cold weather βcar + 1*-1 + 0.5*1 = 1.5
4. in rainy, warm weather βcar + 1* 1 + 0.5*-1 = 2.5

5. the average constant across all contexts = 2 = overall constant

• because of effects coding (+1, -1)
• only true if all contexts levels are balanced in the experiment

20
• main context effects are interactions with the constant!
Regret in Traveler Decision Making
20
Assignment
The interaction perspiration*costs indicates how the cost
parameter changes with perspiration
Vcar = βcar + βp*perspiration + βt*temperature
+ βc *costs + βpc*perspiration*costs
+ βti *time

• βc main cost parameter;

• βpc change in cost parameter due to perspiration;

• assume βc= -1 and βpc = 0.5

• Are travelers more sensitive to car costs in rainy (+1) or in dry (-1)
weather conditions?

Regret in Traveler Decision Making 21

21
answer to assignment

• relevant part of utility function:

• βc*costs +βpc*perspiration*costs = (βc +βpc*perspiration)*costs

• rain (perspiration=1): (-1*+0.5*1)*costs = -0.5*costs

• dry (perspiration= -1): (-1*+0.5*-1)*costs = -1.5*costs

1. thus more sensitive to costs in dry weather condition

• If the parameter would be alternative specific, thus estimated

for car, this appeals to intuition, because car provides door-to-
door transportation
• typically train travel results in more exposure to weather
conditions: more walking, outside waiting, etc.
• Hence, because car provides relatively more protection from
weather, one is willing to pay more in bad weather conditions,
thus less sensitive to costs in rainy
Regretconditions
in Traveler Decision Making 22
22
model per context

• alternatively: estimate a separate model for each context

• simple and gives the same results

• however:
• we cannot test whether differences between contexts are
statistically significant
• and may end up with too many parameters

• thus: include context variables as interactions in single model

• allows testing whether context variables have statistically
significant impact
• which may result in more parsimonious models
Regret in Traveler Decision Making 23
23
 effects coded context impacts

• main context parameter:

• expresses difference of context level with constant
+ interaction parameter for +1 coded group
- interaction parameter for -1 coded group

• the estimated constant

• is the mean (average) constant across all contexts

• if context levels are balanced in the experiment:

• every level appears an equal number of times
• constant is unweighted average
• hence, this average only reflects true average, if the
context levels appear an equal number of times in reality
Regret in Traveler Decision Making 24
24
Assignment

• suppose we found:
• Main cost parameter = -1
• In rainy conditions cost parameter = -0.5
• In dry conditions cost parameter = -1.5

1. Is the main cost (-1) parameter found in this experiment

representative for the cost parameter in the real world?

• Hint: think about how often each context level appears in

the experiment, and compare this with how often the levels
occur in reality.

• So what do you need to do to find a more representative

cost parameter?

Regret in Traveler Decision Making 25

25
answer to assignment

• Main cost parameter = -1

• In rain conditions cost parameter = -0.5
• In dry conditions cost parameter = -1.5

1. No, the value is determined by the frequency of the context

levels in the experiment:
• every level appears exactly 8 times (slide 12)
• thus: 50% of all trips in dry cond. and 50% in rainy cond.
• while in reality, it rains only 6% at any time in the Nl. !

• thus, in context dependent experiments, the main parameters do

not reflect real parameters!
• an approximation: reweight by the proportion occurrence
of the context levels in reality (e.g. 0.06 rain, 0.94 dry)
• 0.06*-0.5 + 0.94*-1.5 = -1.44
Regret in Traveler Decision Making 26
26
An illustration: a context-dependent multi-
modal choice experiment

Arentze, T.A. and E.J.E. Molin (2013), Travelers’ preferences in

multimodal networks: design and results of a comprehensive series
of choice experiments, Transportation Research Part A, 58, 15-28

stated choice
Regret experiments
in Traveler Decision Making 27
Aim of study

• multi-modal transport systems

• seamlessly combining various transport modes
• often regarded as an alternative for car travel

• choice depends on:

1. attributes mode alternatives
2. trip conditions = contexts

• focus study:
• daily and non-daily trips for non-work activities
• in metropolitan region (the Randstad)
• uni- & multi-modal trips
Regret in Traveler Decision Making 28
28
Constructing the mode choice experiment

stated choice
Regret experiments
in Traveler Decision Making 29
Complexity of
multi-modal
choices

Regret in Traveler Decision Making 30

30
Challenges

• Many alternatives

• Many attributes
• Attributes are alternative specific

• Trips may be uni-modal (bike) or

• multimodal (walk, bus, train, tram, walk)

• Distances in Randstad differ widely (5 – 65 km)

• How to obtain realistic time-cost combinations?

• Do car travelers consider PT?

• Do PT travelers consider car?
Regret in Traveler Decision Making 31
31
Solution
• four experiments constructed:

1) 5 km – trips within city

• uni-modal choices: car or PT or bicycle

2) 20 km. – trips between neighboring cities

• only PT

3) 20 km. – trips between neighboring cities

• car, P&R, PT

4) 65 km. – trips between edges of metropolitan region

• only PT:
• train always main mode
• PT & Bicycle as access & egress modes
• in other 3 experiments
Regret only walk
in Traveler Decision Making 32
32
Example: 20 km. – multi-modal experiment

Regret in Traveler Decision Making 33

33
Example of mode choice set

Regret in Traveler Decision Making 34

34
constructing the context profiles

stated choice
Regret experiments
in Traveler Decision Making 35
Context experiment

• context profiles = combinations of context variables

• trip purpose
• travel party (alone, with someone else)
• weather
• rush hour (or not)
• baggage
• flexibility arrival time

Regret in Traveler Decision Making 36

36
context variables, levels and coding

i1 i2 i3 i1 i2 i3
baggage travel party
small bag 1 with friend 1 0 0
heavy bag -1 with child 0 1 0
flexibility arrival time with partner 0 0 1
inflexible 1 alone -1 -1 -1
flexible -1 weather
time of day rainy & cold 1 0 0
no rush hour 1 rainy & not cold 0 1 0
rush hour -1 dry & cold 0 0 1
trip purpose dry & not cold -1 -1 -1
hospital 1 0 0
day out 0 1 0
business 0 0 1
visit -1 -1 -1
Regret in Traveler Decision Making 37
37
8

Assignment

• 3 context variables varying in 2 levels

• 3 context variables varying in 4 levels

1. How many profiles in a full factorial design?

2. Do we need all combinations?
• What can’t we model if we select an orthogonal fractional
factorial design (see basic plans)?
3. How many profiles if we apply a fractional factorial design?
4. Are main effects of context variables independent of interaction
effects?
5. Can we improve the design?

Regret in Traveler Decision Making 38

38
9

construction of experiment

1. full-factorial: 23 * 43 = 512 profiles

2. we are not interested in interaction-effects

• they may play a role, but too complicated to estimate

3. orthogonal fractional factorial design –

• basic plan 3: 16 profiles (= context descriptions)

4. context effects are not independent of their interactions

• they may bias the context effects

5. extending the design with its foldover design

• in total 32 context descriptions
Regret in Traveler Decision Making 39
39
foldover designs

stated choice
Regret experiments
in Traveler Decision Making 40
orthogonal fractional factorial - reconsidered

design
1 2 3 1 price 2 time 3 comfort
1 -1 -1 -1: € 1 -1: 10 min. -1: low
1 1 1
1: € 3 1: 20 min 1: high
-1 -1 1
-1 1 -1
this design is orthogonal

1) € 3, 10 min, low comfort

2) € 3, 20 min, high comfort
3) € 1, 10 min, high comfort
4) € 1, 20 min, low comfort
Regret in Traveler Decision Making 41
41
Assignment

1. Add a column for interaction time * comfort

1 2 3
time price comf
1 -1 -1
1 1 1
-1 -1 1
-1 1 -1

2. Compare this coded column with column 2 (price)

3. What is the consequence of this with respect to estimating

parameters?

4. Thus, what is a potential problem of fractional factorial

designs? Regret in Traveler Decision Making 42
42
Answer to assignment

1. 1 2 3 time * comf
time price comf
1 -1 -1 -1
1 1 1 1
-1 -1 1 1
-1 1 -1 -1
2. Columns are the same, thus correlation = 1 = multicollinearity

3. Price and interaction time*comfort are completely confounded

• hence, if interaction time*comfort played a role, the price
parameter is biased
• e.g. comfort may be more important if travel time is long
• problem: we don’t know and we cannot test

4. in fractional factorial designs main effects may not be independent

of interaction effects
43
• this may bias parameters!Regret in Traveler Decision Making 43
solution: adding foldover design

• foldover design
• fractional factorial design is “mirrored”
• two levels: 0 → 1 and 1 → 0
• three levels: 0 → 2, 1=1, 2→ 0
• four levels: 1→ 4, 2→3, 3→2, 4→1
• etc.

basic plan 0 foldover of basic plan 0

1 0 0 0 1 1
1 1 1 0 0 0
0 0 1 1 1 0
0 1 0 1 0 1

Regret in Traveler Decision Making 44

44
pros and cons of foldover designs

• advantage:
• main effects are uncorrelated with (two-way) interactions!
• so (two-way) interactions cannot bias main parameters

• disadvantage
• the number of choice situations doubles

• always add?
• no, only if it is likely that interactions bias the results

Regret in Traveler Decision Making 45

45
syntax for context profiles
Design
; alts = ctx,c
; orth = sim
; foldover
; rows = 16
; model:
U(ctx) = a1*purpose[1,2,3,4] +
a2*flexibility[1,2] +
a3*timeofday[1,2] +
a4*travelparty[1,2,3,4] +
a5*weather[1,2,3,4] +
a6*luggage[1,2]
$

Regret in Traveler Decision Making 46

46
design

Regret in Traveler Decision Making 47

correlations
• all correlations between main and interaction effects are zero

Regret in Traveler Decision Making 48

constructing context – mode choice sets

stated choice
Regret experiments
in Traveler Decision Making 49
0

Constructing context-choice combinations

• For case multi-modal choice in Randstad

• random selection of choice situations per respondent

• context description was randomly drawn
• combined with 3 randomly drawn mode choice sets
• repeated 3 times, thus 9 choice sets per person

• Note
• combination of context and choice sets is not controlled by
design
• Ngene allows optimization
• but obtaining priors for this complex experiment was
impossible Regret in Traveler Decision Making 50
50
1

example choice task

Regret in Traveler Decision Making 51

51
Assignment

main
parameter (t-values)

• What is the effect of carrying a heavy bag on:

• time (t_) parameters?
• cost (c_) parameter (c_ticket)?
• inclination to use PT bike (a_egress_bike)
• inclination to make a transfer (a_transfer)
• Are these effects plausible? Regret in Traveler Decision Making 52
52
Answer to assignment

main
parameter
(t-values)

• when carrying a heavy bag:

• decreases sensitivity to time & costs components
• less inclined to use PT bike (for egress transport)
• more strongly averse to making a transfer
• all plausible effects Regret in Traveler Decision Making 53
53
summary context dependent experiments

• construct two experiments

1. context experiment
• profiles varying context variables
2. choice experiment
• regular choice experiment

• nesting
• nest all choice sets under all context profiles
• in a balanced way

• modeling
• by interaction effects:
• context variables * constant / parameters
Regret in Traveler Decision Making 54
54
Further reading

• Context effects in multimodal choices

• Molin, E.J.E and T.A. Arentze (2014), The Influence of Trip
Context on Multi-modal Travel Choices, compendium of
papers of the 93-th annual meeting of the Transportation
Research Board, Washington D.C.

• Context-dependent experiment & complex variables (HII)

• Bos, Ilona D.M., Rob E.C.M. Van der Heijden, Eric J.E.
Molin and Harry J.P. Timmermans (2004), The Choice of
Park & Ride Facilities: An Analysis Using a Context-
Dependent Hierarchical Choice Experiment, Environment
and Planning A, 36(9), 1673 – 1686

Regret in Traveler Decision Making 55

55
remaining lectures

• Lecture 6
• methodological reflection
• overview of experimental design options not covered

• Q&A
• pose any questions you have on my part of the course,
including the example exam question (see BS)

Regret in Traveler Decision Making 56

56