Causal Modeling for Business Analytics

Individual Assignment

Due on May 7th 2024 at 11:59pm

You are allowed to discuss with your classmates. However, you must complete and
submit your assignment individually.

1. You are tasked to implement a randomized experiment in order to test the effectiveness
(mea- sured in grade point averages, “GPA”) of a new set of schooling material for high
school students. At one high school, you are planning to randomly give the material to
some subset of students, but not other students at that same school. From historic data
(i.e. before the intervention) you know that the variance of the GPA score is equal to
1.3. The organization expects an increase in GPA scores of 0.1 due to the intervention.
Assume that the treatment / control proportions are equal to 80% / 20%. How large
does the sample size have to be in order to be able to distinguish an effect of this size
from a null effect at the 95 percent level of confidence?

2. You are trying to optimize pricing for a popular product sold at a large number of
stores. You implement an A/B test where you randomly vary price. However, you are
only able to randomly increase or decrease prices for a given store relative to the price
levels each store had historically charged. Because prices differed across stores in the
past, the A/B test only generates random variation in prices WITHIN stores, whereas
price differences across stores are not random. Stores also occasionally advertise the
product (coded up as a dummy variable). Advertising is not randomly assigned. The
exercise below is based on a panel data set (price ABtest) of prices and sales at the
week/store level (with randomized prices being determined as described above).

(a) Randomization checks

i. Load the data and regress price on advertising. Then run a second regression
where you regress price on advertising as well as a set of store fixed effects.
Is the advertising coefficient significant in either regression? Explain which of
the two specifications constitutes the appropriate randomization check given
the way in which the A/B test was implemented.
ii. Run a regression of price on week dummies as well as store fixed effects. You
will notice that several coefficients on the week dummies are statistically signif-
icant. Is this finding sufficient to conclude that price is not randomly assigned
within stores?

1
(b) Main regression
i. Regress sales on price as well as store fixed effects. Interpret the price coeffi-
cient. Does the price coefficient have a causal interpretation in this regression?
ii. Run the same regression as in (a) without store fixed effects. Explain why the
price coefficient does not have causal interpretation in the regression without
fixed effects. Use the omitted variables bias formula to explain the direction
of the bias in the coefficient obtained from the regression without fixed effects.
Assume that the only variables that is systematically correlated with price
across stores is advertising.

(c) Interaction terms

i. Run a regression that allows you to assess whether price sensitivity differs
across small and large stores. Also allow for a different impact of price in
summer versus all other weeks (use the regression framework with store fixed
effect used above). Make sure to incorporate the appropriate interaction effects
as well as baseline controls. Explain which baseline controls are needed in this
regression.
ii. Based on your findings above, at what types of stores and during which time
periods should we charge higher prices?