Scribd Logo
Upload

Welcome to Scribd!

  • Causal Inference

    Uploaded by

    Afif
    19 views15 pages
    The document summarizes key concepts in impact evaluation presented by Dr. Sarah Dong at the IRSA Annual Conference workshop in Solo, Indonesia in 2018. It discusses the counterfactual approach to estimating program effects, and challenges such as the fundamental problem that the outcome for each individual under both treatment and no treatment can never be observed. It then covers different methods for approximating the counterfactual, such as matching, regression discontinuity design, differences-in-differences, and fixed effects. The document also defines average treatment effects, average treatment effects on the treated, and local average treatment effects, and discusses how these estimates may differ depending on how representative the sample is of the target population.

    Original Description:

    Original Title

    Causal inference

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    The document summarizes key concepts in impact evaluation presented by Dr. Sarah Dong at the IRSA Annual Conference workshop in Solo, Indonesia in 2018. It discusses the counterfactual approach to estimating program effects, and challenges such as the fundamental problem that the outcome for each individual under both treatment and no treatment can never be observed. It then covers different methods for approximating the counterfactual, such as matching, regression discontinuity design, differences-in-differences, and fixed effects. The document also defines average treatment effects, average treatment effects on the treated, and local average treatment effects, and discusses how these estimates may differ depending on how representative the sample is of the target population.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    19 views15 pages

    Causal Inference

    Uploaded by

    Afif
    The document summarizes key concepts in impact evaluation presented by Dr. Sarah Dong at the IRSA Annual Conference workshop in Solo, Indonesia in 2018. It discusses the counterfactual approach to estimating program effects, and challenges such as the fundamental problem that the outcome for each individual under both treatment and no treatment can never be observed. It then covers different methods for approximating the counterfactual, such as matching, regression discontinuity design, differences-in-differences, and fixed effects. The document also defines average treatment effects, average treatment effects on the treated, and local average treatment effects, and discusses how these estimates may differ depending on how representative the sample is of the target population.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    of 15

    Impact Evaluation Workshop

    IRSA Annual Conference, Solo, 2018

    Causal Inference

    Dr. Sarah Dong


    Fellow, Indonesia Project
    Australian National University
    sarah.dong@anu.edu.au
    Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 2
    The counterfactual
    What would have happened to
    beneficiaries had they not received the
    intervention?
    Outcome
    Observed from
    Effect = outcome - counterfactual

    Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 3


    The missing counterfactual
    Outcome A: Outcome B:
    Effect = receive - do not
    PKH receive PKH

    Recipient: A - B

    Non-recipient: A - B

    but never observe both A and B for the same


    household!

    Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 4


    Problem with OLS regression
    Y = αTreat+ε
    Treat =1 if receiving program
    Treat = 0 if not receiving program

    How do we interpret α?
    • α is the average difference in Y between people who
    receive the program and people who do not receive the
    program
    • These two groups of people may be different and not
    comparable

    Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 5


    Approximate for the counterfactual?
    • Find a household in non-recipients that is
    similar to the household in recipients
    • The similar household will be the
    counterfactual

    Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 6


    What is a similar household in the non-
    recipient pool?
    • Similar in observed characteristics
    – OLS, Matching
    • Similar in both observed and unobserved
    characteristics
    – RCT, RDD, IV
    • Similar in change overtime
    – DID
    • Same household in a pre-program time
    – Panel fixed effect
    Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 7
    Exercise: good counterfactual vs. bad
    counterfactual vs.

    Program: PKH in Papua


    Outcome: education of children in the recipient household
    Unit: household
    • Counterfactual 1: Non-recipient households in the same
    village
    • Counterfactual 2: Non-recipient household with similar
    size, income, household composition in Jakarta
    • Counterfactual 3: Education of children in the same
    household 5 years ago before introduction of PKH
    • Ideal counterfactual?

    Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 8


    Exercise: good counterfactual vs. bad
    counterfactual vs.
    Program: Dana desa
    Outcome: average per capita expenditure of the village
    Unit: village
    • Counterfactual 1: Villages that receive a lower amount of
    village fund
    • Counterfactual 2: Urban areas that do not receive village
    fund
    • Counterfactual 3: The same village when there was no
    village fund
    • Ideal counterfactual?

    Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 9


    Exercise: good counterfactual vs. bad
    counterfactual vs.

    Program: Tax amnesty


    Outcome: tax revenue
    Unit: national
    • Counterfactual 1: tax revenue in the same months in the
    year before the amnesty
    • Counterfactual 2: tax revenue in the same months in the
    year after the amnesty
    • Ideal counterfactual?

    Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 10


    The treatment effects
    • Average treatment effect (ATE)

    • Average treatment effect on the treated


    (ATT)

    • Local average treatment effects (LATE)


    (for later: IV and RDD)

    Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 11


    Average treatment effect
    Outcome B1:
    Outcome A1: do not
    Individual 1: treatment effect1 = receive receive
    program program

    Individual 2: treatment effect2 = Outcome A2 Outcome B2

    Individual N: treatment effectN = Outcome AN Outcome BN

    Average treatment effect = average(treatment effect1, …, treatment effectN)


    Note: N is the population

    Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 12


    Average treatment effect on the treated
    Individual 1: treatment effect1 = Outcome A1: Outcome B1:

    Treated …
    Individual N1: treatment effectN1 = Outcome AN1 Outcome BN1

    Outcome Outcome
    Individual N1+1: treatment effectN1+1 = BN1+1
    AN1+1
    Not
    Treated …
    Individual N: treatment effectN = Outcome AN Outcome BN

    Average treatment effect on the treated =


    Average(treatment effect1, …, treatment effectN1)

    Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 13


    ATE vs. ATT
    • ATE is usually unobtainable, unless
    – Sample is representative from population
    – Random allocation between treated and
    untreated
    • ATT is usually what we get
    • Therefore should state for what group is
    the estimated treatment effect for

    Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 14


    Exercise: defining treatment effects
    For PKH, define:
    1. The ATE
    2. The ATT
    3. The potential difference between ATE and
    ATT

    Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 15

    You might also like