Discussion: Econometrics The Counterfactual Model

Discussion: Econometrics The Counterfactual Model ORDER NOW FOR CUSTOMIZED AND ORIGINAL ESSAY PAPERS ON Discussion: Econometrics The Counterfactual Model I need help with a question. All explanations and answers will be used to help me learn. Plz choose this assignment only if you how to do it. This assignment is related to counterfactual model and naive estimator (Econometric), all the requirement and report are in the attached files, as well as the lecture slides. Discussion: Econometrics The Counterfactual Model econ803_assignment.pdf _the_counterfactual_model.pptx _naive_estimator.pptx Policy assignment A review of “the review of the sentence of home detention 2007-2011” Due date: Friday, March 22th, 12’noon Hand-in: WY215 or email: [email protected] The purpose of this assessment is to write a guided review of the Ministry of Justice report “Home Detention – A review of the sentence of home detention 2007-2011”. The report has a total word limit of 1500 words and needs to be completed individually. In 2007 the New Zealand Government introduced home detention as a stand-alone sentence. The policy change meant that eligible offenders sentenced to short-term imprisonment could serve their sentence at an approved residence under electronic monitoring. As part of the 2008 New Zealand general election debate the National Party committed to an evaluation of the “appropriateness” of home detention sentence. In October 2011 the Ministry of Justice (MoJ) released the report “Home Detention – A review of the sentence of home detention 2007-2011”. The report can be found here: https://thehub.sia.govt.nz/assets/documents/41113_A_review_of_the_sentence_of_home _detention_2007-2011_0.pdf 1. (~350 ????????????????????). Write a short summary of the MoJ report. Please make sure you include information on: a. Research question(s) b. Data c. Method d. Conclusion 2. (~150 ????????????????????). Re-state the (main) MoJ research question (from Q1) using the notation from the lecture on the counterfactual framework. 3. (~500 ????????????????????). Discuss the empirical strategy in the MoJ report. Make sure to use the notation from the lecture on the counterfactual framework. You are encouraged to derive and sign any possible bias in the MoJ estimate(s) and discuss what assumptions could justify the MoJ approach. 4. (~500 ????????????????????). In an imaginary world without any restrictions what would, in your view, be the ideal experiment to identify the causal effect of home detention? Note that (~ ????????????????????) is a suggestion, feel free to distribute the 1500 words differently if that works better for your answers. ECONOMETRICS Econ803 The Counterfactual Model Peer Ebbesen Skov Motivation Consider the following question: • What is the effect of class size on the educational achievements? • What is the effect of an increase in the minimum wage on employment? • What is the effect of terrorism on economic growth?Discussion: Econometrics The Counterfactual Model • What is the effect of increased cigarette excise tax on smoking? • What is the effect of marriage on crime? The answers to these questions (and many others which affect our daily life) involve the identification and measurement of causal links: an old problem in philosophy and statistics. We need a framework to study causality 2 A formal framework to think about causality We have a population of units; for each unit we observe a variable ?? and a variable ??. We observe that ?? and ?? are correlated. Does correlation imply causation? Reference: David Card. Chapter 30 in Handbook of Labor Economics, 1999, vol. 3, Part A, pp 1801-1863. Elsevier. 3 A formal framework to think about causality Does correlation imply causation? In general no, because of: • • Confounding factors; Reverse causality. We would like to understand in which sense and under which hypothesis one can conclude from the evidence that ?? causes ??. 4 Terminology ?? index for the units in the population under study ???? is the (binary) treatment status where ???? = ? 1 ???????? ?? ????? ???????? ?????????????? ???? ?????????????????? 0 ???????? ?? ????? ?????? ???????? ?????????????? ???? ?????????????????? ???? ???? indicates the potential outcome according to treatment where ???? ???? = ? ???? 1 ???? 0 ?????????????? ???? ???????? ???? ?????????????????? ?????????????? ???? ???????? ???? ???? ?????????????????? The observed outcome for each unit can be written as: ???? = ???? ???? 1 + 1 ? ???? ???? 0 This forces us to think in terms of counterfactuals or socalled potential outcomes. Note that the terminology is borrowed from experimental analysis. 5 The fundamental problem of causal inference DEFINITION: Causal effect For a unit ?? the treatment ???? has a causal effect on the outcome ???? if the event ???? = 1 instead of ???? = 0 implies that ???? = ???? 1 instead of ???? = ???? 0 . In this case the causal effect of ???? on ???? is ??? = ???? 1 ? ???? 0 The identification and the measurement of this effect is logically impossible. PROPOSITION 1: The Fundamental Problem of Causal Inference It is impossible to observe for the same unit ?? the values ???? = 1 and ???? = 0 as well as the values ???? 1 and ???? 0 and therefore it is impossible to observe the effect of ?? on ?? for unit ?? (Holland, 1986) Another way to phrase this problem is to say that we cannot infer the effect of a treatment because we do not have the counterfactual evidence, i.e. what would have happened in the absence of treatment. 6 The fundamental problem of causal inference ?? 1 ?? 0 Treatment status (?? = 1) Treatment status (?? = 0) It is not possible to observe the potential outcome under the treatment state for those observed in the control state. Just like you cannot observe the potential outcome under the control state for those observed in the treatment state. (Morgan and Winship, 2007). Discussion: Econometrics The Counterfactual Model Yet another way of thinking about it: even if you have access to all individuals level values of ???? in the population you only observe half of the information you need: individuals contribute information only from the treatment state in which they are observed. (Morgan and Winship, 2007). 7 Unit homogeneity solution One “solution” to the missing counterfactual problem is to assume unit homogeneity: ??? = ? for all ?? If ???? 1 and ???? 0 are constant across individual units, then cross-sectional comparisons will recover ? = ??? If Yi 1 and Yi 0 are constant across time, then before-and-after comparisons will recover ? = ?i While this might work in physical sciences the assumption seems highly unlikely to be realistic in social sciences Before we get to the statistical ‘solution’ let’s digress for a moment and consider another challenge… Note the following notation: ?????? ? Yi ?? 8 Stable Unit Treatment Value Assumption (SUTVA) Recall the observed outcome for each unit can be written as: ???? = ???? ???? 1 + 1 ? ???? ???? 0 This notation implicitly makes the following assumption: SUTVA: ?? ??1, ??2 , … , ???? ?? = ?? ? ??1? ,??2? , …,???? ?? ???? ???? = ????? In other words: ? There is no interference between units: ? ? Potential outcomes for a unit most not be affected by treatment for any other units. Spill-over effects, contagion, dilution No different versions of treatment ? Nominally identical treatments are in fact identical Variable levels of treatment, technical errors 9 Causal Inference without SUTVA Let ?? = ??1 , ??2 be a vector of binary treatments for N = 2 How many different values can D possible take? How many potential outcomes for unit 1? How many causal effects for unit 1? How many observed outcomes for unit 1? ??1 = ?? ??1 ,??2 1 Without SUTVA, causal inference becomes exponentially more difficult as N increases 10 The statistical solution Statistics proposes to approach the problem by focusing on the average causal effect for the entire population or for some interesting sub-groups.Discussion: Econometrics The Counterfactual Model The effect of treatment on a random unit (ATE): ?? ??? = ?? ???? 1 ? ???? 0 = ?? ???? 1 ? ?? ???? 0 Or equivalently ?? 1 ?? ??? = ? ??1?? ? ?????? ?? ??=1 Note that ATE ?? ??? is still unidentified. The majority of this paper is devoted to various assumptions under which we can identify ATE from observed information. 11 The statistical solution Statistics proposes to approach the problem by focusing on the average causal effect for the entire population or for some interesting sub-groups. The effect of the treatment on the treated (ATT): ?? ??? |???? = 1 = ?? ???? 1 ? ???? 0 |???? = 1 = ?? ???? 1 |???? = 1 ? ?? ???? 0 |???? = 1 Or equivalently ??1 ?? ??=1 1 1 ?? ??? = ? ??1?? ? ?????? where ??1 = ? ???? ??1 When would ?????? ? ??????? When ???? and ?????? are associated. Exercise: define the treatment on control units (ATC). 12 The statistical solution Statistics proposes to approach the problem by focusing on the average causal effect for the entire population or for some interesting sub-groups. The conditional average treatment effect (CATE) ?? ??? |???? = ?? = ?? ???? 1 ? ???? 0 |???? = ?? 13 Illustration: Average Treatment Effect Suppose we observe a population of 4 units: i ???? ???? 1 1 3 2 1 1 3 0 0 4 0 1 What is ATE: ?? ??? = ?? ???? 1 ?????? ?????? ? ?? ???? 0 ??? ? Naïve estimator: ATE: ?? ??? = ?? ???? |???? = 1 ? ?? ???? |???? = 0 (Note this is the observed difference in means) ?? ??? = Is this (likely) an unbiased estimate of ATE? Let’s expand the table 14 Illustration: Average Treatment Effect Suppose we observe a population of 4 units: i ???? ???? 1 1 3 2 1 1 3 0 0 4 0 1 What is ATE: ?? ??? = ?? ???? 1 ?????? ?????? ? ?? ???? 0 ??? ? Naïve estimator is likely biased i.e. we over/under estimate the average treatment effect. To obtain an unbiased estimate of the ATE we need potential outcomes that we do no observe: But suppose we did – let’s complete the expanded table 15 Illustration: Average Treatment Effect Suppose we observe a population of 4 units: i ???? ???? 1 1 3 2 1 1 3 0 0 4 0 1 What is ATE: ?? ??? = ?? ???? 1 ?????? ? ?? ???? 0 ?? ???? 1 = ?? ???? 0 = ? ?? ???? 0 = ?? ??? = ?? ???? 1 ? ???? 0 = ?? ??? = ?? ???? 1 ?????? ??? ? or 16 Illustration: Average Treatment Effect Suppose we observe a population of 4 units: i ???? ???? 1 1 3 2 1 1 3 0 0 4 0 1 ?????? ?????? ??? What is ATT: ?? ??? |???? = 1 = ?? ???? 1 ? ???? 0 |???? = 1 ?? ??? |???? = 1 = Why is the ?????????? ?????????????????? = ?????? ? ??????. Will this always be the case? 17 Is comparison by treatment status informative? Discussion: Econometrics The Counterfactual Model A comparison of outcome between treatment status (the naïve estimator) often gives a biased estimate of the ATT: ?????? = ?? ???? 1 |???? = 1 ? ?? ???? 0 |???? = 1 ?????????? ?????????????????? = ?? ???? 1 |???? = 1 ? ?? ???? 0 |???? = 0 = ?? ???? 1 |???? = 1 ? ?? ???? 0 |???? = 1 + ?? ???? 0 |???? = 1 ? ?? ???? 0 |???? = 0 = ?????? + ?? ???? 0 |???? = 1 ? ?? ???? 0 |???? = 0 Note ?????? = ?? ??? |???? = 1 and the second term ?? ???? 0 |???? = 1 ? ?? ???? 0 |???? = 0 is often referred to as sample selection bias. The difference between the left hand side (which we can estimate) and ?????? is the sample selection bias equal to the difference between the outcomes treated and control subjects in the counterfactual situation of no treatment (i.e. at the baseline). The problem is that the outcome of the treated and the outcome of the control are not identical in the notreatment situation. 18 Is comparison by treatment status informative? ? ? 1. Causal inference requires a good identification strategy The treatment assignment mechanism determines whether average causal effects are identifiable Treatment is randomized by the researcher: 1. 2. 3. 2. Natural experiments 1. 2. 3. 4. 3. Birthday cut-offs Weather Close elections Arbitrary administrative rules/policy Treatment is “as-if” random after statistical control 1. 2. 4. Laboratory experiments Survey experiments Field experiments Marriage (controlling for age ,education and income) Earnings (controlling for age, education and experience) Treatment is self-selected and no plausible control is available. 19 ECONOMETRICS Econ803 The Naïve Estimator Peer Ebbesen Skov Livvy Mitchell Naïve Estimator Naïve estimator for ATT Compares outcomes of participants (D=1) and nonparticipants (D=0) as follows: ?????????? ?????????????????? = ?? ?? ?? = 1 ? ?? ?? ?? = 0 It is unbiased under the assumption of no selection bias (on observed and/or unobserved characteristics) whereby: ?? ??0 ?? = 1 = ?? ??0 ?? = 0 Generally we don’t believe that to be the case. 2 Naïve Estimator Naïve estimator for ATT The ?????????? ?????????????????? is unbiased under the assumption of no selection bias on observed and/or unobserved characteristics: We generally distinguish between two source of bias -Discussion: Econometrics The Counterfactual Model Differences in observed characteristics – Non-overlap (B1) – Different distribution of observables (wrong weighting scheme) (B2) – Selection on unobserved characteristics – Omitted variable bias, e.g., ‘ability bias’ (B3) 3 Naïve Estimator Assessing comparability of groups in terms of observables ?? ?? ?????????????? =? = ?? ?? ???????????????????? But, how can you compare the joint empirical distribution of all the X’s between two samples? Instead.. (a) Variable-by-variable measures – – Moments: means, variances Empirical distributions: densities, CDF, boxplots (b) Overall measures across all X’s – – Or, across some X’s have interaction terms pstest allows factor variables (e.g., foreign##c.age) 4 Naïve Estimator Propensity-Score matching (“pstest”) Use pstest to easily compare the characteristics in two groups: pstest var_list , raw treated (treated) scatter Use it also to quickly graph the non-parametric density or boxplot of a continuous variable (“var”) for two groups: pstest var [if] , raw treated (treated) density|box 5 Naïve Estimator Variable-by-variable: pstest output 6 Naïve Estimator Summaries: pstest output 7 Naïve Estimator Continuous variables: pstest output 8 Naïve Estimator Overall indicators: pstest output 9 Naïve Estimator How do we get a credible counterfactual? ? If no convincing comparison group exists, fancy statistical work can’t recover the true impact. ? Robustness checks Different methods differ in: ? ? How they construct the counterfactual ? Assumptions they make ? Data they require At times: what parameter (ATE, ATT,…) they recover. Naïve comparisons of e.g., participants and nonparticipants or simple before-after differences will not provide the correct counterfactual. 10 Naïve Estimator Validity concepts Internal validity is concerned with the validity of the estimates. • Does the study successfully uncover causal effects for the sample studied? • Are the estimates unbiased? External validity is concerned with the generalisability of the estimates. • Do the study’s findings inform us about different populations? 11 Naïve Estimator Take-away points ? How did the non-treated ‘escape’ treatment? ? People deciding to participate and those decided not to are general fundamentally different. ? Discussion: Econometrics The Counterfactual Model Assess comparability in terms of observables ? Make your life easy by choosing your comparison group wisely: How to choose a comparison group • Randomly deny program access to a sub-group of participants (i.e. randomised experiment) • Arbitrary rules that locally ‘randomise’ people (i.e. regression discontinuity design) • Sources of natural variation in treatment assignment: two very similar groups on average, but one ‘randomly’ has more exposure to treatment (i.e. instrumental variables) • Non-participants who look similar to participants (i.e. regression, matching) • Remove selection bias under pre-program data on the two 12 groups (i.e. difference-in-differences) … Get a 10 % discount on an order above $ 100 Use the following coupon code : NURSING10

RECOMMENDED: Discussion: Interference Statistical Questions