Ateneo - UPO

Causal inference and policy evaluation with micro data: randomised experiments, regression discontinuity, differences-in-differences, instrumental variables. Applications with econometric software (Stata) on real-world data.

- Lecture slides provided by the instructor

- Cunningham, Scott. (2021) Causal Inference: the Mixtape, Yale University Press

- Jousha D. Angrist and Jorn-Steffen Pischke (2015) Mastering Metrics, Princeton University Press

- Jousha D. Angrist and Jorn-Steffen Pischke (2009) Mostly Harmless Econometrics: An Empiricist's Companion. Princeton University Press

- Specific papers indicated in each chapter

The course aims to enable students to answer empirical policy questions using micro-level data. Specifically, the course intends to:

1) enable students to distinguish a causal relationship from a mere correlation and to state explicitly the identification assumptions that make a causal estimate credible;

2) enable students to select and independently apply the appropriate research design (randomised experiment, regression discontinuity, differences-in-differences, instrumental variables) to a specific policy evaluation question;

3) enable students to produce and interpret estimates using econometric software (Stata) and to discuss their limitations and external validity.

Allocation of CFU by type of teaching activity (6 CFU = 150 total hours):

- lecture-based teaching (DE), on campus: 35 hours;
- interactive teaching (DI), guided computer-lab sessions with Stata and in-class discussion of empirical articles: 10 hours;
- self-study (individual study, replication of do-files, reading of the assigned articles): 105 hours.

No distance-learning components (TEL-DE / TEL-DI) are envisaged.

A solid understanding of basic statistics and econometrics.

Lectures, Exercises

Studenti with physical disabilities, Learning Disabilities or Special Education Needs can request specific services and tools via the Staff Sviluppo e Coordinamento Carriere e Servizi alle Studentesse e agli Studenti, consulting the University webpage: https://www.uniupo.it/en/services/services-students-physical-or-learning-disabilities Students with disabilities, learning disabilities or special education needs, once they have contacted the University Staff, can refer to the tutor in charge of the course to define the examination modalities, concerning academic aspects.

Assessment consists of two components: group work carried out during the course and a final written exam. Completing the group work is not a prerequisite for taking the written exam.

Group work (20% of the final grade, equivalent to 6 points out of 30)

In small groups, students apply one of the course's methods to an assigned empirical question, carrying out the analysis in Stata, writing a brief report, and presenting it in class.

Criteria:

appropriateness of the identification strategy to the question (2 points);
correctness of the empirical analysis and robustness checks (2 points);
clarity of the oral presentation and ability to answer questions (2 points).

Descriptors: ability to apply knowledge and understanding; making judgements; communication skills.

Final written exam (90 minutes, 30 points)

The exam is closed-book: no notes, books, or electronic devices are allowed.

Part A – 5 multiple-choice questions, 2 points each (10 points). Each correct answer is worth 2 points; incorrect and unanswered questions are worth 0, with no penalty.
Descriptor: knowledge and understanding.
Part B – three open-ended questions. Each requires setting up the research design, calculating and interpreting the estimates, stating the identification assumption, and discussing threats to validity. Partial credit is given for correct reasoning even where calculation errors are present.
Descriptors: ability to apply knowledge and understanding; making judgements; communication skills; learning skills.

Criteria for determining the overall grade:

Final grade (out of 30) = Part A + Q1 + Q2 + max{Q3; group work grade expressed on a 6-point scale}. Q3 therefore serves as a substitute for the group work: it replaces its score only if doing so improves the final grade, and students who did not complete the group work are graded on Q3. A passing grade requires at least 18/30. Honors (lode) is awarded to students who achieve 30 points while demonstrating, in Part B, the advanced level described in the expected learning outcomes.

Correspondence between level achieved and mark band

- 18–20: the student recognises the core concepts and the identification assumptions of the main methods and correctly interprets a regression table, with gaps in application and imprecise technical language (minimum pass level described in the Intended learning outcomes field).
- 21–23: correct but descriptive knowledge; correct application to cases already discussed in class; threats to validity identified only when prompted.
- 24–26: independent application to new cases; correct computation and quantitative interpretation; adequate discussion of internal validity.
- 27–29: full command of the methods; ability to choose and justify the research design in non-standard cases; critical discussion of internal and external validity; clear and accurate exposition.
- 30 and 30 cum laude: the advanced level described in the Intended learning outcomes field, with an independent proposal of falsification tests and a well-argued link to policy implications.

1. Introduction: The “credibility revolution” in empirical economics
2. Recap of Ordinary Least Squares (OLS) and causality
3. Randomized Experiments
4. Regression Discontinuity Design
5. Differences in Differences
6. Instrumental Variables
7. Recent developments of causal inference and policy evaluation methods

Integration of the gender dimension:
Throughout the course, the different methods presented in the programme are used to analyse the effects of public policies on gender-related issues, such as the gender wage gap, labour supply decisions, or the child gap.

Knowledge and understanding
By the end of the course, students know and understand:

- the potential outcomes framework and the distinction between correlation, conditional association and causal effect;
- the identification assumptions of each of the four research designs covered (randomisation, discontinuity, parallel trends, relevance and exclusion) and the empirical conditions that make
them plausible or implausible;
- the parameters estimated by each design (ATE, ATT, ITT, LATE) and the population they refer to;
- the main recent developments in the policy evaluation literature, with particular reference to staggered-adoption DiD.

Applying knowledge and understanding
By the end of the course, students are able to:

- select, given a policy question and the structure of the available data, the appropriate research design and justify that choice;
- implement the corresponding estimation in Stata on real microdata and produce the standard tables and figures of the method;
- run and interpret the falsification and robustness tests specific to each method;
- interpret the estimated coefficients quantitatively, expressing their magnitude in economically meaningful units.

Making judgements
Students are able to assess the credibility of the identification strategy of an empirical study not already discussed in class, identifying threats to its internal and external validity, and to form a reasoned judgement on the transferability of the results to a policy context different from the one studied.

Communication skills
Students are able to present in writing, in English and in a structured form, an identification strategy and its results, using correct technical terminology and making the conclusions intelligible also to a non-specialist audience.

Learning skills
Students are able to read independently an empirical article published in a scientific journal, to reconstruct its research design from the empirical section alone, and to find and use the technical documentation of the econometric commands required.

Minimum pass level
To pass, students must demonstrate that they can:

- correctly state the identification assumption of each of the four main research designs;
- recognise, in a proposed applied case, which design is appropriate;
- correctly read and interpret the sign, magnitude and significance of the coefficients in a regression table;
- describe at least one threat to the internal validity of the study examined;
- set out their reasoning in intelligible written form, even if technical language is not yet fully mastered.

Advanced level
To achieve an excellent mark, students must in addition demonstrate that they can:

- independently design an identification strategy for a new policy question, justifying the choice against the alternatives discarded;
- propose the appropriate falsification tests and anticipate their expected outcome under the null hypothesis;
- critically discuss the limitations of the methods learned, including the problems of staggered-adoption DiD and the gap between the LATE and the policy parameter of interest;
- connect the empirical results to precise policy implications, stating the conditions under which those implications hold.