Advanced Quantitative Methods I (API-209) Prof.

Dan Levy

Harvard Kennedy School Harvard University

Advanced Quantitative Methods I (API-209)

Faculty: Office: Phone: E-mail:

Dan Levy Littauer 115 (617) 496-4281 [email protected]

Faculty Assistant: Mae Klinger Office: Littauer 107B Phone: (617) 495-4725 E-mail: [email protected]

Key people: Teaching Fellow: Course Assistants: Teddy Svoronos; [email protected] Sandra Naranjo; [email protected] Sarah Oberst; [email protected] Shanthi Philips; [email protected] Course Description: The goal of this course is to prepare you to analyze public policy issues using statistics. Key topics in the course are in the areas of probability theory, sampling, estimation, hypothesis testing, and regression analysis. While many students taking this class will have already taken courses in statistical inference and regression analysis, this course will probably place a much stronger emphasis than typical courses on conceptually understanding the underlying methods. Since the course is targeted to first-year students in the MPA-ID program, we will not shy away from using the mathematical tools needed to develop the conceptual understanding. But the emphasis of the course will be on the conceptual understanding and application of the tools rather than on the math or the mechanics behind the tools. So for example, when studying hypothesis testing, we will place a heavier emphasis on what the test is doing, when to use it and how to interpret its results, than on mathematical proofs underpinning the test or on mechanical repetitions of the calculations involved in conducting the test. The ultimate objective is that by the end of this course you will be able to: (i) Conceptually understand the statistical methods studied in the course and how they can be applied to analyze a variety of public policy issues. (ii) Interpret the results of statistical analyses and think critically about the potential issues that arise when trying to draw conclusions from such results. (iii) Conduct statistical analyses using Excel and a statistical package called Stata. Class Meetings: Classes:

Tuesdays and Thursdays, 11:40-1:00, Starr.

Review sessions: Session #1: Fr 1:10-2:30, L280 Session #2: Fr 2:40-4:00, L280 [You only need to attend one session] Office Hours: Fridays, 2:30 4:30, L115. Office hours for TF and CAs will be posted on the course website.

Advanced Quantitative Methods I (API-209) Prof. Dan Levy

Kennedy School of Government Harvard University

Prerequisites: The main prerequisite for this course is an understanding of calculus at the level of a typical undergraduate course on the subject. However, as far as mathematics goes, the most important prerequisite is a certain level of mathematical sophistication, i.e. comfort in dealing with mathematical constructs and arguments. Experience with computer programming is desirable. Target Students: This course is targeted to first-year students in the MPA/ID program. Students not in the MPA/ID program will be admitted only with permission of the instructor. Familiarity with the concepts indicated in the background section of the schedule (last column of table below) will be assumed. Students looking for a more mathematical course should consider EC2110 and EC2140 (offered by the Economics Department). Students looking for a less mathematical course should consider API-201 (offered by the Kennedy School). Note: Given the number of admitted MPA-ID students this year, it is highly unlikely that there will be space for many non-MPA/IDs. Texts and Other Course Materials: Key Recommended Textbook: Introductory Econometrics: A Modern Approach, by Jeffrey Wooldridge, South-Western, Fourth Edition, 2009. [W]. This textbook will be used mainly for regression analysis, and may be a useful book to have for API-210. Background Textbooks: Mathematical Statistics with Applications, by Wackerly, Mendenhall and Scheaffer, Duxbury, Sixth Edition, 2002. [WMS]. This textbook mainly for the first half of the course. Recommended if you do not already have a good mathematical statistics textbook with you. Probability and Statistics, by DeGroot and Schervish, Addison Wesley, Third Edition, 2002: This textbook is similar to WMS both in style and mathematical sophistication, but its organized slightly differently. Mind on Statistics, by Utts and Heckard, Thomson, Second Edition, 2004. This book is at a much lower mathematical level than WMS but provides more examples of the use of statistics in everyday life. Note: All the textbooks listed above have been put on reserve at the library. Stata: Completing some problem sets will require the use of a statistical package called Stata. The version you need is Intercooled Stata or better; Small Stata will not be adequate for this course. If you dont have easy access to a computer with Stata or dont want to do all your Stata work in the computer labs, you may want to consider purchasing this software package from Stata directly (details here: http://www.stata.com/order/new/edu/gradplans/student-pricing/). Handouts: Handouts will be distributed throughout the course. The main objective of the handouts is to facilitate the process of taking notes so that you can fully engage in class. They are not meant to substitute for class attendance or for studying the assigned reading material. Handouts will contain blank spaces for you to fill in during class, usually in response to questions. Course website: Course materials will be posted on the course website.

Advanced Quantitative Methods I (API-209) Prof. Dan Levy

Kennedy School of Government Harvard University

Grading: The class grade will be based on the following criteria: 10% - Problem sets 15% - Class participation and engagement 15% - Final Exercise 25% - Midterm exam 35% - Final exam Problem Sets (10%) Problem sets will be assigned almost every week. They will give you hands-on experience with the analytic techniques introduced in class. You should plan to spend approximately 8-10 hours on each problem set. Problem sets will be posted on the course website, as will suggested answers. They will be graded on check-plus/check/check-minus basis. Problem sets not received by the deadline will be considered late. There will be no credit for late assignments. The lowest problem set grade will be dropped when calculating the average grade for the problem sets. Under the Kennedy School Academic Code, the problem sets for this course are Type II assignments. You are encouraged to work in a study group, but must submit your own hand- or type-written solutions. Examples of assignments that are not in accordance with the HKS academic code include photocopies or reprints of substantially identical assignments, printouts of substantially identical Excel tables or graphs, and copies of solutions from previous years. The Kennedy School Academic Code is available at:
http://www.hks.harvard.edu/var/ezp_site/storage/fckeditor/file/pdfs/degree-programs/registrar/academic_code.pdf

Instructions for submitting problem sets: Turn them in via the drop boxes, located on the second floor of Littauer behind the Forum Screen. Submit them by 10:00 am on the day they are due. Assignments handed in after class begins will be considered late. Indicate on cover page the names of the students you worked with. Class participation and engagement (15%) I strongly believe that student participation can substantially enrich the learning experience for both the students and the instructor. In this spirit, you are encouraged to ask questions and to share with the class any relevant insights you may have from your work experience or from previous exposure to the topic at hand. Effective class participation requires that you come prepared for class, and that you think about how your comment or question may enrich the learning experience of your classmates. I also ask that the questions and comments be brief and related to the topic being discussed. Given that this is a large class, I will sometimes need to defer questions for a future class or office hours. For a select number of classes, I will ask that you post on the class website a very brief comment (no longer than 150 words). There are two kinds of postings: pre-class and post-class. The preclass postings are about the assigned reading(s) and are due by 4:00 AM the morning of class. The post-class postings are about the class and will be due 24-48 hours after the class. Comments should be posted on the class website and I encourage you to build on one anothers comments . The class participation and engagement grade will depend on three things: (1) your participation in class, (2) your engagement with the course outside class, and (3) your postings on the course website. For all of these, both quantity and quality will count. Midterm Exam (25%): Please note that midterm exam will be held on Friday, Oct 11. Final Exercise (15%) The final exercise will require applying some of the statistical tools learned in class using a real data set. More details will be provided later in the course.
3

Advanced Quantitative Methods I (API-209) Prof. Dan Levy

Kennedy School of Government Harvard University

Regrade Policy Requests for reconsideration of grades on exams are not encouraged, and will be accepted only in writing, with a clear statement of what has been mis-graded, and within one week of receiving your graded exam. Please submit your full exam so grading on all questions can be reconsidered. All course activities, including class meetings, homework assignments, and exams are subject to the HKS Academic Code and Code of Conduct. Letter Grades Grades for each exam and for each component of the course (problem sets, final exercise, and class participation and engagement) will be standardized (i.e. curved) and then an overall score for the course will be calculated for each student. This overall score will be translated into a final course letter grade using the Deans Recommended Grade Distribution (available at http://www.hks.harvard.edu/degrees/registrar/faculty/exams-and-grading/grades)

Tentative Schedule: Tentative schedule is attached. Adjustments may need to be made and will be distributed as soon as they are available. W refers to readings from Wooldridge. For a refresher on the concepts indicated in the background section of the schedule (last column of table below), see Deb Hughes Hallett's excellent handouts (click on "Math Camp" link here: https://knet.hks.harvard.edu/dp_student_affairs/Program_Pages/mathcamp/Pages/default.aspx)

READINGS Orr, Larry L. 1999. Social Experiments: Evaluating Public Programs with Experimental Methods (Thousand Oaks, CA: Sage Publications), pp. 103-115. Shadish, William R., Thomas D. Cook, and Donald T. Campbell. 2002. Statistical Conclusion Validity and Internal Validity, and Construct Validity and External Validity. Chapters 1, 2, and 3 in Experimental and Quasi-Experimental Designs for Generalized Causal Inference (Boston: Houghton Mifflin). pp. 33-47, 53-55, 83-96. Stokey and Zeckhauser, A Primer for Policy Analysis, Chapter 12 The Economist, Signifying Nothing? January 31, 2004 The Economist, Control Freaks: Are randomized evaluations a better way of doing aid and development policy? June 12, 2008. The New York Times "Panel urges mammograms at 50, not 40". November 17, 2009

Date 3-Sep

Class 1

General Topic COURSE OVERVIEW

Specific Topics (Tentative List)

Problem Set Due

Readings
W [Appendix C.1]

Background Material
Basic concepts in descriptive statistics: Mean, Variance, Std Deviation, Mode, Covariance, Correlation, Conditional Mean [W: Appendix B.3] Key concepts in probability: Experiment, Event, Set, Axioms of Probability [W: Appendix B.1]; Random Variables and Probability Distributions (pdf, cdf) Expected Value and Variance of Random Variables [ALZ: 3.1-3.5; WMS: 3.1-3.3, 4.1-4.3; W: B1, B3], Conditional Expectation

5-Sep

PROBABILITY: Bayes Rule

Introduction Key Probability Concepts Bayes Rule Mammogram Application Decision Analysis and Decision Trees The Value of Perfect Information The Value of Imperfect Information Allowing for Risk Aversion General Framework Conditional Probability Bayes Rule Targeting Application Estimators and Sampling Fluctuations Sampling Distribution The Central Limit Theorem Normal Approximation to the Binomial Distribution Confidence Intervals Point Estimators Desirable Properties of Estimators Intuition Behind Hypothesis Testing Steps Involved in Hypothesis Testing Hypothesis Tests Involving the Normal distribution Hypothesis tests involving the t distribution Hypothesis tests in the context of regression analysis Using statistics to detect fraud Chi square tests How not to take statistical tests out of context ANOVA: Conceptual Framework Example Multiple tests Key things about F distribution Two other uses of the F test Relationship between t test and F test Practical Significance Type I and Type II Errors Practice Questions on Hypothesis Testing Types of Sampling Problems with Sampling Appendix: Nonprobability Sampling

Mammograms (NYT 2009)

10-Sep

PROBABILITY: Decision Analysis

PS 1

Stokey and Zeckhouser (Ch 12)

12-Sep

PROBABILITY: Mexico's Pension System

Case Study about Mexico's Pension System

17-Sep 19-Sep 24-Sep

5 6 7

ESTIMATION: Sampling Distribution ESTIMATION: Estimators and Key Properties HYPOTHESIS TESTING: Introduction

PS2

W [C.2, C.3]

Normal Distribution [ALZ: 3.7.1]; Binomial Distribution [ALZ: 3.6.2]

W [C.2, C.3, C.5]

PS3

W [C.6]

26-Sep 1-Oct

8 9

HYPOTHESIS TESTING: t test HYPOTHESIS TESTING: Chi-Square

W [C.6]

Main distributions used for hypothesis testing: Normal, t, Chi-Square, and F [W: B.5]

PS4

W[B.5]

Main distributions used for hypothesis testing: Normal, t, Chi-Square, and F [W: B.5]

3-Oct

10

HYPOTHESIS TESTING: ANOVA

W [B.5]

Main distributions used for hypothesis testing: Normal, t, Chi-Square, and F [W: B.5]

8-Oct

11

HYPOTHESIS TESTING: Final Overview SAMPLING: Sampling Design and Survey Design MIDTERM EXAM (8:00 am; L140)

PS5

The Economist (2004); W [C.6]

10-Oct 11-Oct 15-Oct 17-Oct

12

13 14

REGRESSION: Introduction REGRESSION: Bivariate Regression I

The Counterfactual Average Treatment Effects Regression Population Regression Function Sample Regression Function Ordinary Least Squares

W [1.3, 2.1, 2.2, 2.5, 2.6]

22-Oct 24-Oct

15 16

REGRESSION: Bivariate Regression II REGRESSION: Multiple Regression Estimation I REGRESSION: Multiple Regression Estimation II REGRESSION: Multiple Regression Estimation III REGRESSION: Hypothesis Testing REGRESSION: Functional Form - Dummy Variables REGRESSION: Prediction No Class REGRESSION: Functional Form - Other Non-Linearities REGRESSION: Designing an Empirical Study SAMPLING: Statistical Power NO CLASS: Thanksgiving

Randomized Experiments Bivariate Regression with a Dummy Variable The Model with Two Explanatory Variables The Model with k Explanatory Variables The Expected Value of the OLS Estimators Goodness of fit Variance of OLS Estimators Efficiency of OLS: The Gauss-Markov Theorem Heteroskedasticity Appendix: OLS in matrix algebra notation Appendix: Class Size and Test Scores Omitted Variable Bias - Theory Omitted Variable Bias - Examples Introduction Sampling Distribution of OLS Estimators The t test The F test A Closer Look at Dummy Variables Using Dummy Variables for Multiple Categories Interactive Dummy Variables Dummy Variables as Dependent Variables Appendix - Probit and Logit Prediction

PS6

The Economist (2008)

W [3.1-3.3]

29-Oct

17

PS7

W [2.3, 3.4, 3.5, 6.3]

31-Oct 5-Nov

18 19

W [3.3]

PS8

W [4.1-4.5]

7-Nov

20

W [7.1-7.4]

12-Nov 14-Nov 19-Nov

21 22 23

PS9

Logs Quadratics Appendix: Interpretation of regression coefficients under different functional forms Interpretation of coefficients under Quadratics Assessing Hurricane Mitch - Selecting a design to evaluate the impact of PATH Framework of Statistical Power Factors that Affect Statistical Power Minimum Detectable Differences

PS10

W [6.2, A.4]

21-Nov 26-Nov 28-Nov 3-Dec 5-Dec TBA

24 25

Case Study Evaluation of Jamaica's PATH Program

PS11

Orr (1999)

26 27

OVERALL: Critically Assessing Studies FINAL CLASS: Final Exercise Presentations FINAL EXAM (time and location TBA)

Validity Assessing paper "Do You Have a Voting Plan?"

Final Exercise

Shadish et al. (2004)

W refers to readings from Wooldridge.