Homework 4: IV in the wild and in Experiments (AK)

Purpose

The objective of this homework is for you to practice concepts learned in class and apply them to a real-case scenario. The concepts we will practice in this homework are related to understanding IV. You are building your skills: reading papers, interpreting tables, interpreting coefficients from complex designs, and using research to think about policy implications. Look at you!

Clean your desk, get a bottle of water, pick your favorite beverage, turn on “do not disturb,” set a timer for 45 min (then take breaks), put on some work tunes, and dive into the fun of learning.

Guidelines

• You can work by yourself or with groups of up to two. [You have to work with new people this time.]
• Submit your group answers to Gradescope. One submission per group, please.
• Submit your do-file to Gradescope.
• We encourage you to use the boxes; PDFs, JPGs, and PNGs are preferable over Word documents or CSV. Remember, you can always save something as a PDF. You can also “Screenshot” anything. In Windows, you can do this by using the snipping tool or Windows+Shift+S. In Mac, you can do this by command+shift+4.
• You will get points for correct answers. You will get points deducted if the answer contains more information that’s not necessary or if the answer contains incorrect statements among accurate statements. In short, we are trying to incentivize students to use the least amount of characters while maximizing the accuracy of responses. This will get stricter over time.
• Your responses should be professionally formatted and written.
• The due date is Friday, March 7th, at 11:59 pm EDT.
• You can answer all of your questions to the nearest 0.0x decimal.

Preamble

In this assignment, you will analyze data from the Minneapolis Domestic Violence Experiment (MDVE), a pioneering investigation of criminal recidivism, and you will also have to read a paper on the effects of female participation on crime.

MDVE

The enclosed Police Foundation report by Sherman and Berk (1984b) provides a summary of the MDVE’s goals and design. You should read the report before you begin this assignment. Sherman and Berk also published their findings in the American Sociological Review. You may find it interesting to compare how the authors present their work to different audiences, so we’ve enclosed the journal article with these instructions.

The enclosed Stata file, mdve.dta, contains a modified extract from the MDVE data. The baseline data (pre-randomization) match the original MDVE records, but the recidivism outcomes have been simulated to protect the anonymity of individual cases.

Assessing Randomization & Compliance

The MDVE randomly assigned domestic abuse suspects to one of three interventions: arrest, separation, and advisement. The variable assigned indicates each suspect’s randomly assigned group. We’ll focus on arrests for this homework, so you can include both separation and advisement in your “control” group. Officers in the MDVE were allowed to exercise their judgment when executing the assigned interventions. If a suspect was randomly selected for arrest, the officer could still choose to separate or advise if a less extreme response seemed appropriate and vice versa. The variable delivered indicates which intervention the officer used for each suspect.

1. How many domestic violence incidents appear in the sample? What fraction of all suspects were randomly selected for arrest?
‣
Answer
2. Among the randomly selected incidents for arrest, what fraction involved an intoxicated suspect? For those not assigned to arrest, what fraction of those suspects were intoxicated? Interpret the difference. Is it large? Is it statistically significant?
‣
Answer
3. What fraction of the suspects randomly selected for arrest were arrested?
‣
Answer
4. What fraction of the incidents not randomly selected for arrest led to arrests anyway?
‣
Answer
5. Why do the answers to questions 4 and 5 differ? Why were officers more likely to “override” the assigned instructions from the experiment in some cases than others? Use the data provided to support your explanation.
‣
Answer

Addressing Non-Compliance with IV

In this section, you will use instrumental variables (IV) to estimate the causal effect of arrests on repeat domestic violence offenses. The variable repeat indicates whether the suspect was involved in another domestic violence report within six months of the original incident.

6. What is the instrument? What is one outcome? What is the treatment?
‣
Answer
7. Report the results from your regression in a table. Recall that you can use the esttab.
1. Column 1: Reports the results of a regression representing the first stage.
‣
Answer
2. Column 2: Reports the results of a regression representing the reduced-form stage.
‣
Answer
3. Interpret your reduced-form estimate in words. Is it large? Is it statistically significant?
‣
Answer
4. Column 4: Create a dummy variable that indicates whether a weapon was involved in the incident. Add this dummy to your reduced-form regression as a control and put the results in column 3. Does weapon use predict recidivism? Does adding this control variable change the coefficient on arrest assignment (the instrument)? Why?
‣
Answer
5. Column 5: Use the ivregress command to estimate the causal effect of actual arrests on repeat offenses.
‣
Answer
6. Interpret your IV estimate in words. Is it large? Is it statistically significant?
‣
Answer
7. Write an equation that shows how your IV estimate is related to your answers for 7a and 7b
‣
Answer
8. Is your IV estimate larger or smaller in magnitude than your reduced-form estimate from question 7b? Why? Be sure to explain in words, not just math.
‣
Answer
9. Column 6: Run a simple OLS regression of repeat offenses on actual arrests.
‣
Answer
10. Is your IV estimate larger or smaller in magnitude than the OLS effect?
‣
Answer
11. What does the difference between the IV and OLS estimates suggest about selection bias in arrest rates? Do officers arrest suspects who are more or less likely to recidivate?
‣
Answer

IV in the wild: Sports and Crime

For this part of the homework, you will base your answers on the study by Sabia and Kumpas (2018), “Does Female Sports Participation Reduce Crime? New Evidence from Title IX”. This is a current working paper by the authors, which means it has not gone through peer review yet. You can find this paper in the homework 4 folder

Signing the Bias

When thinking about the basic relationship between two variables, the first thing is to (1) think about main biases and (2) think about how those biases may be pushing the coefficient.

14. Let’s get some practice: Several studies have empirically examined the relationship between sports participation and crime, but nearly all have treated participation as exogenous to crime (Carrell et al. (2011); Crabbe (2000); Crost and Guerrero (2012); Ekholm (2013); Hartmann and Depro (2006); Jacob et al. (2012); Kirk (2008); McGinn et al. (2008); Smith and Waddington (2004); Spruit et al. (2016)). These papers run regression of the following flavor:

Understanding IV

16. The paper “Does Female Sports Participation Reduce Crime? New Evidence from Title IX” attempts to capture the causal effect of sports on crime using an IV strategy. Answer the following question regarding the paper:
1. What are the main outcome variables?
‣
Answer
2. What is the main explanatory variable?
‣
Answer
3. Explain in a non-technical way: what the instrument is in this setting? Who are the "compliers" in this case? (Recall we are not looking for a definition of compliers)
‣
Answer
4. What is the LATE, in this context? (not looking for the definition of LATE, but the interpretation within this context in a non-technical way)
‣
Answer
5. Write down an equation that represents the “First Stage” in this setting.
‣
Answer
6. Write down an equation that represents the “Reduced Form” in this setting.
‣
Answer
7. Which table provides evidence of the relevance assumption?
‣
Answer
8. Compare both columns of table 5 with the last two columns of table 2: What is the main difference between these two columns in terms of how these were estimated?
‣
Answer
9. Does the instrument satisfy the exclusion restriction? Make a compelling argument against it. (Hint: Using counter-examples is always a great way to tackle this question)
‣
Answer
17. A policy-maker doesn’t really buy the results from this paper because they don’t buy the instrument. Instead, they tell you to conduct an analysis with different instruments. Politely respond and explain to your boss why each of the following instruments they propose is valid or not. [That would be the end of the question without guardrails, so try thinking about how you would answer before reading this hint. The hint is that in order to determine if an instrument is valid or not you need to go through the assumptions, so for any proposed instrument, tell your boss which assumptions would not be satisfy and explain why. For example, If for a given instrument two assumptions are not valid, then write which ones and why not]
1. Changes in high school regulation to be more strict with red and yellow cards: The idea is that some schools started becoming more strict about any contact between players during a soccer game play. Your boss argues that since there will be more strict on this area, more people would want to participate in soccer since they will be less likely to be injured.
‣
Answer [These are not exhaustive, and just sample reasoning]
2. Forever 21 starts selling sports clothing: Across stores nationwide, Forever 21 - a pop-ular store among 12-18 population - started selling sporting attire. Your boss thinks that now that Forever 21 started selling sporting clothes, more people would be likely to participate in sports across high-schools, and hence this would be a good instrument.
‣
Answer [These are not exhaustive, and just sample reasoning]
3. Change from single-gender to coed sports: Across some schools, they’ve been banning single-gender sports and started moving to co-ed sports. Your boss thinks this would be a great instrument, since this would mean more people would want to join sports leagues now that they are co-ed.
‣
Answer [These are not exhaustive, and just sample reasoning]

Extensions

This section is not graded and you don’t have to submit, but will help you push your thinking further. Think of the questions of “extensions” as questions that we could ask in this homework, but we decided not to grade them. Therefore, you should be able to know how to answer these questions or think of them as practice questions.

18. Ex-Batten Professor Jennifer Doleac is a crime economist. She also has a podcast on related topics. She will tell you that good data on criminal behavior are hard to come by. Why might we be especially concerned about using reported domestic violence incidents to measure recidivism in the MDVE?
‣
Answer
19. To assess the external validity of the MDVE results, several cities attempted to replicate the MDVE in their communities. Milwaukee included more than 1,200 domestic violence incidents in its experiment from 1987 to 1989. Investigate the Milwaukee Domestic Violence Experiment independently, and briefly explain (< 150 words) how the Milwaukee results differed from the original Minnesota study. Be sure to cite any sources you consult to form your answer.
‣
Answer

References

Carrell, Scott E, Mark Hoekstra, and James E West, “Does Drinking Impair College Perfor-mance? Evidence from a Regression Discontinuity Approach,” Journal of Public Economics, 2011, 95 (1), 54–62.

Crabbe, Tim, “A sporting chance?: Using sport to tackle drug use and crime,” Drugs: education, prevention and policy, 2000, 7 (4), 381–391.

Crost, Benjamin and Santiago Guerrero, “The Effect of Alcohol Availability on Marijuana Use: Evidence from the Minimum Legal Drinking Age,” Journal of Health Economics, 2012, 31 (1), 112–121.

Ekholm, David, “Sport and crime prevention: Individuality and transferability in research,” Jour- nal of sport for development, 2013, 1 (2), 26–38.

Hartmann, Douglas and Brooks Depro, “Rethinking sports-based community crime prevention: A preliminary analysis of the relationship between midnight basketball and urban crime rates,”Journal of sport and social issues, 2006, 30 (2), 180–196.

Jacob, Robin, Pei Zhu, Marie-Andrée Somers, and Howard Bloom, “A Practical Guide to Regression Discontinuity.,” MDRC, 2012.

Kirk, David, “Sport and Crime Reduction: The Role of Sports in Tackling Youth Crime, by G. Nichols,” 2008.

McGinn, Aileen P, Kelly R Evenson, Amy H Herring, Sara L Huston, and Daniel A Ro- driguez, “The association of perceived and objectively measured crime with physical activity: a cross-sectional analysis,” Journal of physical activity and health, 2008, 5 (1), 117–131. Sabia, Joseph and Gokhan Kumpas, “Does Female Sports Participation Reduce Crime? New Evidence from Title IX,” Working Paper, 2018.

Sherman, Lawrence W and Richard A Berk, “The specific deterrent effects of arrest for domes- tic assault,” American sociological review, 1984, pp. 261–272.

Sherman, W and Richard A Berk, “The Minneapolis domestic violence experiment,” 1984. Smith, Andy and Ivan Waddington, “Using sport in the community schemes to tackle crime and drug use among young people: some policy issues and problems,” European physical education review, 2004, 10 (3), 279–298.

Spruit, Anouk, Eveline Van Vugt, Claudia Van Der Put, Trudy Van Der Stouwe, and Geert-Jan Stams, “Sports participation and juvenile delinquency: A meta-analytic review,” Journal of Youth and Adolescence, 2016, 45 (4), 655–671.