<p>Regression discontinuity design just as difference in differences is a method for exploiting natural experiments. It builds on arbitrary rules that give different "treatments" to otherwise similar units.</p>

<p>An example from Wikipedia:</p>

<blockquote>
  <p>If all students above a given grade—for example 80%—are given the
  scholarship, it is possible to elicit the local treatment effect by
  comparing students around the 80% cut-off: The intuition here is that
  a student scoring 79% is likely to be very similar to a student
  scoring 81%—given the pre-defined threshold of 80%, however, one
  student will receive the scholarship while the other will not.
  Comparing the outcome of the awardee (treatment group) to the
  counterfactual outcome of the non-recipient (control group) will hence
  deliver the local treatment effect.</p>
</blockquote>

CC BY-SA 3.0
2015-06-28T19:06:56.657Z
2015-06-28T19:06:56.657Z
snoram
<p>To follow up the comment by @EnergyNumbers, <em>causality flows from your theory</em>.</p>

<p>A key distinction is this: the math in any of the methods in @BKay's answer is designed to spit out numbers at the end of the procedure.  For example, consider a diff-in-diff where your treatment is something silly like being licked in the face by a dog.  You can always set up a diff-in-diff to see whether being licked in the face by dogs causes people to become astronauts.</p>

<p>Silliness aside, thinking about causality from the ground up can be pretty helpful--to include your choice of procedure.  Seminars in economics frequently revolve around the viability of the theory and the validity of the assumptions.</p>

CC BY-SA 3.0
2015-06-29T19:57:15.027Z
2015-06-29T19:57:15.027Z
Pat W.
<p>Difference in Difference is probably the <a href="http://qje.oxfordjournals.org/content/119/1/249.short" rel="nofollow">favourite method in econometrics</a> (although it requires bootstraping, i.e. correcting the data from self-correlation). It basically compares the evolution of two groups, from a point at which none is subject to the given factor to a point at which one of them is subject to the factor. A famous example is <a href="https://en.wikipedia.org/wiki/Difference_in_differences#Card_.26_Krueger_.281994.29_example" rel="nofollow">Card and Krueger's</a> use of the method to investigate the impact of a minimum wage.</p>

<p><img src="https://upload.wikimedia.org/wikipedia/commons/d/da/Illustration_of_Difference_in_Differences.png" width="400" height="400"></p>

CC BY-SA 3.0
2015-06-28T12:45:15.353Z
If anyone knows how I can scale down that picture, feel free to edit ;)
VicAche
CC BY-SA 3.0
2015-06-28T17:49:55.403Z
Just use standard HTML image code: 
(e.g., <img src="URL" width="100" height="100">)
BKay
CC BY-SA 3.0
2015-06-28T19:25:57.440Z
Difference in Difference does not establish causation. It establishes the possible size of an effect; but not the causality.
410 gone
CC BY-SA 3.0
2015-06-28T21:24:36.157Z
Quantifying effects is estimating causality, at least as OP understands it.
VicAche
CC BY-SA 3.0
2015-06-28T11:24:58.317Z
2015-06-28T17:49:40.657Z
2015-06-28T17:49:40.657Z
VicAche
<p>Natural experiments are usually a setting for causal inference rather than a causal inference tool per se. You often need to employ something like difference-in-difference or instrumental variables anyway even when you have a natural experiment.</p>

<h2> Here a list of statistical causal inference approaches (Approach: Lay description)</h2> 

<ul>
<li><a href="https://en.wikipedia.org/wiki/Instrumental_variable">Instrumental Variables</a>: Randomly assigned variable X influences Z only through Y </li>
<li><a href="https://en.wikipedia.org/wiki/Difference_in_differences">Difference in Differences</a>:  If two groups have a common trend and only one group is treated then the change in the difference between the groups is the treatment effect</li>
<li><a href="https://en.wikipedia.org/wiki/Regression_discontinuity_design">Regression Discontinuity</a>: If a hard threshold determined treatment, look at difference right around that threshold</li>
<li><a href="https://en.wikipedia.org/wiki/Propensity_score_matching">Propensity score matching</a>: Create a control group by matching untreated observations that were likely to be treated (but not in fact treated) with treated observations with a similar probability of treatment. </li>
<li><a href="http://www.statsols.com/mahalanobis-distance-matching-method/">Manhalobis distance matching</a>:   Create a control group by matching untreated observations that look similar to treated. Another notable distance measure is <a href="http://gking.harvard.edu/cem">Coarsened Exact Matching</a>.  </li>
<li><a href="https://web.stanford.edu/~jhain/synthpage.html">Synthetic control</a>: When you have only one treated observation, create a composite of untreated observations that individually are imperfect controls but collectively act as a good control.</li>
<li>Synthetic Cohorts - Treat repeated observations of groups like a panel of individuals and employ panel techniques. </li>
<li>Selection bias modeling like <a href="https://en.wikipedia.org/wiki/Heckman_correction">Heckman correction</a>: Assume a parametric form for selection bias and remove it, so the corrected regression results can be interpreted causally.</li>
<li><a href="http://www.nber.org/papers/w14083.pdf">Sample weighting more broadly</a> - fix bias resulting from endogenous participation and un-modeled heterogeneity by weighting sample units to look more like the true population of interest.</li>
</ul>

CC BY-SA 3.0
2016-02-12T14:59:58.007Z
For a [list of methods](http://replication.uni-goettingen.de/wiki/index.php/Category:Article_methods_and_estimation) you can take a look at our ReplicationWiki. Studies that use them are listed, and you can search which of them have data and code available, [like here for difference in differences](http://tinyurl.com/ReplicationWiki-DID).
Jan Höffler
CC BY-SA 3.0
2015-06-28T19:08:43.277Z
2015-06-29T12:44:37.983Z
2015-06-29T12:44:37.983Z
BKay
<p>A simple linear regression only shows correlation between two variables. To establish causation, two commonly taught methods are IV regression and natural experiments. What are the other methods people use to establish causation?</p>

CC BY-SA 3.0
2015-06-28T19:35:46.960Z
It's disputed whether IV regression says much about causation
410 gone
<econometrics><identification>
What are the empirical techniques to show causation?
Bravo
# What are the empirical techniques to show causation?

(Asked by: username_0 on 2015-06-28)

A simple linear regression only shows correlation between two variables. To establish causation, two commonly taught methods are IV regression and natural experiments. What are the other methods people use to establish causation?

**Tags:** econometrics, identification
> **Comments:**
> **username_1**: It's disputed whether IV regression says much about causation  

---

## 4 answers

### Answer by username_5 on 2015-06-28. Score: 16 | Accepted answer

Natural experiments are usually a setting for causal inference rather than a causal inference tool per se. You often need to employ something like difference-in-difference or instrumental variables anyway even when you have a natural experiment.

## Here a list of statistical causal inference approaches (Approach: Lay description)

* [Instrumental Variables](https://en.wikipedia.org/wiki/Instrumental_variable): Randomly assigned variable X influences Z only through Y
* [Difference in Differences](https://en.wikipedia.org/wiki/Difference_in_differences): If two groups have a common trend and only one group is treated then the change in the difference between the groups is the treatment effect
* [Regression Discontinuity](https://en.wikipedia.org/wiki/Regression_discontinuity_design): If a hard threshold determined treatment, look at difference right around that threshold
* [Propensity score matching](https://en.wikipedia.org/wiki/Propensity_score_matching): Create a control group by matching untreated observations that were likely to be treated (but not in fact treated) with treated observations with a similar probability of treatment.
* [Manhalobis distance matching](http://www.statsols.com/mahalanobis-distance-matching-method/): Create a control group by matching untreated observations that look similar to treated. Another notable distance measure is [Coarsened Exact Matching](http://gking.harvard.edu/cem).
* [Synthetic control](https://web.stanford.edu/~jhain/synthpage.html): When you have only one treated observation, create a composite of untreated observations that individually are imperfect controls but collectively act as a good control.
* Synthetic Cohorts - Treat repeated observations of groups like a panel of individuals and employ panel techniques.
* Selection bias modeling like [Heckman correction](https://en.wikipedia.org/wiki/Heckman_correction): Assume a parametric form for selection bias and remove it, so the corrected regression results can be interpreted causally.
* [Sample weighting more broadly](http://www.nber.org/papers/w14083.pdf) - fix bias resulting from endogenous participation and un-modeled heterogeneity by weighting sample units to look more like the true population of interest.

> **Comments:**
> **username_6**: For a [list of methods](http://replication.uni-goettingen.de/wiki/index.php/Category:Article_methods_and_estimation) you can take a look at our ReplicationWiki. Studies that use them are listed, and you can search which of them have data and code available, [like here for difference in differences](http://tinyurl.com/ReplicationWiki-DID).  

---

### Answer by username_2 on 2015-06-28. Score: 1

Regression discontinuity design just as difference in differences is a method for exploiting natural experiments. It builds on arbitrary rules that give different "treatments" to otherwise similar units.

An example from Wikipedia:

> If all students above a given grade—for example 80%—are given the
> scholarship, it is possible to elicit the local treatment effect by
> comparing students around the 80% cut-off: The intuition here is that
> a student scoring 79% is likely to be very similar to a student
> scoring 81%—given the pre-defined threshold of 80%, however, one
> student will receive the scholarship while the other will not.
> Comparing the outcome of the awardee (treatment group) to the
> counterfactual outcome of the non-recipient (control group) will hence
> deliver the local treatment effect.


---

### Answer by username_3 on 2015-06-29. Score: 1

To follow up the comment by @EnergyNumbers, *causality flows from your theory*.

A key distinction is this: the math in any of the methods in @BKay's answer is designed to spit out numbers at the end of the procedure. For example, consider a diff-in-diff where your treatment is something silly like being licked in the face by a dog. You can always set up a diff-in-diff to see whether being licked in the face by dogs causes people to become astronauts.

Silliness aside, thinking about causality from the ground up can be pretty helpful--to include your choice of procedure. Seminars in economics frequently revolve around the viability of the theory and the validity of the assumptions.


---

### Answer by username_4 on 2015-06-28. Score: 1

Difference in Difference is probably the [favourite method in econometrics](http://qje.oxfordjournals.org/content/119/1/249.short) (although it requires bootstraping, i.e. correcting the data from self-correlation). It basically compares the evolution of two groups, from a point at which none is subject to the given factor to a point at which one of them is subject to the factor. A famous example is [Card and Krueger's](https://en.wikipedia.org/wiki/Difference_in_differences#Card_.26_Krueger_.281994.29_example) use of the method to investigate the impact of a minimum wage.

![](https://upload.wikimedia.org/wikipedia/commons/d/da/Illustration_of_Difference_in_Differences.png)

> **Comments:**
> **username_4**: If anyone knows how I can scale down that picture, feel free to edit ;)  
> **username_5**: Just use standard HTML image code:  (e.g., <img src="URL" width="100" height="100">)  
> **username_1**: Difference in Difference does not establish causation. It establishes the possible size of an effect; but not the causality.  
> **username_4**: Quantifying effects is estimating causality, at least as OP understands it.  

---

