<p>I think this paper might be useful to you. It's a job market paper by one of Heckman's students at UChicago, named <a href="http://home.uchicago.edu/~rodrig/" rel="nofollow">Rodrigo Pinto</a>. The paper is titled <a href="http://home.uchicago.edu/~rodrig/JMP.pdf" rel="nofollow">"Selection Bias in a Controlled Experiment: The Case of Moving to Opportunity."</a>
In the MTO experiment, the voucher assignment mechanism was random but only approximately half that received the voucher ended up actually moving. This creates a problem because the usual analysis (treatment-effect-on-the-treated or intent-to-treat) will only tell us the causal effect of receiving a voucher. However, we are interested in the causal effect of the new neighborhood, not of receiving the voucher. He shows how to decompose the typical treatment-on-the-treated parameter into components that have unambiguous interpretations. Namely, he isolates the causal effect of the new neighborhood.</p>

CC BY-SA 3.0
2015-01-19T22:50:26.040Z
2015-04-30T05:31:27.660Z
2015-04-30T05:31:27.660Z
jmbejara
<p>Another thing you can look at is <a href="http://en.wikipedia.org/wiki/Intention-to-treat_analysis" rel="nofollow">"Intention-to-treat analysis"</a>. From Wikipedia,</p>

<blockquote>
  <p>An intention-to-treat (ITT) analysis of the results of an experiment is based on the initial treatment assignment and not on the treatment eventually received. ITT analysis is intended to avoid various misleading artifacts that can arise in intervention research such as non-random attrition of participants from the study or crossover.</p>
</blockquote>

<p>This appears to match what you were looking for: your treatment is randomized initially and people drop out non-randomly.</p>

CC BY-SA 3.0
2015-04-14T22:11:48.440Z
2015-04-14T22:11:48.440Z
jmbejara
<p>Apparently this is called <a href="http://ije.oxfordjournals.org/content/34/1/87">attrition bias</a>. It's very similar to <a href="http://en.wikipedia.org/wiki/Survivorship_bias">survivorship bias</a>. <a href="http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1044&amp;context=famconfacpub">This paper</a> suggest correcting for it using <a href="http://en.wikipedia.org/wiki/Heckman_correction">Heckman correction</a>. <a href="http://en.wikipedia.org/wiki/Propensity_score_matching">Propensity score matching</a> may also <a href="http://econweb.umd.edu/~ham/Rand_Ham_Li%20submit%20july%202011.pdf">help somewhat</a>. My experience with both has been mixed, but they are commonly used. You should figure out what exact approach is most appropriate for your setting.</p>

<p><strong>One last edit:</strong> <a href="http://ftp.iza.org/dp4162.pdf">These</a> <a href="https://www.princeton.edu/~davidlee/wp/resrevision8.pdf">two</a> papers, which talk about bounding the average treatment effect, may also be of use to you.</p>

CC BY-SA 3.0
2014-11-20T01:39:32.127Z
2014-11-20T01:57:05.247Z
2014-11-20T01:57:05.247Z
jayk
<p>Suppose you are running a randomized experiment to assess the effect of $X$, say some training program for unemployed people, on $Y$, say the chance of finding a job in the coming year. Suppose also that $X$ takes time : maybe it lasts for several month.</p>

<p>Because you randomize, you do not need to worry about self-selection bias initially. But during the course of $X$, some people will likely realize that $X$ is beneficial to them, and others may realize that they are wasting their time. </p>

<p>As a result, one might expect that among people who drop from the program, there is a higher proportion of agents for which the treatment effect would have been smaller. This might induce an over-estimation of the treatment effect.</p>

<p><strong>My questions are</strong> :</p>

<ul>
<li>Is this kind of bias discussed in the literature on randomized experiments?</li>
<li>Does it have a canonical name ?</li>
<li>Do researcher try to control for this, and if yes, how?</li>
</ul>

<randomized-experiment><treatment-effect><selection-bias>
Self-selection bias during the course of experiments
Martin Van der Linden
# Self-selection bias during the course of experiments

(Asked by: username_0 on 2014-11-19)

Suppose you are running a randomized experiment to assess the effect of $X$, say some training program for unemployed people, on $Y$, say the chance of finding a job in the coming year. Suppose also that $X$ takes time : maybe it lasts for several month.

Because you randomize, you do not need to worry about self-selection bias initially. But during the course of $X$, some people will likely realize that $X$ is beneficial to them, and others may realize that they are wasting their time.

As a result, one might expect that among people who drop from the program, there is a higher proportion of agents for which the treatment effect would have been smaller. This might induce an over-estimation of the treatment effect.

**My questions are** :

* Is this kind of bias discussed in the literature on randomized experiments?
* Does it have a canonical name ?
* Do researcher try to control for this, and if yes, how?

**Tags:** randomized-experiment, treatment-effect, selection-bias

---

## 3 answers

### Answer by username_2 on 2014-11-20. Score: 12 | Accepted answer

Apparently this is called [attrition bias](http://ije.oxfordjournals.org/content/34/1/87). It's very similar to [survivorship bias](http://en.wikipedia.org/wiki/Survivorship_bias). [This paper](http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1044&context=famconfacpub) suggest correcting for it using [Heckman correction](http://en.wikipedia.org/wiki/Heckman_correction). [Propensity score matching](http://en.wikipedia.org/wiki/Propensity_score_matching) may also [help somewhat](http://econweb.umd.edu/~ham/Rand_Ham_Li%20submit%20july%202011.pdf). My experience with both has been mixed, but they are commonly used. You should figure out what exact approach is most appropriate for your setting.

**One last edit:** [These](http://ftp.iza.org/dp4162.pdf) [two](https://www.princeton.edu/~davidlee/wp/resrevision8.pdf) papers, which talk about bounding the average treatment effect, may also be of use to you.


---

### Answer by username_1 on 2015-01-19. Score: 6

I think this paper might be useful to you. It's a job market paper by one of Heckman's students at UChicago, named [Rodrigo Pinto](http://home.uchicago.edu/~rodrig/). The paper is titled ["Selection Bias in a Controlled Experiment: The Case of Moving to Opportunity."](http://home.uchicago.edu/~rodrig/JMP.pdf)
In the MTO experiment, the voucher assignment mechanism was random but only approximately half that received the voucher ended up actually moving. This creates a problem because the usual analysis (treatment-effect-on-the-treated or intent-to-treat) will only tell us the causal effect of receiving a voucher. However, we are interested in the causal effect of the new neighborhood, not of receiving the voucher. He shows how to decompose the typical treatment-on-the-treated parameter into components that have unambiguous interpretations. Namely, he isolates the causal effect of the new neighborhood.


---

### Answer by username_1 on 2015-04-14. Score: 4

Another thing you can look at is ["Intention-to-treat analysis"](http://en.wikipedia.org/wiki/Intention-to-treat_analysis). From Wikipedia,

> An intention-to-treat (ITT) analysis of the results of an experiment is based on the initial treatment assignment and not on the treatment eventually received. ITT analysis is intended to avoid various misleading artifacts that can arise in intervention research such as non-random attrition of participants from the study or crossover.

This appears to match what you were looking for: your treatment is randomized initially and people drop out non-randomly.


---

