<p>Diff-in-Diff is a parametric model. A rule of thumb for parametric models is that you should have at least 25-30 observations (different authors might disagree but it is around 30) per independent regressor used in your model (see discussion in Verbeek A Guide to Modern Econometrics pp 36). However, note this is a rule of thumb and as Verbeek points out (my comments are in []):</p>
<blockquote>
<p>Unfortunately, there is no unambiguous definition of what is ‘sufficiently large’ [talking about sample sizes to justify asymptotic properties]. In
simple circumstances a sample size of 30 may be sufficient, whereas in more complicated or extreme cases a sample of 1000 may still be insufficient for the asymptotic
approximation to be reasonably accurate.</p>
</blockquote>
<p>This being said getting precise number for every individual case would be too difficult (it would involve Monte Carlo simulations) so often people  follow the rules of thumb unless they have suspicion something is awry.</p>
<p>Consequently, if you would just follow the rule of thumb in a diff-in-diff model with 5 independent regressors you should have at least 150 observations, in model with 7 independent regressors 210, in model with 10 independent regressors 300 and so on.</p>

CC BY-SA 4.0
2021-02-25T12:39:00.520Z
2021-02-25T12:39:00.520Z
1muflon1
<p>I was thinking about running a diff-in-diff with fixed effect in order to deal with a panel data experiment. The problem is that I don't know how many datapoints I need in order to the experiment be reliable.</p>
<p>This is the thing: I want to see how my customers react to a change in price. An AB test doesn't look like a good alternative, due to <a href="https://blog.wiser.com/what-is-product-cannibalization/" rel="nofollow noreferrer">cannibalization</a>. So I would like to change the price of a bundle of products in some cities. Let's say for cities A, B, C and D, cleaning products will be 10% cheaper. Then I would run a diff-in-diff with fixed effects using each city datapoint. The question is: how many cities do I need in order to run this experiment? How many sales each city would have to have in order to the experiment be reliable?</p>

<econometrics><elasticity><applied-econometrics><regression><price>
What's the minimum number of datapoints in order to run a diff-in-diff?
dummyds
# What's the minimum number of datapoints in order to run a diff-in-diff?

(Asked by: username_0 on 2021-02-25)

I was thinking about running a diff-in-diff with fixed effect in order to deal with a panel data experiment. The problem is that I don't know how many datapoints I need in order to the experiment be reliable.

This is the thing: I want to see how my customers react to a change in price. An AB test doesn't look like a good alternative, due to [cannibalization](https://blog.wiser.com/what-is-product-cannibalization/). So I would like to change the price of a bundle of products in some cities. Let's say for cities A, B, C and D, cleaning products will be 10% cheaper. Then I would run a diff-in-diff with fixed effects using each city datapoint. The question is: how many cities do I need in order to run this experiment? How many sales each city would have to have in order to the experiment be reliable?

**Tags:** econometrics, elasticity, applied-econometrics, regression, price

---

## 1 answers

### Answer by username_1 on 2021-02-25. Score: 0 | Accepted answer

Diff-in-Diff is a parametric model. A rule of thumb for parametric models is that you should have at least 25-30 observations (different authors might disagree but it is around 30) per independent regressor used in your model (see discussion in Verbeek A Guide to Modern Econometrics pp 36). However, note this is a rule of thumb and as Verbeek points out (my comments are in []):

> Unfortunately, there is no unambiguous definition of what is ‘sufficiently large’ [talking about sample sizes to justify asymptotic properties]. In
> simple circumstances a sample size of 30 may be sufficient, whereas in more complicated or extreme cases a sample of 1000 may still be insufficient for the asymptotic
> approximation to be reasonably accurate.

This being said getting precise number for every individual case would be too difficult (it would involve Monte Carlo simulations) so often people follow the rules of thumb unless they have suspicion something is awry.

Consequently, if you would just follow the rule of thumb in a diff-in-diff model with 5 independent regressors you should have at least 150 observations, in model with 7 independent regressors 210, in model with 10 independent regressors 300 and so on.


---

