<p>Based on OP's explanation in the question and comments, suppose that the movies treated and untreated are like</p>
<pre><code>-----------------------------------------
month    treated movie   untreated movies
-------- --------------  ----------------
  1            1          2,  3, ..., 10
  2           11         12, 13, ..., 20
  3           21         22, 23, ..., 30
...
-----------------------------------------
</code></pre>
<p>and there are multiple reviews in each month. I guess OP wants a convenient way of getting the average treatment effect (averaged over multiple months). Suppose that data are pooled.</p>
<p>The method OP considers in his second comment to the question is (i) <code>reg y i.tgroup##i.post x1 x2 if month==1, vce(r)</code>, etc. and then averaging the DID estimates. Let us see if we can do something similar using a single regression. Let's move step by step.</p>
<p>Consider the regression (ii) <code>reg y i.month##(i.tgroup##i.post x1 x2), vce(r)</code>. Then (ii) should be the same as (i). Standard errors should be OK because heteroskedasticity-robust standard errors are used. The problem with (ii) is that there are <span class="math-container">$m$</span> (# of months) treatment effects, which OP wants to average.</p>
<p>So let us impose the restriction that the coefficient on the interaction term is common (the same over all months), and our regression is (iii) <code>reg y i.month##(tgroup post x1 x2) i.tgroup#i.post, vce(r)</code>. Then a sort of &quot;average treatment effect&quot; is estimated, and all the rest coefficients are month-specific. So far so good.</p>
<p>Remarkably, (i) <span class="math-container">$\neq$</span> (iii) in general; (i) is an unweighted average of <span class="math-container">$\hat\beta_k$</span>'s and (iii) is a particular weighted average of them. The reason follows. Due to the Frisch-Waugh decomposition, (i) equals <span class="math-container">$m^{-1} \sum_{k=1}^m \hat\beta_k$</span>, where <span class="math-container">$\hat\beta_k = (X_k'M_{Z_k} X_k)^{-1} X_k' M_{Z_k} y_k$</span> (DID coefficient for month <span class="math-container">$k$</span>), <span class="math-container">$X_k$</span> is the vector of the interaction terms for month <span class="math-container">$k$</span>, <span class="math-container">$Z_k$</span> the matrix of all other regressors, and <span class="math-container">$y_k$</span> the vector of review rates in month <span class="math-container">$k$</span>, while (iii) gives (according to algebra) <span class="math-container">$(\sum_{k=1}^m X_k'M_{Z_k} X_k)^{-1} \sum_{k=1}^m X_k'M_{Z_k} y_k = \sum_{k=1}^m C_k \hat\beta_k$</span>, where
<span class="math-container">$$C_k = \left( \sum_{j=1}^m X_j'M_{Z_j} X_j \right)^{-1} X_k'M_{Z_k} X_k.$$</span>
Note that the mean-group estimator of (i) uses <span class="math-container">$1/m$</span> for <span class="math-container">$C_k$</span>. (Note that <span class="math-container">$X_k$</span> is a column vector so <span class="math-container">$C_k$</span> is a scalar.) If <span class="math-container">$X_k'M_{Z_k} X_k$</span> is the same for all <span class="math-container">$k$</span> (months), then <span class="math-container">$C_k = 1/m$</span> and hence (iii) = (i).</p>
<p><span class="math-container">$C_k\ne 1/m$</span> due to two reasons. First, the numbers of observations (reviews) are different across months. Second, the covariates (<code>x1</code> and <code>x2</code> in the regressions). (i) and (iii) are treatments averaged differently. I think it's a matter of choice. You could also consider other more restricted models such as <code>reg y i.tgroup##t.post x1 x2, vce(r)</code> as long as you can tell the difference. Why not?</p>
<p><em>I haven't rigorously checked the following.</em></p>
<p>If there are no covariates, we can change (iii) to WLS to construct the results from (i), where the weights are <span class="math-container">$1/\sqrt{n_k}$</span> with <span class="math-container">$n_k$</span> denoting the number of observations in month <span class="math-container">$k$</span>. (Note: It's WLS to construct the mean-group regression. It has nothing to do with GLS.) That is,</p>
<pre><code>by month, sort: gen nobs = 1/sqrt(_N)
reg y i.tgroup##t.post [aw = wgt], vce(r)
</code></pre>
<p>If there are <code>x</code> variables, then we can compute <span class="math-container">$C_k$</span> manually and construct the weights for WLS. Because <span class="math-container">$1/\sqrt{X_k'M_{Z_k} X_k}$</span> can be used as weights (for WLS) and <span class="math-container">$X_k'M_{Z_k} X_k$</span> is nothing but the SSR from the regression of <span class="math-container">$X_k$</span> on <span class="math-container">$Z_k$</span>, the computation is conceptually straightforward, though Stata coding could be somewhat messy.</p>
<pre><code>gen tr = tgroup*post
gen xmx = .
forv k=1/m {
  reg tr tgroup post x1 x2 if month==`k'
  replace xmx = e(rss) if month==`k' /* does it work? */
}
gen wgt = 1/sqrt(xmx)
reg y i.tgroup##i.post [aw = wgt], vce(r)
</code></pre>

CC BY-SA 4.0
2022-06-12T08:48:06.120Z
Thank you a lot, you answer was really helpful!
Nicolas L
CC BY-SA 4.0
2022-06-11T08:53:55.533Z
2022-06-11T09:48:55.203Z
2022-06-11T09:48:55.203Z
chan1142
<p>Say each year I observe the effect of a movie-level treatment on the reviews of 10 movies (1 treated movie and 9 non-treated movies) over two periods (pre and post treatment). Each year, the treatment concerns 10 different movies.</p>
<p>How can I analyze those data using a diff-in-diff approach?</p>
<p>I suspect the conventional diff-in-diff approach from below to be wrong as it will pool together all the movies, without taking into account that they belong to different years/treatments:</p>
<p><span class="math-container">$$
(1) y_{ijt} = \alpha_j + \phi_t + \beta I_{jt} + \epsilon_{ijt}  
$$</span>
where <span class="math-container">$y_{ijt}$</span> is the outcome for review <span class="math-container">$i$</span> of movie <span class="math-container">$j$</span> on period <span class="math-container">$t$</span>, <span class="math-container">$\alpha_j$</span> are movie fixed effects, <span class="math-container">$\phi_t$</span> is a dummy for the post-treatment period, and <span class="math-container">$I_{jt}$</span> is a dummy equal to 1 if movie <span class="math-container">$j$</span> is treated in the post-treatment period.</p>
<p>I also think the staggered diff-in-diff approach to not be suitable as (i) I have a repeated cross-sectional dataset and not a panel, and (ii) the treatment each time concerns different movies, meaning no movie is treated several times.</p>
<p>Thank you in advance for your kind help.</p>

CC BY-SA 4.0
2022-06-11T06:25:24.903Z
Please check if my understanding is correct. There are 10 movies. In "time" 1, movie 1 is treated and the others are untreated; in "time" 2, movie 2 is treated and the others are untreated, .... For every "time", data are collected over two periods (pre and post teatment) for the 10 movies. For example, if "time" means month, you have a data set collected over 10 months, and in month $k$, the $k$th movie is treated and the others are untreated. Every month, you have $y$ before and after the treatment for all 10 movies. Is that right?
chan1142
CC BY-SA 4.0
2022-06-11T07:06:39.993Z
@chan1142 thank you for your answer. A small correction. In month 1, movie 1 is treated and movie 2 to 10 are untreated. In month 2, movie 11 is treated and movie 12-20 are untreated, and so on. So each month, there are different movies which might get treated or not. For each movie, I indeed observe $y$ before and after the treatment. So Intuitively speaking, I want to restrict the analysis to variations within each treatment (=month) so the outcome of movie 1 is only compared to movies 2 to 10, and not the outcome of the other movies.
Nicolas L
CC BY-SA 4.0
2022-06-11T07:17:42.587Z
The idea would be for example to estimate Equation (1) for month == 1, month == 2,, monht == 3, and so on, and then take the average across those different regressions. I am looking for a way to achieve something similar with (1), as averaging accross regressions is not the best thing one can do.
Nicolas L
<econometrics><difference-in-difference>
Handling diff in diff with multiple treatments, but each treatment is applied once to differents groups
Nicolas L
# Handling diff in diff with multiple treatments, but each treatment is applied once to differents groups

(Asked by: username_0 on 2022-06-10)

Say each year I observe the effect of a movie-level treatment on the reviews of 10 movies (1 treated movie and 9 non-treated movies) over two periods (pre and post treatment). Each year, the treatment concerns 10 different movies.

How can I analyze those data using a diff-in-diff approach?

I suspect the conventional diff-in-diff approach from below to be wrong as it will pool together all the movies, without taking into account that they belong to different years/treatments:

$$
(1) y\_{ijt} = \alpha\_j + \phi\_t + \beta I\_{jt} + \epsilon\_{ijt}
$$
where $y\_{ijt}$ is the outcome for review $i$ of movie $j$ on period $t$, $\alpha\_j$ are movie fixed effects, $\phi\_t$ is a dummy for the post-treatment period, and $I\_{jt}$ is a dummy equal to 1 if movie $j$ is treated in the post-treatment period.

I also think the staggered diff-in-diff approach to not be suitable as (i) I have a repeated cross-sectional dataset and not a panel, and (ii) the treatment each time concerns different movies, meaning no movie is treated several times.

Thank you in advance for your kind help.

**Tags:** econometrics, difference-in-difference
> **Comments:**
> **username_1**: Please check if my understanding is correct. There are 10 movies. In "time" 1, movie 1 is treated and the others are untreated; in "time" 2, movie 2 is treated and the others are untreated, .... For every "time", data are collected over two periods (pre and post teatment) for the 10 movies. For example, if "time" means month, you have a data set collected over 10 months, and in month $k$, the $k$th movie is treated and the others are untreated. Every month, you have $y$ before and after the treatment for all 10 movies. Is that right?  
> **username_0**: @chan1142 thank you for your answer. A small correction. In month 1, movie 1 is treated and movie 2 to 10 are untreated. In month 2, movie 11 is treated and movie 12-20 are untreated, and so on. So each month, there are different movies which might get treated or not. For each movie, I indeed observe $y$ before and after the treatment. So Intuitively speaking, I want to restrict the analysis to variations within each treatment (=month) so the outcome of movie 1 is only compared to movies 2 to 10, and not the outcome of the other movies.  
> **username_0**: The idea would be for example to estimate Equation (1) for month == 1, month == 2,, monht == 3, and so on, and then take the average across those different regressions. I am looking for a way to achieve something similar with (1), as averaging accross regressions is not the best thing one can do.  

---

## 1 answers

### Answer by username_1 on 2022-06-11. Score: 0 | Accepted answer

Based on OP's explanation in the question and comments, suppose that the movies treated and untreated are like
-----------------------------------------
month    treated movie   untreated movies
-------- --------------  ----------------
  1            1          2,  3, ..., 10
  2           11         12, 13, ..., 20
  3           21         22, 23, ..., 30
...
-----------------------------------------

and there are multiple reviews in each month. I guess OP wants a convenient way of getting the average treatment effect (averaged over multiple months). Suppose that data are pooled.
The method OP considers in his second comment to the question is (i) reg y i.tgroup##i.post x1 x2 if month==1, vce(r), etc. and then averaging the DID estimates. Let us see if we can do something similar using a single regression. Let's move step by step.
Consider the regression (ii) reg y i.month##(i.tgroup##i.post x1 x2), vce(r). Then (ii) should be the same as (i). Standard errors should be OK because heteroskedasticity-robust standard errors are used. The problem with (ii) is that there are $m$ (# of months) treatment effects, which OP wants to average.
So let us impose the restriction that the coefficient on the interaction term is common (the same over all months), and our regression is (iii) reg y i.month##(tgroup post x1 x2) i.tgroup#i.post, vce(r). Then a sort of "average treatment effect" is estimated, and all the rest coefficients are month-specific. So far so good.
Remarkably, (i) $\neq$ (iii) in general; (i) is an unweighted average of $\hat\beta_k$'s and (iii) is a particular weighted average of them. The reason follows. Due to the Frisch-Waugh decomposition, (i) equals $m^{-1} \sum_{k=1}^m \hat\beta_k$, where $\hat\beta_k = (X_k'M_{Z_k} X_k)^{-1} X_k' M_{Z_k} y_k$ (DID coefficient for month $k$), $X_k$ is the vector of the interaction terms for month $k$, $Z_k$ the matrix of all other regressors, and $y_k$ the vector of review rates in month $k$, while (iii) gives (according to algebra) $(\sum_{k=1}^m X_k'M_{Z_k} X_k)^{-1} \sum_{k=1}^m X_k'M_{Z_k} y_k = \sum_{k=1}^m C_k \hat\beta_k$, where
$$C_k = \left( \sum_{j=1}^m X_j'M_{Z_j} X_j \right)^{-1} X_k'M_{Z_k} X_k.$$
Note that the mean-group estimator of (i) uses $1/m$ for $C_k$. (Note that $X_k$ is a column vector so $C_k$ is a scalar.) If $X_k'M_{Z_k} X_k$ is the same for all $k$ (months), then $C_k = 1/m$ and hence (iii) = (i).
$C_k\ne 1/m$ due to two reasons. First, the numbers of observations (reviews) are different across months. Second, the covariates (x1 and x2 in the regressions). (i) and (iii) are treatments averaged differently. I think it's a matter of choice. You could also consider other more restricted models such as reg y i.tgroup##t.post x1 x2, vce(r) as long as you can tell the difference. Why not?
I haven't rigorously checked the following.
If there are no covariates, we can change (iii) to WLS to construct the results from (i), where the weights are $1/\sqrt{n_k}$ with $n_k$ denoting the number of observations in month $k$. (Note: It's WLS to construct the mean-group regression. It has nothing to do with GLS.) That is,
by month, sort: gen nobs = 1/sqrt(_N)
reg y i.tgroup##t.post [aw = wgt], vce(r)

If there are x variables, then we can compute $C_k$ manually and construct the weights for WLS. Because $1/\sqrt{X_k'M_{Z_k} X_k}$ can be used as weights (for WLS) and $X_k'M_{Z_k} X_k$ is nothing but the SSR from the regression of $X_k$ on $Z_k$, the computation is conceptually straightforward, though Stata coding could be somewhat messy.
gen tr = tgroup*post
gen xmx = .
forv k=1/m {
  reg tr tgroup post x1 x2 if month==`k'
  replace xmx = e(rss) if month==`k' /* does it work? */
}
gen wgt = 1/sqrt(xmx)
reg y i.tgroup##i.post [aw = wgt], vce(r)

> **Comments:**
> **username_0**: Thank you a lot, you answer was really helpful!  

---

