<p>I haven't read the paper you reference, but I think I can give the general idea of how the methodology works.</p>
<h2>Derivation of <span class="math-container">$\boldsymbol{\beta}^*$</span></h2>
<p>To derive the <span class="math-container">$\beta^*$</span> expression, we take the common FWL approach of &quot;washing&quot; out one dimension of the noise in the dependent variable. In this case, by washing out the noise in <span class="math-container">$Z_i$</span> explainable by <span class="math-container">$(X_i,E_i)$</span>, we get the component of <span class="math-container">$Z_i$</span> which is orthogonal to <span class="math-container">$f(X_i,E_i)$</span>.</p>
<p>First, the model under consideration is something like <span class="math-container">$$Z_i=f(X_i,E_i)+\beta^T\boldsymbol{\tau}_{i,E_i}+\varepsilon_i \qquad \text{Given } S_i^{tr}=1$$</span> along with the strict exogeneity assumption that <span class="math-container">$\mathbb{E}[\varepsilon_i|X_i,E_i,\boldsymbol{\tau}_{i,E_i},S_i^{tr}=1]$</span>=0, (cf. expression (5) in the paper).</p>
<p>Consider the conditional expectation when we are given <span class="math-container">$(X_i,E_i)$</span>, <span class="math-container">$$\mathbb{E}[Z_i|X_i,E_i]=f(X_i,E_i)+\beta^T\mathbb{E}[\boldsymbol{\tau}_{i,E_i}|X_i,E_i]+\mathbb{E}[\varepsilon_i|X_i,E_i]=f(X_i,E_i)+\beta^T\mathbb{E}[\boldsymbol{\tau}_{i,E_i}|X_i,E_i]$$</span>
where the final equality follows by the law of iterated expectation and the strict exogeneity assumption. One may have to additionally condition on <span class="math-container">$S_i^{tr}=1$</span> in the above, but this does not affect the conclusion since <span class="math-container">$\boldsymbol{\tau}_{i,E_i}$</span> is a function of <span class="math-container">$(X_i,E_i)$</span> only (see below).</p>
<p>Whence, we obtain that <span class="math-container">$$\begin{align} (\beta^*,f)&amp;=\arg\min_{\beta,f\in \mathcal{F}}\mathbb{E}[(Z_i-\mathbb{E}[Z_i|X_i,E_i]+\mathbb{E}[Z_i|X_i,E_i]-f(X_i,E_i)-\beta^T\boldsymbol{\tau}_{i,E_i})^2|S_i^{tr}=1] \\
\beta^*&amp;=\arg\min_{\beta}\mathbb{E}[(Z_i-\mathbb{E}[Z_i|X_i,E_i]-\beta^T(\boldsymbol{\tau}_{i,E_i}-\mathbb{E}[\boldsymbol{\tau}_{i,E_i}|X_i,E_i]))^2|S_i^{tr}=1] \end{align}$$</span></p>
<p>Applying the standard regression result, we find <span class="math-container">$$\beta^*=(\mathbb{E}[(\boldsymbol{\tau}_{i,E_i}-\mathbb{E}[\boldsymbol{\tau}_{i,E_i}|X_i,E_i])(\boldsymbol{\tau}_{i,E_i}-\mathbb{E}[\boldsymbol{\tau}_{i,E_i}|X_i,E_i])^T|S_i^{tr}=1])^{-1}\mathbb{E}[(\boldsymbol{\tau}_{i,E_i}-\mathbb{E}[\boldsymbol{\tau}_{i,E_i}|X_i,E_i])(Z_i-\mathbb{E}[Z_i|X_i,E_i])|S_i^{tr}=1]$$</span></p>
<p>This is not quite the stated result yet. First, we simplify the denominator.</p>
<p>By law of iterated expectation, we can turn our expression into <span class="math-container">$$\beta^*=(\mathbb{E}[\mathbb{E}[(\boldsymbol{\tau}_{i,E_i}-\mathbb{E}[\boldsymbol{\tau}_{i,E_i}|X_i,E_i])(\boldsymbol{\tau}_{i,E_i}-\mathbb{E}[\boldsymbol{\tau}_{i,E_i}|X_i,E_i])^T|S_i^{tr}=1,X_i,E_i]|S_i^{tr}=1])^{-1}\mathbb{E}[(\boldsymbol{\tau}_{i,E_i}-\mathbb{E}[\boldsymbol{\tau}_{i,E_i}|X_i,E_i])(Z_i-\mathbb{E}[Z_i|X_i,E_i])|S_i^{tr}=1]$$</span></p>
<p>On page 17 of the paper, it is stated that the policy paths are deterministic functions of <span class="math-container">$(X_i,E_i)$</span> and that therefore <span class="math-container">$\boldsymbol{\tau}_{i,E_i}$</span> are functions of only <span class="math-container">$(X_i,E_i)$</span>. I believe this means we can remove the <span class="math-container">$S_i^{tr}$</span> conditioning from the inner expectation. This results in, <span class="math-container">$$\beta^*=(\mathbb{E}[\mathbb{V}[\boldsymbol{\tau}_{i,E_i}|X_i,E_i]|S_i^{tr}=1])^{-1}\mathbb{E}[(\boldsymbol{\tau}_{i,E_i}-\mathbb{E}[\boldsymbol{\tau}_{i,E_i}|X_i,E_i])(Z_i-\mathbb{E}[Z_i|X_i,E_i])|S_i^{tr}=1]$$</span></p>
<p>Next, we simplify the numerator. To do this, simply note that by law of iterated expectation, the numerator is the covariance of <span class="math-container">$\boldsymbol{\tau}_{i,E_i}$</span> and <span class="math-container">$Z_i$</span> given <span class="math-container">$(X_i,E_i)$</span>. As this question <a href="https://math.stackexchange.com/questions/2513153/show-that-the-covariance-is-also-mathrmcovx-y-ex-exy-exy-ey">https://math.stackexchange.com/questions/2513153/show-that-the-covariance-is-also-mathrmcovx-y-ex-exy-exy-ey</a> points out, we can simplify this covariance to remove the <span class="math-container">$\mathbb{E}[Z_i|X_i,E_i]$</span> term. Therefore, <span class="math-container">$$\beta^*=(\mathbb{E}[\mathbb{V}[\boldsymbol{\tau}_{i,E_i}|X_i,E_i]|S_i^{tr}=1])^{-1}\mathbb{E}[(\boldsymbol{\tau}_{i,E_i}-\mathbb{E}[\boldsymbol{\tau}_{i,E_i}|X_i,E_i])Z_i|S_i^{tr}=1]$$</span></p>
<p>as desired.</p>
<hr />
<h2>Utility of This Methodology</h2>
<p>This sort of methodology of washing out the noise associated with more complicated components is extremely useful, especially in partially linear models such as this one. Finding the optimal <span class="math-container">$f$</span> is much more complicated than finding the optimal <span class="math-container">$\beta$</span>, since it is a functional problem rather than a standard multivariate optimisation problem. By netting out the noise associated with <span class="math-container">$(X_i,E_i)$</span>, we have derived a succinct closed-form expression for <span class="math-container">$\beta^*$</span> which does not include <span class="math-container">$f$</span>. Therefore, we can analyse <span class="math-container">$\beta^*$</span> without ever having to derive a closed-form solution for optimal <span class="math-container">$f$</span>, which, as stated, is intractable.</p>
<p>This approach is precisely what the Robinson (1988, referenced in the paper) estimator does, which was likely part of the inspiration for this methodology. The asymptotics of this sort of estimator for <span class="math-container">$\beta$</span> are often exceptionally good, having a <span class="math-container">$\sqrt{n}$</span> rate convergence no matter the dimension of <span class="math-container">$X_i$</span>. In the paper, there are more complications compared with the Robinson estimator, but the approach is similar.</p>
<p>In the paper, they go on to analyse the components of this <span class="math-container">$\beta^*$</span> expression, which was only tractable due to the application of the FWL theorem. Without applying this methodology, they would have no closed form expression for <span class="math-container">$\beta^*$</span>.</p>

CC BY-SA 4.0
2024-07-29T15:26:29.003Z
Thanks a lot for your explanation. That cleared up a lot for me!
Frédéric Chopin
CC BY-SA 4.0
2024-07-29T12:53:35.497Z
2024-07-29T12:53:35.497Z
Joseph Basford
<p>Let's say we have the following model:
<span class="math-container">$$(\beta^{\star},f^{\star}) := \arg\min_{\beta,f \in \mathcal{F}} \mathbb{E}[\left(Z_i - f(X_i, E_i) -  \beta^\top \boldsymbol{\tau}_{i,E_i}\right)^2|S_i^{tr} = 1],$$</span>
where <span class="math-container">$S_i^{tr}$</span> is an indicator function for treated units and <span class="math-container">$\mathcal{F}$</span> is the set of all possible functions of <span class="math-container">$(X_i,E_i)$</span>. Specifically, I'm considering the case in the paper <a href="https://arxiv.org/pdf/2403.19563" rel="nofollow noreferrer">https://arxiv.org/pdf/2403.19563</a>. The authors say that by the Frisch-Waugh-Lovell (FWL) theorem, we can write
<span class="math-container">$$\beta^{\star} = \left(\mathbb{E}[\mathbb{V}[\boldsymbol{\tau}_{i,E_i}|X_i,E_i]|S_i^{tr} = 1]\right)^{-1}\mathbb{E}[\left(\boldsymbol{\tau}_{i,E_i} - \mathbb{E}[\boldsymbol{\tau}_{i,E_i}|X_i,E_i]\right)Z_i|S_i^{tr} = 1]$$</span> but I do not understand why and how this can solve the problem. My guess is that it serves to isolate the effect of a set of variables on both sides of a regression equation. Yet, I want some mathematical insights to this way of solving this problem. Any help or explanation is appreciated.</p>

CC BY-SA 4.0
2024-07-29T09:45:35.287Z
Are you asking how the authors have used the FWL theorem to get that expression for $\beta^*$? Or are you asking why that expression for $\beta^*$ is useful?
Joseph Basford
CC BY-SA 4.0
2024-07-29T10:24:03.687Z
@JosephBasford Well, the answer to both would be helpful.
Frédéric Chopin
<regression><optimization>
A question on the optimisation problem and FWL theorem
Frédéric Chopin
# A question on the optimisation problem and FWL theorem

(Asked by: username_0 on 2024-07-28)

Let's say we have the following model:
$$(\beta^{\star},f^{\star}) := \arg\min\_{\beta,f \in \mathcal{F}} \mathbb{E}[\left(Z\_i - f(X\_i, E\_i) - \beta^\top \boldsymbol{\tau}\_{i,E\_i}\right)^2|S\_i^{tr} = 1],$$
where $S\_i^{tr}$ is an indicator function for treated units and $\mathcal{F}$ is the set of all possible functions of $(X\_i,E\_i)$. Specifically, I'm considering the case in the paper <https://arxiv.org/pdf/2403.19563>. The authors say that by the Frisch-Waugh-Lovell (FWL) theorem, we can write
$$\beta^{\star} = \left(\mathbb{E}[\mathbb{V}[\boldsymbol{\tau}\_{i,E\_i}|X\_i,E\_i]|S\_i^{tr} = 1]\right)^{-1}\mathbb{E}[\left(\boldsymbol{\tau}\_{i,E\_i} - \mathbb{E}[\boldsymbol{\tau}\_{i,E\_i}|X\_i,E\_i]\right)Z\_i|S\_i^{tr} = 1]$$ but I do not understand why and how this can solve the problem. My guess is that it serves to isolate the effect of a set of variables on both sides of a regression equation. Yet, I want some mathematical insights to this way of solving this problem. Any help or explanation is appreciated.

**Tags:** regression, optimization
> **Comments:**
> **username_1**: Are you asking how the authors have used the FWL theorem to get that expression for $\beta^*$? Or are you asking why that expression for $\beta^*$ is useful?  
> **username_0**: @JosephBasford Well, the answer to both would be helpful.  

---

## 1 answers

### Answer by username_1 on 2024-07-29. Score: 3 | Accepted answer

I haven't read the paper you reference, but I think I can give the general idea of how the methodology works.

## Derivation of $\boldsymbol{\beta}^\*$

To derive the $\beta^\*$ expression, we take the common FWL approach of "washing" out one dimension of the noise in the dependent variable. In this case, by washing out the noise in $Z\_i$ explainable by $(X\_i,E\_i)$, we get the component of $Z\_i$ which is orthogonal to $f(X\_i,E\_i)$.

First, the model under consideration is something like $$Z\_i=f(X\_i,E\_i)+\beta^T\boldsymbol{\tau}\_{i,E\_i}+\varepsilon\_i \qquad \text{Given } S\_i^{tr}=1$$ along with the strict exogeneity assumption that $\mathbb{E}[\varepsilon\_i|X\_i,E\_i,\boldsymbol{\tau}\_{i,E\_i},S\_i^{tr}=1]$=0, (cf. expression (5) in the paper).

Consider the conditional expectation when we are given $(X\_i,E\_i)$, $$\mathbb{E}[Z\_i|X\_i,E\_i]=f(X\_i,E\_i)+\beta^T\mathbb{E}[\boldsymbol{\tau}\_{i,E\_i}|X\_i,E\_i]+\mathbb{E}[\varepsilon\_i|X\_i,E\_i]=f(X\_i,E\_i)+\beta^T\mathbb{E}[\boldsymbol{\tau}\_{i,E\_i}|X\_i,E\_i]$$
where the final equality follows by the law of iterated expectation and the strict exogeneity assumption. One may have to additionally condition on $S\_i^{tr}=1$ in the above, but this does not affect the conclusion since $\boldsymbol{\tau}\_{i,E\_i}$ is a function of $(X\_i,E\_i)$ only (see below).

Whence, we obtain that $$\begin{align} (\beta^\*,f)&=\arg\min\_{\beta,f\in \mathcal{F}}\mathbb{E}[(Z\_i-\mathbb{E}[Z\_i|X\_i,E\_i]+\mathbb{E}[Z\_i|X\_i,E\_i]-f(X\_i,E\_i)-\beta^T\boldsymbol{\tau}\_{i,E\_i})^2|S\_i^{tr}=1] \\
\beta^\*&=\arg\min\_{\beta}\mathbb{E}[(Z\_i-\mathbb{E}[Z\_i|X\_i,E\_i]-\beta^T(\boldsymbol{\tau}\_{i,E\_i}-\mathbb{E}[\boldsymbol{\tau}\_{i,E\_i}|X\_i,E\_i]))^2|S\_i^{tr}=1] \end{align}$$

Applying the standard regression result, we find $$\beta^\*=(\mathbb{E}[(\boldsymbol{\tau}\_{i,E\_i}-\mathbb{E}[\boldsymbol{\tau}\_{i,E\_i}|X\_i,E\_i])(\boldsymbol{\tau}\_{i,E\_i}-\mathbb{E}[\boldsymbol{\tau}\_{i,E\_i}|X\_i,E\_i])^T|S\_i^{tr}=1])^{-1}\mathbb{E}[(\boldsymbol{\tau}\_{i,E\_i}-\mathbb{E}[\boldsymbol{\tau}\_{i,E\_i}|X\_i,E\_i])(Z\_i-\mathbb{E}[Z\_i|X\_i,E\_i])|S\_i^{tr}=1]$$

This is not quite the stated result yet. First, we simplify the denominator.

By law of iterated expectation, we can turn our expression into $$\beta^\*=(\mathbb{E}[\mathbb{E}[(\boldsymbol{\tau}\_{i,E\_i}-\mathbb{E}[\boldsymbol{\tau}\_{i,E\_i}|X\_i,E\_i])(\boldsymbol{\tau}\_{i,E\_i}-\mathbb{E}[\boldsymbol{\tau}\_{i,E\_i}|X\_i,E\_i])^T|S\_i^{tr}=1,X\_i,E\_i]|S\_i^{tr}=1])^{-1}\mathbb{E}[(\boldsymbol{\tau}\_{i,E\_i}-\mathbb{E}[\boldsymbol{\tau}\_{i,E\_i}|X\_i,E\_i])(Z\_i-\mathbb{E}[Z\_i|X\_i,E\_i])|S\_i^{tr}=1]$$

On page 17 of the paper, it is stated that the policy paths are deterministic functions of $(X\_i,E\_i)$ and that therefore $\boldsymbol{\tau}\_{i,E\_i}$ are functions of only $(X\_i,E\_i)$. I believe this means we can remove the $S\_i^{tr}$ conditioning from the inner expectation. This results in, $$\beta^\*=(\mathbb{E}[\mathbb{V}[\boldsymbol{\tau}\_{i,E\_i}|X\_i,E\_i]|S\_i^{tr}=1])^{-1}\mathbb{E}[(\boldsymbol{\tau}\_{i,E\_i}-\mathbb{E}[\boldsymbol{\tau}\_{i,E\_i}|X\_i,E\_i])(Z\_i-\mathbb{E}[Z\_i|X\_i,E\_i])|S\_i^{tr}=1]$$

Next, we simplify the numerator. To do this, simply note that by law of iterated expectation, the numerator is the covariance of $\boldsymbol{\tau}\_{i,E\_i}$ and $Z\_i$ given $(X\_i,E\_i)$. As this question <https://math.stackexchange.com/questions/2513153/show-that-the-covariance-is-also-mathrmcovx-y-ex-exy-exy-ey> points out, we can simplify this covariance to remove the $\mathbb{E}[Z\_i|X\_i,E\_i]$ term. Therefore, $$\beta^\*=(\mathbb{E}[\mathbb{V}[\boldsymbol{\tau}\_{i,E\_i}|X\_i,E\_i]|S\_i^{tr}=1])^{-1}\mathbb{E}[(\boldsymbol{\tau}\_{i,E\_i}-\mathbb{E}[\boldsymbol{\tau}\_{i,E\_i}|X\_i,E\_i])Z\_i|S\_i^{tr}=1]$$

as desired.

---

## Utility of This Methodology

This sort of methodology of washing out the noise associated with more complicated components is extremely useful, especially in partially linear models such as this one. Finding the optimal $f$ is much more complicated than finding the optimal $\beta$, since it is a functional problem rather than a standard multivariate optimisation problem. By netting out the noise associated with $(X\_i,E\_i)$, we have derived a succinct closed-form expression for $\beta^\*$ which does not include $f$. Therefore, we can analyse $\beta^\*$ without ever having to derive a closed-form solution for optimal $f$, which, as stated, is intractable.

This approach is precisely what the Robinson (1988, referenced in the paper) estimator does, which was likely part of the inspiration for this methodology. The asymptotics of this sort of estimator for $\beta$ are often exceptionally good, having a $\sqrt{n}$ rate convergence no matter the dimension of $X\_i$. In the paper, there are more complications compared with the Robinson estimator, but the approach is similar.

In the paper, they go on to analyse the components of this $\beta^\*$ expression, which was only tractable due to the application of the FWL theorem. Without applying this methodology, they would have no closed form expression for $\beta^\*$.

> **Comments:**
> **username_0**: Thanks a lot for your explanation. That cleared up a lot for me!  

---

