<p>We know that $M_X\mathbf{u}= \mathbf{\hat u}$, where $\mathbf{y}=\mathbf{x\beta}+\mathbf{u}$. So even if the error terms are independent, the residuals will not be independent of the other residuals. This is makes the usual assumption of no autocorrelation invalid. </p>

<p>Of course, if we use a HAC estimator then we can circumvent the existence of autocorrelation. However, the HAC estimators that I know of are just of asymptotic behaviour. So, it wouldn't be useful for a small sample size.</p>

CC BY-SA 3.0
2017-05-25T19:39:54.303Z
I don't think I really get what you mean. This is the case in every FWL-Regression. However, this does not change the numerical value of beta. In addition, precision of beta should go up substantially in the extreme example above. Can you elaborate a little bit?
yrx1702
CC BY-SA 3.0
2017-05-25T20:01:07.307Z
What I'm trying to say is that we can use the last equation to estimate the betas by the usual formula, but not to do inference with, since it doesn't comply with the usual hypothesis. In this way, the formula for the estimate works, even though the last equation doesn't comply with the hypothesis of no correlation. At least this is my take on your question.
An old man in the sea.
CC BY-SA 3.0
2017-05-25T20:31:02.457Z
Thanks for your answer! So that would mean that the FWL theorem has mainly theoretical applications?
yrx1702
CC BY-SA 3.0
2017-05-26T09:51:31.070Z
Just wanted to leave it open in case someone else wants to give it a try. Thanks!
yrx1702
CC BY-SA 3.0
2017-05-25T19:14:28.260Z
2017-05-25T19:14:28.260Z
An old man in the sea.
<p>I was talking to a colleague a few days ago and the following question came up:</p>

<p>Why can't we just use projection matrices and the FWL Theorem to "gain" a few degrees of freedom in case of very few observations?</p>

<p>Imagine a case where you want to estimate</p>

<p>$$y_t = \beta_0 + z\beta_1 + x_2\beta_2 + x_3\beta_3 + u_t$$</p>

<p>but unfortunately, you just have 5 observations (extreme example). Luckily, you are just interested in $\beta_1$. Now you could just create some projection matrices in the form </p>

<p>$$ M_x = I - x'(x'x)^{-1}x'$$</p>

<p>and project off your "unwanted" $x_2$ and $x_3$ (following Davidson and MacKinnon). You would be left with the regression</p>

<p>$$M_x y = M_x  z  \beta_1 + residuals$$</p>

<p>where $\beta_1$ is numerically the same as in the regression above. But now you are able to estimate just one (beta-)parameter with 5 observations compared to three parameters before. So, obviously, the estimates improve, don't they?</p>

<p>Where is our mistake?</p>

CC BY-SA 3.0
2017-05-25T18:37:52.867Z
Zen, by gaining degrees of freedom, do you mean when using an F-test to test the significance of $\beta_1$?
An old man in the sea.
CC BY-SA 3.0
2017-05-25T18:53:59.897Z
Degrees of freedom in an F-test can be increased in this way as well as it seems to be possible to get estimates with higher precision this way. I don't really have an analytical argument for it, but especially in cases with few observations it seems that decreasing parameters to estimate might have a big effect on the quality of your regression and this, in a way, feels like "cheating" somehow.
yrx1702
CC BY-SA 3.0
2017-05-25T19:02:15.423Z
I think the last equation doesn't satisfy the usual assumptions, namely that of no autocorrelation. The each residuals is a linear combination of all $u_t$. So, even if $u_t$ are independent, each $\hat u_t$ is not.
An old man in the sea.
CC BY-SA 3.0
2017-05-30T16:45:30.743Z
This would better fit [Cross Validated](http://stats.stackexchange.com) than Economics Stack Exchange.
Richard Hardy
CC BY-SA 3.0
2017-05-30T22:03:23.277Z
I think that the question fits just fine here. See here for previous discussions regarding this: https://economics.meta.stackexchange.com/questions/1484/what-kind-of-math-stat-questions-allowed
jmbejara
<econometrics>
OLS, Frisch-Waugh-Lovell Theorem and few observations
yrx1702
# OLS, Frisch-Waugh-Lovell Theorem and few observations

(Asked by: username_0 on 2017-05-25)

I was talking to a colleague a few days ago and the following question came up:

Why can't we just use projection matrices and the FWL Theorem to "gain" a few degrees of freedom in case of very few observations?

Imagine a case where you want to estimate

$$y\_t = \beta\_0 + z\beta\_1 + x\_2\beta\_2 + x\_3\beta\_3 + u\_t$$

but unfortunately, you just have 5 observations (extreme example). Luckily, you are just interested in $\beta\_1$. Now you could just create some projection matrices in the form

$$ M\_x = I - x'(x'x)^{-1}x'$$

and project off your "unwanted" $x\_2$ and $x\_3$ (following Davidson and MacKinnon). You would be left with the regression

$$M\_x y = M\_x z \beta\_1 + residuals$$

where $\beta\_1$ is numerically the same as in the regression above. But now you are able to estimate just one (beta-)parameter with 5 observations compared to three parameters before. So, obviously, the estimates improve, don't they?

Where is our mistake?

**Tags:** econometrics
> **Comments:**
> **username_1**: Zen, by gaining degrees of freedom, do you mean when using an F-test to test the significance of $\beta_1$?  
> **username_0**: Degrees of freedom in an F-test can be increased in this way as well as it seems to be possible to get estimates with higher precision this way. I don't really have an analytical argument for it, but especially in cases with few observations it seems that decreasing parameters to estimate might have a big effect on the quality of your regression and this, in a way, feels like "cheating" somehow.  
> **username_1**: I think the last equation doesn't satisfy the usual assumptions, namely that of no autocorrelation. The each residuals is a linear combination of all $u_t$. So, even if $u_t$ are independent, each $\hat u_t$ is not.  
> **username_2**: This would better fit [Cross Validated](http://stats.stackexchange.com) than Economics Stack Exchange.  
> **username_3**: I think that the question fits just fine here. See here for previous discussions regarding this: https://economics.meta.stackexchange.com/questions/1484/what-kind-of-math-stat-questions-allowed  

---

## 1 answers

### Answer by username_1 on 2017-05-25. Score: 0 | Accepted answer

We know that $M\_X\mathbf{u}= \mathbf{\hat u}$, where $\mathbf{y}=\mathbf{x\beta}+\mathbf{u}$. So even if the error terms are independent, the residuals will not be independent of the other residuals. This is makes the usual assumption of no autocorrelation invalid.

Of course, if we use a HAC estimator then we can circumvent the existence of autocorrelation. However, the HAC estimators that I know of are just of asymptotic behaviour. So, it wouldn't be useful for a small sample size.

> **Comments:**
> **username_0**: I don't think I really get what you mean. This is the case in every FWL-Regression. However, this does not change the numerical value of beta. In addition, precision of beta should go up substantially in the extreme example above. Can you elaborate a little bit?  
> **username_1**: What I'm trying to say is that we can use the last equation to estimate the betas by the usual formula, but not to do inference with, since it doesn't comply with the usual hypothesis. In this way, the formula for the estimate works, even though the last equation doesn't comply with the hypothesis of no correlation. At least this is my take on your question.  
> **username_0**: Thanks for your answer! So that would mean that the FWL theorem has mainly theoretical applications?  
> **username_0**: Just wanted to leave it open in case someone else wants to give it a try. Thanks!  

---

