<p>First,
<span class="math-container">$$\sqrt{N}(\hat\beta - \beta) = \left( \frac{1}{N} \sum_{i=1}^N X_i X_i^T \right)^{-1} \frac{1}{\sqrt{N}} \sum_{i=1}^N X_i e_i.$$</span>
(No. You can't take plim.)</p>

<p>Next, apply LLN to the "denominator", and apply CLT to the "numerator". Then, under the assumption that <span class="math-container">$plim \frac{1}{N} \sum_{i=1}^n X_iX_i^T$</span> is nonsingular and that <span class="math-container">$\frac{1}{\sqrt{N}} \sum_{i=1}^N X_i e_i$</span> converges in distribution to a centered Gaussian distribution, you will be able to show that
<span class="math-container">$$\sqrt{N}(\hat\beta - \beta) \rightarrow N(0, C)$$</span>
for some <span class="math-container">$C$</span>, where <span class="math-container">$C$</span> is called the asymptotic variance of <span class="math-container">$\sqrt{N}(\hat\beta - \beta)$</span>. (Please derive <span class="math-container">$C$</span> yourself for this OLS case using "if <span class="math-container">$plim A_n = A$</span> and <span class="math-container">$b_n \rightarrow N(0,B)$</span>, then <span class="math-container">$A_n b_n \rightarrow N(0, ABA^T)$</span>.)</p>

<p>Finally, the "asymptotic variance" of <span class="math-container">$\hat\beta$</span> is <em>defined</em> as
<span class="math-container">$$AV(\hat\beta) = \frac{1}{N} AV( \sqrt{N}(\hat\beta - \beta)).$$</span>
(This is my definition. Others may define it differently. I don't know yours.)
According to this definition, <span class="math-container">$AV(\hat\beta) = \frac{1}{N} C$</span>. You have already derived <span class="math-container">$C$</span> above. Divide it by <span class="math-container">$N$</span>.</p>

<p>One step further: I don't know how you define asymptotic variances. Depending on the exact definition of <span class="math-container">$AV$</span>, you may need to remove <span class="math-container">$\lim$</span> and <span class="math-container">$plim$</span> from <span class="math-container">$C$</span> and cancel <span class="math-container">$N$</span>'s here and there. Some people also mean by <span class="math-container">$AV(\hat\beta)$</span> a consistent estimator of <span class="math-container">$C$</span> divided by <span class="math-container">$N$</span>.</p>

CC BY-SA 4.0
2019-10-09T16:17:46.550Z
The question was how to derive $C$.
ahorn
CC BY-SA 4.0
2019-10-10T05:48:06.933Z
Well, if _iid_, $A = E[X_i X_i^T]^{-1}$ and $B = E[X_i e_i^2 X_i^T]$ as OP writes, and $C=ABA^T$. OP derives $C$ (though a bit clumsy); what OP misses is $AV(\hat\beta)$, I thought. Only OP knows.
chan1142
CC BY-SA 4.0
2019-10-03T23:41:59.960Z
2019-10-03T23:41:59.960Z
chan1142
<p>Note that is it customary in econometrics to put the explanatory variables on the columns of <span class="math-container">$\boldsymbol X$</span>, and the observations on the rows. The question has switched these around; I will use the notation in the question.</p>

<p>The <strong>covariance matrix of <span class="math-container">$\boldsymbol{{\hat\beta}}$</span></strong> is <span class="math-container">$$\sigma^2\cdot \mathrm E_{\boldsymbol X}\left[\left(\boldsymbol {XX^T}\right)^{-1}\right]$$</span>
where an unbiased estimate of <span class="math-container">$\sigma^2$</span> is <span class="math-container">$\frac 1{N-K}\sum_{i=1}^N e_i e_i $</span>. This setting (with the expectation operation used) assumes that <span class="math-container">$\boldsymbol X$</span> is <a href="https://en.wikipedia.org/wiki/Stochastic" rel="nofollow noreferrer">stochastic</a>, i.e. that we cannot fix <span class="math-container">$\boldsymbol X$</span> in repeated sampling. My point is that this is not a <em>distribution</em>, as claimed in the question. The following is a distribution: <span class="math-container">$$\sqrt N \left(\boldsymbol{\hat\beta} - \boldsymbol\beta\right) \overset a\sim \mathrm N\left(\ 0\,,\ \ N \sigma^2\cdot \mathrm E_{\boldsymbol X}\left[\left(\boldsymbol {XX^T}\right)^{-1}\right]\right).$$</span>
Note that the above states that <span class="math-container">$\sqrt N \left(\boldsymbol{\hat\beta} - \boldsymbol\beta\right)$</span> is asymptotically normally distributed and centred on <span class="math-container">$0$</span> (if we didn't multiply by <span class="math-container">$\sqrt N$</span>, the distribution would converge to a spike, because <span class="math-container">$\boldsymbol{\hat\beta}$</span> is consistent). </p>

<p>In your question, you've looked at the <strong>bias</strong> of <span class="math-container">$\boldsymbol{\hat\beta}$</span>, which is expected to be <span class="math-container">$0$</span>. Note that <span class="math-container">$\sum_{i=1}^N \boldsymbol {x_i}e_i$</span> is expected to be <span class="math-container">$0$</span>.</p>

CC BY-SA 4.0
2019-10-09T16:17:03.007Z
2019-10-09T16:23:36.787Z
2019-10-09T16:23:36.787Z
ahorn
<p>The asymptotic sampling distribution, after taking plim, of the OLS estimator is given by <span class="math-container">$\sqrt{N}(\hat{\beta}-\beta) = E[X_iX_i^T]^{-1} \left(1/\sqrt{N} \sum_{I=1}^NX_ie_i \right) $</span></p>

<p>It must be shown that the asymptotic variance can be written as: <span class="math-container">$ Var[E[X_iX_i^T]^{-1} 1/\sqrt{N} \sum_{I=1}^NX_ie_i] = \left( E[X_iX_i^T]^{-1} \right) 
   \left( E[X_iX_i^T e_i^2]\right)
    \left( E[X_i X_i^T]^{-1} \right) $</span></p>

<p>Because the variance is linear in parameters, my idea was to start like this:</p>

<p><span class="math-container">$V[\sqrt{N}(\hat{\beta}-\beta)] = \sqrt{N}Var[\hat{\beta}] + 0$</span></p>

<p>Because the <span class="math-container">$Var[\beta]=0$</span>. Thus one could use the fact that <span class="math-container">$Var[\hat{\beta}]=E[(\hat{\beta}-\beta)(\hat{\beta}-\beta)^T]$</span>. </p>

<p>However, while there seems to be a relationship I stuck at this point, mostly because of the <span class="math-container">$\sqrt{N}$</span>. Thanks for help.</p>

CC BY-SA 4.0
2019-10-03T00:26:23.793Z
That is unreadable to me.  Try using LaTeX surrounded by $ signs.  And perhaps also what you have attempted yourself
Henry
CC BY-SA 4.0
2019-10-09T16:22:30.303Z
I'm downvoting this because I don't understand why you're looking at the bias of $\boldsymbol{\hat\beta}$.
ahorn
<econometrics>
How to derive the asymptotic variance from the sampling distribution of the OLS estimator?
user24435
# How to derive the asymptotic variance from the sampling distribution of the OLS estimator?

(Asked by: username_0 on 2019-10-02)

The asymptotic sampling distribution, after taking plim, of the OLS estimator is given by $\sqrt{N}(\hat{\beta}-\beta) = E[X\_iX\_i^T]^{-1} \left(1/\sqrt{N} \sum\_{I=1}^NX\_ie\_i \right) $

It must be shown that the asymptotic variance can be written as: $ Var[E[X\_iX\_i^T]^{-1} 1/\sqrt{N} \sum\_{I=1}^NX\_ie\_i] = \left( E[X\_iX\_i^T]^{-1} \right)
\left( E[X\_iX\_i^T e\_i^2]\right)
\left( E[X\_i X\_i^T]^{-1} \right) $

Because the variance is linear in parameters, my idea was to start like this:

$V[\sqrt{N}(\hat{\beta}-\beta)] = \sqrt{N}Var[\hat{\beta}] + 0$

Because the $Var[\beta]=0$. Thus one could use the fact that $Var[\hat{\beta}]=E[(\hat{\beta}-\beta)(\hat{\beta}-\beta)^T]$.

However, while there seems to be a relationship I stuck at this point, mostly because of the $\sqrt{N}$. Thanks for help.

**Tags:** econometrics
> **Comments:**
> **username_1**: That is unreadable to me.  Try using LaTeX surrounded by $ signs.  And perhaps also what you have attempted yourself  
> **username_2**: I'm downvoting this because I don't understand why you're looking at the bias of $\boldsymbol{\hat\beta}$.  

---

## 2 answers

### Answer by username_3 on 2019-10-03. Score: 0

First,
$$\sqrt{N}(\hat\beta - \beta) = \left( \frac{1}{N} \sum\_{i=1}^N X\_i X\_i^T \right)^{-1} \frac{1}{\sqrt{N}} \sum\_{i=1}^N X\_i e\_i.$$
(No. You can't take plim.)

Next, apply LLN to the "denominator", and apply CLT to the "numerator". Then, under the assumption that $plim \frac{1}{N} \sum\_{i=1}^n X\_iX\_i^T$ is nonsingular and that $\frac{1}{\sqrt{N}} \sum\_{i=1}^N X\_i e\_i$ converges in distribution to a centered Gaussian distribution, you will be able to show that
$$\sqrt{N}(\hat\beta - \beta) \rightarrow N(0, C)$$
for some $C$, where $C$ is called the asymptotic variance of $\sqrt{N}(\hat\beta - \beta)$. (Please derive $C$ yourself for this OLS case using "if $plim A\_n = A$ and $b\_n \rightarrow N(0,B)$, then $A\_n b\_n \rightarrow N(0, ABA^T)$.)

Finally, the "asymptotic variance" of $\hat\beta$ is *defined* as
$$AV(\hat\beta) = \frac{1}{N} AV( \sqrt{N}(\hat\beta - \beta)).$$
(This is my definition. Others may define it differently. I don't know yours.)
According to this definition, $AV(\hat\beta) = \frac{1}{N} C$. You have already derived $C$ above. Divide it by $N$.

One step further: I don't know how you define asymptotic variances. Depending on the exact definition of $AV$, you may need to remove $\lim$ and $plim$ from $C$ and cancel $N$'s here and there. Some people also mean by $AV(\hat\beta)$ a consistent estimator of $C$ divided by $N$.

> **Comments:**
> **username_2**: The question was how to derive $C$.  
> **username_3**: Well, if _iid_, $A = E[X_i X_i^T]^{-1}$ and $B = E[X_i e_i^2 X_i^T]$ as OP writes, and $C=ABA^T$. OP derives $C$ (though a bit clumsy); what OP misses is $AV(\hat\beta)$, I thought. Only OP knows.  

---

### Answer by username_2 on 2019-10-09. Score: 0

Note that is it customary in econometrics to put the explanatory variables on the columns of $\boldsymbol X$, and the observations on the rows. The question has switched these around; I will use the notation in the question.

The **covariance matrix of $\boldsymbol{{\hat\beta}}$** is $$\sigma^2\cdot \mathrm E\_{\boldsymbol X}\left[\left(\boldsymbol {XX^T}\right)^{-1}\right]$$
where an unbiased estimate of $\sigma^2$ is $\frac 1{N-K}\sum\_{i=1}^N e\_i e\_i $. This setting (with the expectation operation used) assumes that $\boldsymbol X$ is [stochastic](https://en.wikipedia.org/wiki/Stochastic), i.e. that we cannot fix $\boldsymbol X$ in repeated sampling. My point is that this is not a *distribution*, as claimed in the question. The following is a distribution: $$\sqrt N \left(\boldsymbol{\hat\beta} - \boldsymbol\beta\right) \overset a\sim \mathrm N\left(\ 0\,,\ \ N \sigma^2\cdot \mathrm E\_{\boldsymbol X}\left[\left(\boldsymbol {XX^T}\right)^{-1}\right]\right).$$
Note that the above states that $\sqrt N \left(\boldsymbol{\hat\beta} - \boldsymbol\beta\right)$ is asymptotically normally distributed and centred on $0$ (if we didn't multiply by $\sqrt N$, the distribution would converge to a spike, because $\boldsymbol{\hat\beta}$ is consistent).

In your question, you've looked at the **bias** of $\boldsymbol{\hat\beta}$, which is expected to be $0$. Note that $\sum\_{i=1}^N \boldsymbol {x\_i}e\_i$ is expected to be $0$.


---

