<p>Are the $\hat{u}_t^2$ independent? If yes, the result is indeed a version of the law of large numbers due to Kolmogorov, and that deals with independent (but not necessarily identically distributed) processes.</p>

<p>Here is the formal statement. Suppose that $\{u_t\}_{t=1}^{\infty}$ is a sequence of independent real-valued random variables that satisfies 
\begin{equation*}
\sum_{t=1}^{\infty}{\dfrac{Var(u_t)}{t^2}}&lt;\infty
\end{equation*}</p>

<p>Then the random variable
\begin{equation*}
\dfrac{1}{n} \sum_{t=1}^{n}{[u_t-\mathbb{E}(u_t)]}
\end{equation*}
converges to 0 almost surely.</p>

<p>Edit: the assumption of independence seems stronger than needed, and the following result (an adapted version of the law of large numbers for non-independent processes) might be useful. If $\sigma_{n,m}=Cov(u_n,u_m)$ exists, and if there is $0&lt; \alpha &lt;1$ and $M \in \mathbb{R}$ such that $\sigma_{n,m} &lt; M \alpha^{|n-m|}$ for all $(n,m)$, then
\begin{equation*}
\mathbb{P}(|\dfrac{1}{n}\sum_{t=1}^{n}u_t- \dfrac{1}{n}\sum_{t=1}^{n}\mathbb{E}(u_t)|&gt;\epsilon] &lt; \dfrac{M}{n \epsilon^2}
\end{equation*}
for any $\epsilon$.</p>

CC BY-SA 3.0
2016-03-05T10:29:17.537Z
That's it. Many thanks for the answer. The $û_t$ are not independent, but I think they are 'asymptotically independent'. Either way, I don't like much this way of proving that the variance estimator for the error terms is consistent. I prefer others. I just wanted to know the theorem that would allow that equality to be written. Thanks once again. ;)
An old man in the sea.
CC BY-SA 3.0
2016-03-05T10:32:38.210Z
@Anoldmaninthesea. my pleasure ! :)
Oliv
CC BY-SA 3.0
2016-03-05T10:33:14.740Z
If you happen to know how we could ensure that the theorem may be applied to this I would much obliged. ;)
An old man in the sea.
CC BY-SA 3.0
2016-03-05T10:44:31.577Z
@Anoldmaninthesea. I apologize, but I am not sure I understand. What is the point you are missing? If you replace $u_t$ by $\hat{u}_t^2$ in the statement of the theorem, you get exactly the inequality $\lim_{n \rightarrow +\infty} \frac{1}{n}\sum_{t=1}^{n}{\hat{u}_t^2} = \lim_{n \rightarrow +\infty} \frac{1}{n} \sum_{t=1}^{n}{\mathbb{E}(\hat{u}_t^2)}$, don't you?
Oliv
CC BY-SA 3.0
2016-03-05T10:48:59.210Z
Yes. The thing is the textbook says to assume that I can apply the theorem to $\{\hat u_t\}$ even though they are not independent. I just asked that if you knew why the textbook states that we can use the that theorem, then I would be thankful if you could tell me. ;)
An old man in the sea.
CC BY-SA 3.0
2016-03-05T11:20:12.953Z
@Anoldmaninthesea. I see, sorry for the misunderstanding. I think asymptotic independence is indeed sufficient (I will update my answer).
Oliv
CC BY-SA 3.0
2016-03-06T09:57:03.253Z
+1 Thanks Oliv. Could you tell me where you found this LLN? Thanks ;)
An old man in the sea.
CC BY-SA 3.0
2016-03-05T09:28:15.723Z
2016-03-05T11:26:22.203Z
2016-03-05T11:26:22.203Z
Oliv
<p>A bit more intuitive formulation of a sufficient condition for the  (weak) law of large numbers to hold(which is the one associated with the consistency property), for the average of a collection of non-independent, non-identically distributed random variables with finite variances and covariances, is the following (the "Markov condition"):</p>

<p>$$\text {Var}\left (\frac 1{n}\sum_{t=1}^n x_t\right) \rightarrow 0 $$</p>

<p>This simply says that it is <em>sufficient</em> that the variance of the average goes to zero, which has intuition. Decomposing,</p>

<p>$$\text {Var}\left (\frac 1{n}\sum_{t=1}^n x_t\right) = \frac 1{n^2}\sum_{t=1}^n \text {Var}(x_t) +\frac 1{n^2}\sum_{t\neq s} \text {Cov}(x_t,x_s) \rightarrow 0$$</p>

<p>Since all variances are finite, the first sum goes to zero. As regards the second sum, if each element is correlated with all others irrespective of how many there are, then this (double) sum has $n^2-n$ strictly non-zero elements, i.e of order $O(n^2)$. If this is the case, then <em>it doesn't</em> go to zero, and the sufficient condition does not hold. </p>

<p>The easiest way to see this is to assume that all rv's are equicorrelated: if </p>

<p>$$\text {Cov}(x_t,x_s) = c\;\forall t,s \implies \text {Var}\left (\frac 1{n}\sum_{t=1}^n x_t\right) = \frac 1{n^2}\sum_{t=1}^n \text {Var}(x_t) +\left(1-\frac 1n\right) c\rightarrow c$$</p>

<p>By the way this is how one can glimpse why, with "everybody with everybody" correlation, the sample mean remains a random variable irrespective of the sample size.</p>

<p>So what it takes to obtain the weak $\text{LLN}$?  </p>

<p>We may assume $m-$dependence, namely that each rv is correlated only with $m$ others, with $m$ being a fixed number. This will send the variance of the sample mean to zero.</p>

<p>We may assume that as the sample size increases, the number of non-zero covariances increases with it, but not at the same rate: $m(n)/n \rightarrow 0$.</p>

<p>If we want to maintain that each rv is correlated with every other (which is the case of the OP since it deals with estimation residuals), then we arrive at the condition stated in @Oliv's answer: it has intuitive meaning (and justification) as "covariance diminishes as distance increases" only when there is a natural ordering of the sample (temporal or spatial). When the sample is bona-fide cross-sectional and one can re-order the rv's at will, the condition is just mathematical.</p>

CC BY-SA 3.0
2016-03-05T22:26:24.250Z
Alecos, thanks once again for your answer. By the way, where did you find this? In a textbook, some class notes? Thanks ;)
An old man in the sea.
CC BY-SA 3.0
2016-03-05T22:54:20.220Z
What, the "Markov condition"? The first books that come to mind is Aris Spanos' book, http://www.cambridge.org/gr/academic/subjects/economics/econometrics-statistics-and-mathematical-economics/probability-theory-and-statistical-inference-econometric-modeling-observational-data   chapter 9.
Alecos Papadopoulos
CC BY-SA 3.0
2016-03-05T19:17:35.530Z
2016-03-05T19:17:35.530Z
Alecos Papadopoulos
<p>The usual LLN I've seen, the theorem states that $1/n \sum u_t \rightarrow^p E(u_t)=\mu$,where the expected value is independent of t. However, I cannot apply it to the exercise below. </p>

<p>In the picture, the author seems to be using $ \displaystyle \frac{1}{n} \sum u_t \rightarrow^p \lim_{n \rightarrow \infty}\frac{1}{n} \sum E(u_t)$</p>

<p><a href="https://i.sstatic.net/fFg2o.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/fFg2o.png" alt="enter image description here"></a></p>

<p>Result (3.44) is $Var(\hat u_t^2)=E(\hat u_t^2)=(1-h_t)\sigma^2$, and it's also the result they refer to as 'in the previous exercise', and (3.46) is just the first equation, without the plims.</p>

<p>So, what LLN can I apply?</p>

<p>Any help would be appreciated.</p>

<econometrics>
Is this use of a Law of Large Numbers correct?
An old man in the sea.
# Is this use of a Law of Large Numbers correct?

(Asked by: username_0 on 2016-03-04)

The usual LLN I've seen, the theorem states that $1/n \sum u\_t \rightarrow^p E(u\_t)=\mu$,where the expected value is independent of t. However, I cannot apply it to the exercise below.

In the picture, the author seems to be using $ \displaystyle \frac{1}{n} \sum u\_t \rightarrow^p \lim\_{n \rightarrow \infty}\frac{1}{n} \sum E(u\_t)$

[![enter image description here](https://i.sstatic.net/fFg2o.png)](https://i.sstatic.net/fFg2o.png)

Result (3.44) is $Var(\hat u\_t^2)=E(\hat u\_t^2)=(1-h\_t)\sigma^2$, and it's also the result they refer to as 'in the previous exercise', and (3.46) is just the first equation, without the plims.

So, what LLN can I apply?

Any help would be appreciated.

**Tags:** econometrics

---

## 2 answers

### Answer by username_1 on 2016-03-05. Score: 2 | Accepted answer

Are the $\hat{u}\_t^2$ independent? If yes, the result is indeed a version of the law of large numbers due to Kolmogorov, and that deals with independent (but not necessarily identically distributed) processes.

Here is the formal statement. Suppose that $\{u\_t\}\_{t=1}^{\infty}$ is a sequence of independent real-valued random variables that satisfies
\begin{equation\*}
\sum\_{t=1}^{\infty}{\dfrac{Var(u\_t)}{t^2}}<\infty
\end{equation\*}

Then the random variable
\begin{equation\*}
\dfrac{1}{n} \sum\_{t=1}^{n}{[u\_t-\mathbb{E}(u\_t)]}
\end{equation\*}
converges to 0 almost surely.

Edit: the assumption of independence seems stronger than needed, and the following result (an adapted version of the law of large numbers for non-independent processes) might be useful. If $\sigma\_{n,m}=Cov(u\_n,u\_m)$ exists, and if there is $0< \alpha <1$ and $M \in \mathbb{R}$ such that $\sigma\_{n,m} < M \alpha^{|n-m|}$ for all $(n,m)$, then
\begin{equation\*}
\mathbb{P}(|\dfrac{1}{n}\sum\_{t=1}^{n}u\_t- \dfrac{1}{n}\sum\_{t=1}^{n}\mathbb{E}(u\_t)|>\epsilon] < \dfrac{M}{n \epsilon^2}
\end{equation\*}
for any $\epsilon$.

> **Comments:**
> **username_0**: That's it. Many thanks for the answer. The $û_t$ are not independent, but I think they are 'asymptotically independent'. Either way, I don't like much this way of proving that the variance estimator for the error terms is consistent. I prefer others. I just wanted to know the theorem that would allow that equality to be written. Thanks once again. ;)  
> **username_1**: @Anoldmaninthesea. my pleasure ! :)  
> **username_0**: If you happen to know how we could ensure that the theorem may be applied to this I would much obliged. ;)  
> **username_1**: @Anoldmaninthesea. I apologize, but I am not sure I understand. What is the point you are missing? If you replace $u_t$ by $\hat{u}_t^2$ in the statement of the theorem, you get exactly the inequality $\lim_{n \rightarrow +\infty} \frac{1}{n}\sum_{t=1}^{n}{\hat{u}_t^2} = \lim_{n \rightarrow +\infty} \frac{1}{n} \sum_{t=1}^{n}{\mathbb{E}(\hat{u}_t^2)}$, don't you?  
> **username_0**: Yes. The thing is the textbook says to assume that I can apply the theorem to $\{\hat u_t\}$ even though they are not independent. I just asked that if you knew why the textbook states that we can use the that theorem, then I would be thankful if you could tell me. ;)  
> **username_1**: @Anoldmaninthesea. I see, sorry for the misunderstanding. I think asymptotic independence is indeed sufficient (I will update my answer).  
> **username_0**: +1 Thanks Oliv. Could you tell me where you found this LLN? Thanks ;)  

---

### Answer by username_2 on 2016-03-05. Score: 0

A bit more intuitive formulation of a sufficient condition for the (weak) law of large numbers to hold(which is the one associated with the consistency property), for the average of a collection of non-independent, non-identically distributed random variables with finite variances and covariances, is the following (the "Markov condition"):

$$\text {Var}\left (\frac 1{n}\sum\_{t=1}^n x\_t\right) \rightarrow 0 $$

This simply says that it is *sufficient* that the variance of the average goes to zero, which has intuition. Decomposing,

$$\text {Var}\left (\frac 1{n}\sum\_{t=1}^n x\_t\right) = \frac 1{n^2}\sum\_{t=1}^n \text {Var}(x\_t) +\frac 1{n^2}\sum\_{t\neq s} \text {Cov}(x\_t,x\_s) \rightarrow 0$$

Since all variances are finite, the first sum goes to zero. As regards the second sum, if each element is correlated with all others irrespective of how many there are, then this (double) sum has $n^2-n$ strictly non-zero elements, i.e of order $O(n^2)$. If this is the case, then *it doesn't* go to zero, and the sufficient condition does not hold.

The easiest way to see this is to assume that all rv's are equicorrelated: if

$$\text {Cov}(x\_t,x\_s) = c\;\forall t,s \implies \text {Var}\left (\frac 1{n}\sum\_{t=1}^n x\_t\right) = \frac 1{n^2}\sum\_{t=1}^n \text {Var}(x\_t) +\left(1-\frac 1n\right) c\rightarrow c$$

By the way this is how one can glimpse why, with "everybody with everybody" correlation, the sample mean remains a random variable irrespective of the sample size.

So what it takes to obtain the weak $\text{LLN}$?

We may assume $m-$dependence, namely that each rv is correlated only with $m$ others, with $m$ being a fixed number. This will send the variance of the sample mean to zero.

We may assume that as the sample size increases, the number of non-zero covariances increases with it, but not at the same rate: $m(n)/n \rightarrow 0$.

If we want to maintain that each rv is correlated with every other (which is the case of the OP since it deals with estimation residuals), then we arrive at the condition stated in @Oliv's answer: it has intuitive meaning (and justification) as "covariance diminishes as distance increases" only when there is a natural ordering of the sample (temporal or spatial). When the sample is bona-fide cross-sectional and one can re-order the rv's at will, the condition is just mathematical.

> **Comments:**
> **username_0**: Alecos, thanks once again for your answer. By the way, where did you find this? In a textbook, some class notes? Thanks ;)  
> **username_2**: What, the "Markov condition"? The first books that come to mind is Aris Spanos' book, http://www.cambridge.org/gr/academic/subjects/economics/econometrics-statistics-and-mathematical-economics/probability-theory-and-statistical-inference-econometric-modeling-observational-data   chapter 9.  

---

