<p>I will attempt to explain the distinction using the simplest example: the sample mean. Suppose we have an iid sample of random variables <span class="math-container">$\{X_i\}_{i=1}^n$</span>. Then define the sample mean as <span class="math-container">$\bar{X}_n$</span>. As the sample size grows, our value of the sample mean changes, hence the subscript <span class="math-container">$n$</span> to emphasize that our sample mean depends on the sample size. </p>

<p>Noting that <span class="math-container">$\bar{X}_n$</span> itself is a random variable, we can define a sequence of random variables, where elements of the sequence are indexed by different samples (sample size is growing), i.e. <span class="math-container">$\{\bar{X}_n\}_{n=1}^{\infty}$</span>. The weak law of large numbers (WLLN) tells us that so long as <span class="math-container">$E(X_1^2)&lt;\infty$</span>, that 
<span class="math-container">$$plim\bar{X}_n = \mu,$$</span>
or equivalently 
<span class="math-container">$$\bar{X}_n \rightarrow_P \mu,$$</span></p>

<p>where <span class="math-container">$\mu=E(X_1)$</span>. Formally, convergence in probability is defined as
<span class="math-container">$$\forall \epsilon&gt;0, \lim_{n \rightarrow \infty} P(|\bar{X}_n - \mu| &lt;\epsilon)=1. $$</span>
In other words, the probability of our estimate being within <span class="math-container">$\epsilon$</span> from the true value tends to 1 as <span class="math-container">$n \rightarrow \infty$</span>. Convergence in probability gives us confidence our estimators perform well with large samples.</p>

<p>Convergence in distribution tell us something very different and is primarily used for hypothesis testing. Under the same distributional assumptions described above, CLT gives us that
<span class="math-container">$$\sqrt{n}(\bar{X}_n-\mu) \rightarrow_D N(0,E(X_1^2)).$$</span>
Convergence in distribution means that the cdf of the left-hand size converges at all continuity points to the cdf of the right-hand side, i.e.
<span class="math-container">$$\lim_{n \rightarrow \infty} F_n(x) = F(x),$$</span>
where <span class="math-container">$F_n(x)$</span> is the cdf of <span class="math-container">$\sqrt{n}(\bar{X}_n-\mu)$</span> and <span class="math-container">$F(x)$</span> is the cdf for a <span class="math-container">$N(0,E(X_1^2))$</span> distribution. Knowing the limiting distribution allows us to test hypotheses about the sample mean (or whatever estimate we are generating). </p>

CC BY-SA 4.0
2019-03-17T16:59:50.420Z
Your definition of convergence in probability is more demanding than the standard definition. For example, suppose $X_n = 1$ with probability $1/n$, with $X_n = 0$ otherwise. It’s clear that $X_n$ must converge in probability to $0$. However, $X_n$ does not converge to $0$ according to your definition, because we always have that $P(|X_n| < \varepsilon ) \neq 1$ for $\varepsilon < 1$ and any $n$.
Theoretical Economist
CC BY-SA 4.0
2019-03-17T18:09:42.133Z
Yes, you are right. I posted my answer too quickly and made an error in writing the definition of weak convergence. I have corrected my post.
dlnB
CC BY-SA 4.0
2019-03-16T15:09:09.980Z
2019-03-17T18:09:07.733Z
2019-03-17T18:09:07.733Z
dlnB
<p>Im a little confused about the difference of these two concepts, especially the convergence of probability. I understand that <span class="math-container">$X_{n} \overset{p}{\to} Z $</span>  if    <span class="math-container">$Pr(|X_{n} - Z|&gt;\epsilon)=0$</span> for any <span class="math-container">$\epsilon &gt;0$</span> when <span class="math-container">$n \rightarrow \infty$</span>. </p>

<p>I just need some clarification on what the subscript <span class="math-container">$n$</span> means and what <span class="math-container">$Z$</span> means. Is <span class="math-container">$n$</span> the sample size? is <span class="math-container">$Z$</span> a specific value, or another random variable? If it is another random variable, then wouldn't that mean that convergence in probability implies convergence in distribution? Also, Could you please give me some examples of things that are convergent in distribution but not in probability?</p>

CC BY-SA 4.0
2019-04-28T03:11:39.437Z
A quick example: $X_n = (-1)^n Z$, where $Z \sim N(0,1)$. Then $X_n$ does not converge in probability but $X_n$ converges in distribution to $N(0,1)$ because the distribution of $X_n$ is $N(0,1)$ for all $n$.
chan1142
CC BY-SA 4.0
2019-04-28T03:17:41.943Z
And, no, $n$ is not the sample size. It is just the index of a sequence $X_1,X_2,\ldots$. And $Z$ is a random variable, whatever it may be. In econometrics, your $Z$ is usually nonrandom, but it doesn’t have to be in general.
chan1142
<econometrics><statistics>
Convergence in probability and convergence in distribution
Martin 
# Convergence in probability and convergence in distribution

(Asked by: username_0 on 2019-03-16)

Im a little confused about the difference of these two concepts, especially the convergence of probability. I understand that $X\_{n} \overset{p}{\to} Z $ if $Pr(|X\_{n} - Z|>\epsilon)=0$ for any $\epsilon >0$ when $n \rightarrow \infty$.

I just need some clarification on what the subscript $n$ means and what $Z$ means. Is $n$ the sample size? is $Z$ a specific value, or another random variable? If it is another random variable, then wouldn't that mean that convergence in probability implies convergence in distribution? Also, Could you please give me some examples of things that are convergent in distribution but not in probability?

**Tags:** econometrics, statistics
> **Comments:**
> **username_1**: A quick example: $X_n = (-1)^n Z$, where $Z \sim N(0,1)$. Then $X_n$ does not converge in probability but $X_n$ converges in distribution to $N(0,1)$ because the distribution of $X_n$ is $N(0,1)$ for all $n$.  
> **username_1**: And, no, $n$ is not the sample size. It is just the index of a sequence $X_1,X_2,\ldots$. And $Z$ is a random variable, whatever it may be. In econometrics, your $Z$ is usually nonrandom, but it doesn’t have to be in general.  

---

## 1 answers

### Answer by username_2 on 2019-03-16. Score: 5 | Accepted answer

I will attempt to explain the distinction using the simplest example: the sample mean. Suppose we have an iid sample of random variables $\{X\_i\}\_{i=1}^n$. Then define the sample mean as $\bar{X}\_n$. As the sample size grows, our value of the sample mean changes, hence the subscript $n$ to emphasize that our sample mean depends on the sample size.

Noting that $\bar{X}\_n$ itself is a random variable, we can define a sequence of random variables, where elements of the sequence are indexed by different samples (sample size is growing), i.e. $\{\bar{X}\_n\}\_{n=1}^{\infty}$. The weak law of large numbers (WLLN) tells us that so long as $E(X\_1^2)<\infty$, that
$$plim\bar{X}\_n = \mu,$$
or equivalently
$$\bar{X}\_n \rightarrow\_P \mu,$$

where $\mu=E(X\_1)$. Formally, convergence in probability is defined as
$$\forall \epsilon>0, \lim\_{n \rightarrow \infty} P(|\bar{X}\_n - \mu| <\epsilon)=1. $$
In other words, the probability of our estimate being within $\epsilon$ from the true value tends to 1 as $n \rightarrow \infty$. Convergence in probability gives us confidence our estimators perform well with large samples.

Convergence in distribution tell us something very different and is primarily used for hypothesis testing. Under the same distributional assumptions described above, CLT gives us that
$$\sqrt{n}(\bar{X}\_n-\mu) \rightarrow\_D N(0,E(X\_1^2)).$$
Convergence in distribution means that the cdf of the left-hand size converges at all continuity points to the cdf of the right-hand side, i.e.
$$\lim\_{n \rightarrow \infty} F\_n(x) = F(x),$$
where $F\_n(x)$ is the cdf of $\sqrt{n}(\bar{X}\_n-\mu)$ and $F(x)$ is the cdf for a $N(0,E(X\_1^2))$ distribution. Knowing the limiting distribution allows us to test hypotheses about the sample mean (or whatever estimate we are generating).

> **Comments:**
> **username_3**: Your definition of convergence in probability is more demanding than the standard definition. For example, suppose $X_n = 1$ with probability $1/n$, with $X_n = 0$ otherwise. It’s clear that $X_n$ must converge in probability to $0$. However, $X_n$ does not converge to $0$ according to your definition, because we always have that $P(|X_n| < \varepsilon ) \neq 1$ for $\varepsilon < 1$ and any $n$.  
> **username_2**: Yes, you are right. I posted my answer too quickly and made an error in writing the definition of weak convergence. I have corrected my post.  

---

