<p>Yes your vague recollection is correct. You can verify it with a trivial example.</p>
<p>Lets take <span class="math-container">$\bar{X}=10$</span>, with <span class="math-container">$\sigma(X)=2$</span> and <span class="math-container">$n=30$</span>. So the <span class="math-container">$95\%$</span> confidence interval (using student's t distribution) is approximately given by  <span class="math-container">$10 ± 0.75$</span>.</p>
<p>Now suppose <span class="math-container">$\bar{Y}=11$</span> with <span class="math-container">$\sigma(Y)=1$</span> and <span class="math-container">$n=30$</span> so the <span class="math-container">$95\%$</span> confidence interval is given by approximately <span class="math-container">$11 ± 0.37$</span>.</p>
<p>Confidence intervals above clearly overlap since for <span class="math-container">$\bar{X}$</span> the upper bound is 10.75, but for <span class="math-container">$\bar{Y}$</span> the lower bound is <span class="math-container">$10.63$</span> but when we actually do the t-test for difference in their means we get:</p>
<p><span class="math-container">$$ t=\frac{10-11}{\sqrt{\left(\frac{2^2}{30}+\frac{1^2}{30}\right)}} \\
 \approx -2.5 $$</span></p>
<p>Hence in this case we clearly can reject the null hypothesis that the means are same even at <span class="math-container">$5\%$</span> (in fact the <span class="math-container">$t$</span>-stat is very close to being rejected at <span class="math-container">$1\%$</span> even) despite that their confidence levels happen to overlap.</p>

CC BY-SA 4.0
2021-02-11T13:38:45.193Z
Thanks, very helpful! But I remain a bit confused. Say $\bar{Y} > \bar{X}$ as in your example. Then the intervals overlap (using the CLT approximation) iff $\bar{Y} - 1.96SE(\bar{Y}) > \bar{X} + 1.96SE(\bar{X})$, or $\bar{Y} -  \bar{X} > 1.96[SE(\bar{X}) + SE(\bar{Y})]$. Meanwhile, the means are `statistically different' iff $\bar{Y} - \bar{X} - 1.96SE(\bar{X} - \bar{Y}) = \bar{Y} - \bar{X} - 1.96[SE(\bar{X}) + SE(\bar{Y})] > 0$: same inequality! So what am I missing?
afreelunch
CC BY-SA 4.0
2021-02-11T13:40:37.683Z
Perhaps I am wrong that the standard error of the difference in means is the sum of the standard errors of the means?
afreelunch
CC BY-SA 4.0
2021-02-11T13:51:16.463Z
@afreelunch yes that does not hold since generally (save for special cases) $se(X)-se(Y)\neq se(Y-X)$  (also inside se() you will not add bar - bar $\bar{X}$ stands for mean of random variable $X$ whether se(X) is standard error of random variable X).
1muflon1
CC BY-SA 4.0
2021-02-11T16:16:45.390Z
By 'standard error', I mean standard deviation of the sample mean; not the standard deviation of the random variable (I think this is the usual terminology?)
afreelunch
CC BY-SA 4.0
2021-02-11T16:22:43.860Z
But I think I have found my mistake: $$SE[\bar{X} - \bar{Y}] = Var[\bar{X} - \bar{Y}]^{0.5} = [Var[\bar{X}] + Var[\bar{Y}]]^{0.5} = [SE[\bar{X}]^2 + SE[\bar{Y}]^2]^{0.5}$$ which does not generally equal $$SE[\bar{X}] + SE[\bar{Y}]$$ So the standard error of the difference in means is not equal to the sum of standard errors.
afreelunch
CC BY-SA 4.0
2021-02-11T16:28:33.487Z
@afreelunch usually people use X to denote random variable from which you make sample e.g. $X_1,X_2...$. Regardless $se(X) -se(Y) \neq se(X-Y)$ will hold whether we are talking about standard deviation of random variable or standard deviation of sample means
1muflon1
CC BY-SA 4.0
2021-02-11T16:29:21.803Z
Yes I am using bars to denote sample means! (And $X$ and $Y$ to denote random variables)
afreelunch
CC BY-SA 4.0
2021-02-11T16:30:50.247Z
NB The relevant point is that $se(\bar{X} - \bar{Y}) \neq se(\bar{X}) + se(\bar{Y})$ (plus not minus)
afreelunch
CC BY-SA 4.0
2021-02-10T22:00:34.413Z
2021-02-10T22:00:34.413Z
1muflon1
<p>Suppose that I estimate a (frequentist) confidence interval for the sample mean of a variable <span class="math-container">$X$</span>, say at the 95% level. Suppose I also estimate a 95% confidence for the sample mean of a variable <span class="math-container">$Y$</span>.</p>
<blockquote>
<p>Claim: we cannot reject the hypothesis that <span class="math-container">$X$</span> and <span class="math-container">$Y$</span> have the same
mean at the 5% level if and only if these confidence intervals
overlap.</p>
</blockquote>
<p>Is this true? I have a vague recollection that the answer is 'no'.</p>

<econometrics>
Very basic question about confidence intervals
afreelunch
# Very basic question about confidence intervals

(Asked by: username_0 on 2021-02-10)

Suppose that I estimate a (frequentist) confidence interval for the sample mean of a variable $X$, say at the 95% level. Suppose I also estimate a 95% confidence for the sample mean of a variable $Y$.

> Claim: we cannot reject the hypothesis that $X$ and $Y$ have the same
> mean at the 5% level if and only if these confidence intervals
> overlap.

Is this true? I have a vague recollection that the answer is 'no'.

**Tags:** econometrics

---

## 1 answers

### Answer by username_1 on 2021-02-10. Score: 2 | Accepted answer

Yes your vague recollection is correct. You can verify it with a trivial example.

Lets take $\bar{X}=10$, with $\sigma(X)=2$ and $n=30$. So the $95\%$ confidence interval (using student's t distribution) is approximately given by $10 ± 0.75$.

Now suppose $\bar{Y}=11$ with $\sigma(Y)=1$ and $n=30$ so the $95\%$ confidence interval is given by approximately $11 ± 0.37$.

Confidence intervals above clearly overlap since for $\bar{X}$ the upper bound is 10.75, but for $\bar{Y}$ the lower bound is $10.63$ but when we actually do the t-test for difference in their means we get:

$$ t=\frac{10-11}{\sqrt{\left(\frac{2^2}{30}+\frac{1^2}{30}\right)}} \\
\approx -2.5 $$

Hence in this case we clearly can reject the null hypothesis that the means are same even at $5\%$ (in fact the $t$-stat is very close to being rejected at $1\%$ even) despite that their confidence levels happen to overlap.

> **Comments:**
> **username_0**: Thanks, very helpful! But I remain a bit confused. Say $\bar{Y} > \bar{X}$ as in your example. Then the intervals overlap (using the CLT approximation) iff $\bar{Y} - 1.96SE(\bar{Y}) > \bar{X} + 1.96SE(\bar{X})$, or $\bar{Y} -  \bar{X} > 1.96[SE(\bar{X}) + SE(\bar{Y})]$. Meanwhile, the means are `statistically different' iff $\bar{Y} - \bar{X} - 1.96SE(\bar{X} - \bar{Y}) = \bar{Y} - \bar{X} - 1.96[SE(\bar{X}) + SE(\bar{Y})] > 0$: same inequality! So what am I missing?  
> **username_0**: Perhaps I am wrong that the standard error of the difference in means is the sum of the standard errors of the means?  
> **username_1**: @afreelunch yes that does not hold since generally (save for special cases) $se(X)-se(Y)\neq se(Y-X)$  (also inside se() you will not add bar - bar $\bar{X}$ stands for mean of random variable $X$ whether se(X) is standard error of random variable X).  
> **username_0**: By 'standard error', I mean standard deviation of the sample mean; not the standard deviation of the random variable (I think this is the usual terminology?)  
> **username_0**: But I think I have found my mistake: $$SE[\bar{X} - \bar{Y}] = Var[\bar{X} - \bar{Y}]^{0.5} = [Var[\bar{X}] + Var[\bar{Y}]]^{0.5} = [SE[\bar{X}]^2 + SE[\bar{Y}]^2]^{0.5}$$ which does not generally equal $$SE[\bar{X}] + SE[\bar{Y}]$$ So the standard error of the difference in means is not equal to the sum of standard errors.  
> **username_1**: @afreelunch usually people use X to denote random variable from which you make sample e.g. $X_1,X_2...$. Regardless $se(X) -se(Y) \neq se(X-Y)$ will hold whether we are talking about standard deviation of random variable or standard deviation of sample means  
> **username_0**: Yes I am using bars to denote sample means! (And $X$ and $Y$ to denote random variables)  
> **username_0**: NB The relevant point is that $se(\bar{X} - \bar{Y}) \neq se(\bar{X}) + se(\bar{Y})$ (plus not minus)  

---

