<p>This is because of several reasons. The standard logit test uses t-test to test whether coefficients are individually significant.</p>
<p>The t-test is given by:</p>
<p><span class="math-container">$$t=\frac{\beta_i}{se_{\beta_i}}$$</span></p>
<p>where <span class="math-container">$\beta_i$</span> is the coefficient you estimated and <span class="math-container">$se_{\beta_i}$</span> is standard error of the estimated coefficient. In turn standard error is given by <span class="math-container">$se_{\beta_i}=\frac{s}{\sqrt{n}}$</span> where <span class="math-container">$s$</span> is the estimate of standard deviation and <span class="math-container">$n$</span> is the number of observations.</p>
<p>Lastly, to determine whether the variable is significant you compare whether the <span class="math-container">$t$</span> statistics is greater than critical value of the t-statistics (<span class="math-container">$t^*$</span>).</p>
<p>The <span class="math-container">$t^*$</span> depends on several factors. First, it depends on the significance level you want to use (a commonly used level of significance will be 5% or <span class="math-container">$\alpha =0.05$</span>. Second it depends whether you are using two-sided or one sided test, although in practice people would use almost always two sided test when running regression. Third, it depends on the degrees of freedom of the test which depend on number of observations (<span class="math-container">$df_t =n-1$</span>). The magnitude of <span class="math-container">$t^*$</span> has inverse relationship with the number of observations <span class="math-container">$n$</span>, that is higher <span class="math-container">$n$</span> lowers <span class="math-container">$t^*$</span> holding <span class="math-container">$\alpha$</span> and type of test constant.</p>
<p>Now given all the information above there are the following possible reasons why you found that the observations are no longer significant in the subsamples:</p>
<ol>
<li><p><span class="math-container">$\beta_i$</span> in a subsamples is lower than in the joint sample. Perhaps there are some outliers that have bigger impact on the coefficient size when you split the sample into sub-samples.</p>
</li>
<li><p>Standard error in the subsamples is higher. Since subsamples have lower number of observations and since standard error is given as  <span class="math-container">$se_{\beta_i}=\frac{s}{\sqrt{n}}$</span> having lower number of observation, ceteris paribus, increases standard errors of your beta coefficients. Now the standard deviation <span class="math-container">$s$</span> itself can be different in a subsamples so it is not guaranteed the standard errors will be lower but they usually are.</p>
</li>
<li><p>The <span class="math-container">$t^*$</span> is higher in a smaller sampler since smaller samples have less degrees of freedom. For example, in a sample with 100 observations it is approximately 1.984 whereas with 1000 observations it would be approximately 1.962. Now you have quite a large number of observations and difference in <span class="math-container">$t^*$</span> in a sample with 2000 observations vs 4000 is very small, but there are certainly cases where that might be sufficient to make marginally significant coefficient insignificant.</p>
</li>
<li><p>Of course, combination of all of the above can be responsible.</p>
</li>
</ol>
<p>To determine exact cause you can have a look at the <span class="math-container">$\beta$</span>, <span class="math-container">$se$</span> and <span class="math-container">$t^*$</span> that are estimated, or in <span class="math-container">$t^*$</span> case used, in the regression you run. That will tell you exactly what combination of the above is responsible for coefficients becoming insignificant.</p>

CC BY-SA 4.0
2022-03-27T12:50:26.317Z
2022-03-27T12:50:26.317Z
1muflon1
<p>I built an ordered logit using a sample of more than 4000 observations and the variable of the interest was significant. But then I split the sample into two subsamples: women and men and the variable became insignificant in both. How can I explain it?</p>

CC BY-SA 4.0
2022-03-27T13:38:16.213Z
Is it possible that the two samples are not IID?
EB3112
CC BY-SA 4.0
2022-03-27T23:54:56.207Z
A very common outcome is that a significant variable is actually driven by another factor, and by filtering out the real driving factor it really has become insignificant.
Steven Armstrong
<econometrics><logit>
Why significant variables become insignificant in subsamples?
Daria
# Why significant variables become insignificant in subsamples?

(Asked by: username_0 on 2022-03-27)

I built an ordered logit using a sample of more than 4000 observations and the variable of the interest was significant. But then I split the sample into two subsamples: women and men and the variable became insignificant in both. How can I explain it?

**Tags:** econometrics, logit
> **Comments:**
> **username_1**: Is it possible that the two samples are not IID?  
> **username_2**: A very common outcome is that a significant variable is actually driven by another factor, and by filtering out the real driving factor it really has become insignificant.  

---

## 1 answers

### Answer by username_3 on 2022-03-27. Score: 3

This is because of several reasons. The standard logit test uses t-test to test whether coefficients are individually significant.

The t-test is given by:

$$t=\frac{\beta\_i}{se\_{\beta\_i}}$$

where $\beta\_i$ is the coefficient you estimated and $se\_{\beta\_i}$ is standard error of the estimated coefficient. In turn standard error is given by $se\_{\beta\_i}=\frac{s}{\sqrt{n}}$ where $s$ is the estimate of standard deviation and $n$ is the number of observations.

Lastly, to determine whether the variable is significant you compare whether the $t$ statistics is greater than critical value of the t-statistics ($t^\*$).

The $t^\*$ depends on several factors. First, it depends on the significance level you want to use (a commonly used level of significance will be 5% or $\alpha =0.05$. Second it depends whether you are using two-sided or one sided test, although in practice people would use almost always two sided test when running regression. Third, it depends on the degrees of freedom of the test which depend on number of observations ($df\_t =n-1$). The magnitude of $t^\*$ has inverse relationship with the number of observations $n$, that is higher $n$ lowers $t^\*$ holding $\alpha$ and type of test constant.

Now given all the information above there are the following possible reasons why you found that the observations are no longer significant in the subsamples:

1. $\beta\_i$ in a subsamples is lower than in the joint sample. Perhaps there are some outliers that have bigger impact on the coefficient size when you split the sample into sub-samples.
2. Standard error in the subsamples is higher. Since subsamples have lower number of observations and since standard error is given as $se\_{\beta\_i}=\frac{s}{\sqrt{n}}$ having lower number of observation, ceteris paribus, increases standard errors of your beta coefficients. Now the standard deviation $s$ itself can be different in a subsamples so it is not guaranteed the standard errors will be lower but they usually are.
3. The $t^\*$ is higher in a smaller sampler since smaller samples have less degrees of freedom. For example, in a sample with 100 observations it is approximately 1.984 whereas with 1000 observations it would be approximately 1.962. Now you have quite a large number of observations and difference in $t^\*$ in a sample with 2000 observations vs 4000 is very small, but there are certainly cases where that might be sufficient to make marginally significant coefficient insignificant.
4. Of course, combination of all of the above can be responsible.

To determine exact cause you can have a look at the $\beta$, $se$ and $t^\*$ that are estimated, or in $t^\*$ case used, in the regression you run. That will tell you exactly what combination of the above is responsible for coefficients becoming insignificant.


---

