<p>They matter whenever you are interested in learning from the data.</p>
<p>If you do not have prior knowledge about the effect you are studying, the point estimate alongside its confidence interval and statistical significance tell a lot of useful things. The point estimate tells you the best guess of the effect size and its direction, as seen in the data through the lense of your model. Statistical significance puts that into perspective (how likely you are to obtain such an extreme, or even more extreme, estimated effect size if <span class="math-container">$H_0$</span> of zero effect is in fact true in population) but does not invalidate the fact that this is the best guess. Replacing it with a zero (which in many cases in economics, unlike genetics, is known with certainty to be incorrect) is generally not justified.</p>
<p>If you do have prior knowledge, you could combine it with the (new) data using e.g. Bayesian estimation. The resulting estimate would again be your best guess, and it would not make much sense to replace it with anything else (such as a zero).</p>
<p>This presumes you are following a sound methodology rather than trying to cheat your way to a desirable result one way or another.</p>

CC BY-SA 4.0
2021-10-23T13:24:11.430Z
2021-10-23T13:29:22.063Z
2021-10-23T13:29:22.063Z
Richard Hardy
<p>When insignificant (not significantly different from <span class="math-container">$0$</span>), we should never interprete the sign and magnitude of the estimated coefficient. Because, with high probability the sign of the coefficient could be the opposite to the estimated one, and the magnitude of the coefficient either much smaller or much higher. The estimations are specific values taken by random variables which could be fully different for other <span class="math-container">$X$</span> values.<br />
More precisely, the only interpretation which makes sense in a linear model, is that the impact of the variable <span class="math-container">$X_j$</span> is not significantly different from zero, the anti-corruption law is not an effective tool to fight against corruption, because in the treated sample there is comparable corruption as in the untreated sample for instance (ceteris paribus, and conditionally to the chosen model specification).</p>

CC BY-SA 4.0
2021-10-13T10:03:16.307Z
"high probability the sign of the coefficient could be the opposite to the estimated one" can I ask the reason why? I am wondering why it is high probability opposite to the estimated one?
Nguyen Lis
CC BY-SA 4.0
2021-10-13T12:59:34.220Z
Assume that $\widehat{\beta_j}=1.23$ and that the p-value is $Pr[|T|>|t||\beta_j=0]=0.37$. In this case $\beta_j=0$ cannot be rejected at the 5% threshold. This means that if the null hypothesis is true there is 37/2% chance to draw a value such that $\widehat{\beta_j}=1.23$ or higher. And as the distribution is symmetric wrt 0, there is also 37/2 % chance to draw a value lower than $\widehat{\beta_j}=-1.23$,
Bertrand
CC BY-SA 4.0
2021-10-23T13:05:31.493Z
-1. Your first paragraph suggests to ignore a data-based point estimate in favor of a hypothesized value under $H_0$. I do not think this is generally a smart choice. E.g. one could systematically lose money if one grounded one's bets on it, unless the $H_0$ is selected in some superior way, e.g. based on additional data. Your second paragraph seems to fall in the trap of equating absence of evidence with evidence of absence.
Richard Hardy
CC BY-SA 4.0
2021-10-23T13:29:36.000Z
@Richard Hardy: under $H_0: \beta_j=0$, and the data-point based evidence does not contradict this hypothesis (at the $5\%$ theshold).
Bertrand
CC BY-SA 4.0
2021-10-23T14:19:15.313Z
What I mean is that *only interpretation <...> is that <...> the anti-corruption law is not an effective tool* is a way of arguing for absence of an effect, while the statistical result merely indicates absence of (or more precisely, lack of sufficient) evidence to the contrary. We do not prove the effect is zero; we merely fail (to collect enough data) to prove otherwise.
Richard Hardy
CC BY-SA 4.0
2021-10-13T06:12:03.257Z
2021-10-13T06:12:03.257Z
Bertrand
<p><strong>answer to new edit</strong></p>
<p>If the coefficient is not significant then you cannot reject the hypothesis that true coefficient is zero. In that case, magnitude or sign of the coefficient is not very relevant.</p>
<p>You could still care about it a bit because if you find large coefficient with sign  you would expect to find, it might be that it is insignificant only because there is a lot of noise in your data (remember test statistics depends not just on coefficient size but also standard errors <span class="math-container">$\hat{\beta}/se(\hat{\beta})$</span>. So finding large coefficient with expected sign might motivate you to perhaps find larger dataset where there is less noise, but other than this it would not be very relevant.</p>
<p><strong>Answer to original question:</strong></p>
<p>Of course you should care about sign and magnitude of the coefficient. This is especially true when it comes to policy analysis.</p>
<p>I am not familiar with the anti-corruption laws research, so let me give you another example. Consider effect of minimum wage laws on employment.</p>
<p>Sign of the treatment dummy clearly matters as it would be a whole world of difference if research would show that minimum wages have positive impact on employment, to case where they have negative impact on employment (which implies difficult trade-off between higher wages for low income people vs their employment).</p>
<p>Second, magnitude of the coefficient matters as well as again if the relationship between minimum wage and employment is such that 1% increase in minimum wage leads to 10% decrease in employment that implies the trade-off would be very severe. However, if 1% increase in minimum wages increases unemployment just by 0.0001% then no matter whether the coefficient is statistically significant or not the effect is so small it could be safely ignored and you do not even need to worry about it.</p>

CC BY-SA 4.0
2021-10-12T21:46:09.193Z
I mean, when should we care about the magnitude and sign of the coefficient **if it is insignificant**. Sorry i did not explain my concern well.
Nguyen Lis
CC BY-SA 4.0
2021-10-23T13:26:06.550Z
I disagree with your point in general and have posted an alternative answer.
Richard Hardy
CC BY-SA 4.0
2021-10-23T13:30:53.353Z
@RichardHardy thanks for letting me know, I think your answer makes some good points, but i disagree that we should not assume that the true coefficient is zero if it’s not significant. Sure the estimations are our best guess of what the coefficient is, but if the standard errors are too high there is high chance that the coefficient is just random noise anyway I upvoted your answer because I think it makes some good points
1muflon1
CC BY-SA 4.0
2021-10-23T13:35:17.217Z
I suppose the choice between the point estimate and zero has to do with prior knowledge. Without prior knowledge, what is the rationale to pick a random number (zero) over an estimate based on the data? (Zero is just as good a number as any if we lack prior knowledge.)
Richard Hardy
CC BY-SA 4.0
2021-10-23T13:37:07.837Z
@RichardHardy well because zero is considered to be your prior null hypothesis. In public policy you conventionally start by assuming that there is no effect of policy on variable of interest not that there actually is some effect and you are just quantifying it
1muflon1
CC BY-SA 4.0
2021-10-23T13:41:26.737Z
I was trying to provide a statistical point of view. Referring to conventional practice takes us to, hmm, political or institutional point of view. That is OK in itself, but it is probably tangential to my point, unless the conventional practice is based on prior information.
Richard Hardy
CC BY-SA 4.0
2021-10-23T13:46:40.893Z
@RichardHardy well this maybe transcends the statistics itself but generally when testing scientific theories even outside public policy, common practice is to assume that relationship does not exist unless there is an sufficient evidence that there is one
1muflon1
CC BY-SA 4.0
2021-10-23T14:15:22.037Z
@1muflon1, I think many a reasonable economist would not question the hypothesis that everything depends on everything (nonzero effects throughout) even if most of the effects are negligible. *testing scientific theories* might be a crucial bit in your comment. For that goal one is often OK with taking an unreasonable $H_0$ of zero effect (a straw man) and trying to beat it. In other words, I think there is a big difference between trying to figure out the truth and testing a scientific theory.
Richard Hardy
CC BY-SA 4.0
2021-10-12T20:29:44.810Z
2021-10-13T10:22:26.943Z
2021-10-13T10:22:26.943Z
1muflon1
<p>I am wondering when the sign and magnitude of coefficients of a variable of interest matter <strong>if it is statistically insignificant</strong>. Normally, I am concerning a coefficient of variable if it is significant at 5% level or lower. I am wondering when we should care about the magnitude and sign of coefficient if it is insignificant? Saying for example, my variable of interest saying about the impact of anticorruption laws on asset growth in Difference - in -Difference setting.</p>
<p>Please guide me to clarify my question more if it is not yet clear.
Sorry @1muflon1 that I did not clarify my question that confused you</p>

<econometrics><nonexpert>
When do the sign and magnitude of coefficient of variable of interest matter if it is insignificant?
Nguyen Lis
# When do the sign and magnitude of coefficient of variable of interest matter if it is insignificant?

(Asked by: username_0 on 2021-10-12)

I am wondering when the sign and magnitude of coefficients of a variable of interest matter **if it is statistically insignificant**. Normally, I am concerning a coefficient of variable if it is significant at 5% level or lower. I am wondering when we should care about the magnitude and sign of coefficient if it is insignificant? Saying for example, my variable of interest saying about the impact of anticorruption laws on asset growth in Difference - in -Difference setting.

Please guide me to clarify my question more if it is not yet clear.
Sorry @1muflon1 that I did not clarify my question that confused you

**Tags:** econometrics, nonexpert

---

## 3 answers

### Answer by username_3 on 2021-10-12. Score: 2 | Accepted answer

**answer to new edit**

If the coefficient is not significant then you cannot reject the hypothesis that true coefficient is zero. In that case, magnitude or sign of the coefficient is not very relevant.

You could still care about it a bit because if you find large coefficient with sign you would expect to find, it might be that it is insignificant only because there is a lot of noise in your data (remember test statistics depends not just on coefficient size but also standard errors $\hat{\beta}/se(\hat{\beta})$. So finding large coefficient with expected sign might motivate you to perhaps find larger dataset where there is less noise, but other than this it would not be very relevant.

**Answer to original question:**

Of course you should care about sign and magnitude of the coefficient. This is especially true when it comes to policy analysis.

I am not familiar with the anti-corruption laws research, so let me give you another example. Consider effect of minimum wage laws on employment.

Sign of the treatment dummy clearly matters as it would be a whole world of difference if research would show that minimum wages have positive impact on employment, to case where they have negative impact on employment (which implies difficult trade-off between higher wages for low income people vs their employment).

Second, magnitude of the coefficient matters as well as again if the relationship between minimum wage and employment is such that 1% increase in minimum wage leads to 10% decrease in employment that implies the trade-off would be very severe. However, if 1% increase in minimum wages increases unemployment just by 0.0001% then no matter whether the coefficient is statistically significant or not the effect is so small it could be safely ignored and you do not even need to worry about it.

> **Comments:**
> **username_0**: I mean, when should we care about the magnitude and sign of the coefficient **if it is insignificant**. Sorry i did not explain my concern well.  
> **username_1**: I disagree with your point in general and have posted an alternative answer.  
> **username_3**: @RichardHardy thanks for letting me know, I think your answer makes some good points, but i disagree that we should not assume that the true coefficient is zero if it’s not significant. Sure the estimations are our best guess of what the coefficient is, but if the standard errors are too high there is high chance that the coefficient is just random noise anyway I upvoted your answer because I think it makes some good points  
> **username_1**: I suppose the choice between the point estimate and zero has to do with prior knowledge. Without prior knowledge, what is the rationale to pick a random number (zero) over an estimate based on the data? (Zero is just as good a number as any if we lack prior knowledge.)  
> **username_3**: @RichardHardy well because zero is considered to be your prior null hypothesis. In public policy you conventionally start by assuming that there is no effect of policy on variable of interest not that there actually is some effect and you are just quantifying it  
> **username_1**: I was trying to provide a statistical point of view. Referring to conventional practice takes us to, hmm, political or institutional point of view. That is OK in itself, but it is probably tangential to my point, unless the conventional practice is based on prior information.  
> **username_3**: @RichardHardy well this maybe transcends the statistics itself but generally when testing scientific theories even outside public policy, common practice is to assume that relationship does not exist unless there is an sufficient evidence that there is one  
> **username_1**: @1muflon1, I think many a reasonable economist would not question the hypothesis that everything depends on everything (nonzero effects throughout) even if most of the effects are negligible. *testing scientific theories* might be a crucial bit in your comment. For that goal one is often OK with taking an unreasonable $H_0$ of zero effect (a straw man) and trying to beat it. In other words, I think there is a big difference between trying to figure out the truth and testing a scientific theory.  

---

### Answer by username_1 on 2021-10-23. Score: 3

They matter whenever you are interested in learning from the data.

If you do not have prior knowledge about the effect you are studying, the point estimate alongside its confidence interval and statistical significance tell a lot of useful things. The point estimate tells you the best guess of the effect size and its direction, as seen in the data through the lense of your model. Statistical significance puts that into perspective (how likely you are to obtain such an extreme, or even more extreme, estimated effect size if $H\_0$ of zero effect is in fact true in population) but does not invalidate the fact that this is the best guess. Replacing it with a zero (which in many cases in economics, unlike genetics, is known with certainty to be incorrect) is generally not justified.

If you do have prior knowledge, you could combine it with the (new) data using e.g. Bayesian estimation. The resulting estimate would again be your best guess, and it would not make much sense to replace it with anything else (such as a zero).

This presumes you are following a sound methodology rather than trying to cheat your way to a desirable result one way or another.


---

### Answer by username_2 on 2021-10-13. Score: 0

When insignificant (not significantly different from $0$), we should never interprete the sign and magnitude of the estimated coefficient. Because, with high probability the sign of the coefficient could be the opposite to the estimated one, and the magnitude of the coefficient either much smaller or much higher. The estimations are specific values taken by random variables which could be fully different for other $X$ values.  
More precisely, the only interpretation which makes sense in a linear model, is that the impact of the variable $X\_j$ is not significantly different from zero, the anti-corruption law is not an effective tool to fight against corruption, because in the treated sample there is comparable corruption as in the untreated sample for instance (ceteris paribus, and conditionally to the chosen model specification).

> **Comments:**
> **username_0**: "high probability the sign of the coefficient could be the opposite to the estimated one" can I ask the reason why? I am wondering why it is high probability opposite to the estimated one?  
> **username_2**: Assume that $\widehat{\beta_j}=1.23$ and that the p-value is $Pr[|T|>|t||\beta_j=0]=0.37$. In this case $\beta_j=0$ cannot be rejected at the 5% threshold. This means that if the null hypothesis is true there is 37/2% chance to draw a value such that $\widehat{\beta_j}=1.23$ or higher. And as the distribution is symmetric wrt 0, there is also 37/2 % chance to draw a value lower than $\widehat{\beta_j}=-1.23$,  
> **username_1**: -1. Your first paragraph suggests to ignore a data-based point estimate in favor of a hypothesized value under $H_0$. I do not think this is generally a smart choice. E.g. one could systematically lose money if one grounded one's bets on it, unless the $H_0$ is selected in some superior way, e.g. based on additional data. Your second paragraph seems to fall in the trap of equating absence of evidence with evidence of absence.  
> **username_2**: @Richard Hardy: under $H_0: \beta_j=0$, and the data-point based evidence does not contradict this hypothesis (at the $5\%$ theshold).  
> **username_1**: What I mean is that *only interpretation <...> is that <...> the anti-corruption law is not an effective tool* is a way of arguing for absence of an effect, while the statistical result merely indicates absence of (or more precisely, lack of sufficient) evidence to the contrary. We do not prove the effect is zero; we merely fail (to collect enough data) to prove otherwise.  

---

