<p>Relative efficiency between two unbiased estimators <span class="math-container">$\widehat{\theta }_{A}$</span>
and <span class="math-container">$\widehat{\theta }_{B}$</span> of an unknown parameter vector <span class="math-container">$\theta _{0}\in 
\mathbb{R}^{K}$</span> is usually defined as follow (see for instance Ruud, 2001).
The estimator <span class="math-container">$\widehat{\theta }_{A}$</span> is said to be efficient relative to <span class="math-container">$
\widehat{\theta }_{B}$</span> if we have:
<span class="math-container">$$ \mathrm{V}\left[ \widehat{\theta }_{A}\right] \equiv \Omega _{A}&lt;&lt;\Omega
_{B}\equiv \mathrm{V}\left[ \widehat{\theta }_{B}\right] .$$</span></p>

<p>If <span class="math-container">$\Omega _{B}-\Omega _{A}$</span> is positive definite, then the diagonal terms
of <span class="math-container">$\Omega _{B}$</span> and <span class="math-container">$\Omega _{A}$</span> are necessarily such that <span class="math-container">$\sigma
_{Bii}^{2}&gt;\sigma _{Aii}^{2}.$</span> Indeed if <span class="math-container">$v^{T}\left( \Omega _{B}-\Omega
_{A}\right) v&gt;0$</span> for any <span class="math-container">$v\neq 0$</span> then for <span class="math-container">$v=e_{i}$</span> we catch the diagonal
terms out of <span class="math-container">$\Omega _{B}-\Omega _{A}$</span> and find that <span class="math-container">$\sigma
_{Bii}^{2}&gt;\sigma _{Aii}^{2}.$</span> The converse, however, it is not true: the
mere conditions <span class="math-container">$\sigma _{Bii}^{2}&gt;\sigma _{Aii}^{2}$</span> do not garantee that <span class="math-container">$
\Omega _{B}-\Omega _{A}$</span> be positive definite. Covariances matter.  </p>

<ul>
<li><p>The answer to your second question is: no it never happens that <span class="math-container">$\sigma _{Bii}^{2}&lt;\sigma _{Aii}^{2}$</span> when <span class="math-container">$\Omega _{B}&gt;&gt;\Omega _{A}$</span>.  </p></li>
<li><p>The answer to your first question is: it is important to consider all covariance terms. If we want that the confidence ellipse (at any threshold) of <span class="math-container">$\widehat{\theta }_{A}$</span> be nested within the confidence ellipse of <span class="math-container">$\widehat{\theta }_{B}$</span> then we need <span class="math-container">$\Omega _{A}&lt;&lt;\Omega _{B}$</span> and not just <span class="math-container">$\sigma _{Aii}^{2}&lt;\sigma _{Bii}^{2}$</span>.   </p></li>
</ul>

<p>See Ruud, (2001, chapter 9) for a proof and detailed explanations. An example is provided here, illustrating that the confidence ellipses are nested when <span class="math-container">$\Omega _{B}-\Omega _{A}$</span> positive definite, and not nested if <span class="math-container">$\Omega _{B}-\Omega _{A}$</span> is not positive definite.  </p>

<p>Example: 
<span class="math-container">\begin{eqnarray*}
\widehat{\theta }_{A}\left( a\right) &amp;\sim &amp;\mathcal{N}\left( 0,\Omega
_{A}\left( a\right) \right) ,\qquad \widehat{\theta }_{B}\sim \mathcal{N}%
\left( 0,\Omega _{B}\right) \\
\Omega _{A}\left( a\right) &amp;=&amp;\left( 
\begin{array}{cc}
4 &amp; a \\ 
a &amp; 9%
\end{array}%
\right) ,\qquad \Omega _{B}=\left( 
\begin{array}{cc}
5 &amp; 0 \\ 
0 &amp; 10%
\end{array}%
\right).
\end{eqnarray*}</span></p>

<p>For <span class="math-container">$a=0$</span> the matrix <span class="math-container">$\Omega _{B}-\Omega _{A}\left( 0\right) $</span> is positive
definite and the <span class="math-container">$95\%$</span> threshold confidence ellipses are nested. The Figure
below (left panel) represents the iso-curves centered on <span class="math-container">$\theta _{0}=0$</span> and
whose equations are given by <span class="math-container">$v^{T}\left( \Omega _{A}\left( 0\right) \right)
^{-1}v=5.99$</span> and <span class="math-container">$x^{T}\Omega _{B}^{-1}x=5.99$</span> and illustrates that for <span class="math-container">$a=0$</span>
the later is nested within the former. This is no longer true for <span class="math-container">$a=-5$</span>
(right panel) in which case <span class="math-container">$\Omega _{B}-\Omega _{A}\left( -5\right) $</span> is no
longer positive definite. In this case the probability that <span class="math-container">$\widehat{\theta 
}_{A}$</span> is further away than <span class="math-container">$\widehat{\theta }_{B}$</span> from the true value <span class="math-container">$%
\theta _{0}=0$</span> is positive, and <span class="math-container">$\widehat{\theta }_{A}$</span> is no longer
efficient (case with <span class="math-container">$a=-5$</span>) relatively to <span class="math-container">$\widehat{\theta }_{B}$</span>. Note
that the variances always satisfy <span class="math-container">$\sigma _{Bii}^{2}&gt;\sigma _{Aii}^{2}$</span> but
this is not sufficient for the ellipses to be nested, the covariances must
also satisfy <span class="math-container">$\left( \sigma _{B11}^{2}-\sigma _{A11}^{2}\right) \left(
\sigma _{B22}^{2}-\sigma _{A22}^{2}\right) -\left( \sigma _{B12}-\sigma
_{A12}\right) ^{2}&gt;0$</span>, which is not the case in this example for <span class="math-container">$a=-5$</span>.
<a href="https://i.sstatic.net/eGGuO.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/eGGuO.png" alt="enter image description here"></a></p>

CC BY-SA 4.0
2019-01-22T20:59:04.080Z
Betrand: That's interesting but I don't think OP is interested in efficiency. I think he's interested in what it means when one says that one covariance matrix $B$ is greater than another covariance matrix $A$, in the sense of $(B-A) > 0$. That's my take but I could be wrong.  Thanks for insights.
mark leeds
CC BY-SA 4.0
2019-01-22T21:15:37.790Z
Mark, I tried in the answer to clarify the question and had to introduce two characters $\widehat{\theta }_{A}$ and $\Omega_A$ instead of using only $A$. Well, I will delete the answer when it is downvoted.
Bertrand
CC BY-SA 4.0
2019-01-23T00:28:37.540Z
Bertrand, I have found your answer very helpful. Could you please clarify (if relation to my first question) what do you mean by the "confidence ellipse" (at any threshold) of an estimator be nested within the confidence ellipse of another? Thank you!
econ86
CC BY-SA 4.0
2019-01-23T15:55:01.890Z
Bertrand: I think your answer was great. I just wasn't sure if that was the question the OP was asking. Sounds like it was so my mistake and thanks for great explanation.
mark leeds
CC BY-SA 4.0
2019-01-23T16:13:34.697Z
econ86: I have added an example in my last edit, hope that it helps.
Bertrand
CC BY-SA 4.0
2019-02-15T09:09:25.677Z
This issue is also discussed here: https://stats.stackexchange.com/questions/81285/appropriate-measure-to-find-smallest-covariance-matrix
Bertrand
CC BY-SA 4.0
2019-01-22T12:01:10.800Z
2019-01-23T16:16:04.643Z
2019-01-23T16:16:04.643Z
Bertrand
<p>When comparing asymptotic variances, you usually conclude that a covariance matrix A is more efficient whenever the difference B - A ≥ 0 (where ≥ 0 means that the difference is a positive semidefinite matrix).</p>

<p>However, this comparison involves all elements in the matrices - both the diagonal elements (asymptotic variances of each beta estimator) and the off-diagonal ones (covariances between estimators) -, while we are usually interested in estimators for a particular beta coefficient having small standard errors.</p>

<p>Could anyone explain why is this method preferred to a comparison that entails only the diagonal elements? Could it be the case that some diagonal element is smaller in B while B - A ≥ 0?</p>

<p>Thank you!</p>

CC BY-SA 4.0
2019-01-21T15:59:30.093Z
Hi: the  covariance of a random vector, $X$, is its covariance matrix along with the diagonal elements. But the scalar variance of a random vector, $X$, is $X^\prime A X$ which is a scalar. So, this is why they compare them that way. If one matrix is "greater", then the variance of a given random vector, $X$ is always less if it has $A$ as its covariance matrix rather than $B$ as its covariance matrix.
mark leeds
CC BY-SA 4.0
2019-01-21T16:41:52.467Z
An estimator can be more efficient than another, not clear what it means to write "a covariance matrix A is more efficient"
Bertrand
CC BY-SA 4.0
2019-01-22T02:39:28.027Z
Note that efficiency denotes whether the variance of some estimator reaches it's CR lower bound so I don't think the term "efficient" applies here, atleast the way you explained the notion of $B-A$ being greater than or equal to zero.
mark leeds
CC BY-SA 4.0
2019-01-23T00:26:42.467Z
Thank you! Just to clarify, 1) Bertrand: indeed "a covariance matrix A is more efficient" makes little sense. I meant an estimator with covariance matrix A being more efficient than another estimator. 2) Mark, indeed I should have used the term "relative efficiency" .
econ86
<econometrics>
Smaller variance of estimators in matrix sense
econ86
# Smaller variance of estimators in matrix sense

(Asked by: username_0 on 2019-01-21)

When comparing asymptotic variances, you usually conclude that a covariance matrix A is more efficient whenever the difference B - A ≥ 0 (where ≥ 0 means that the difference is a positive semidefinite matrix).

However, this comparison involves all elements in the matrices - both the diagonal elements (asymptotic variances of each beta estimator) and the off-diagonal ones (covariances between estimators) -, while we are usually interested in estimators for a particular beta coefficient having small standard errors.

Could anyone explain why is this method preferred to a comparison that entails only the diagonal elements? Could it be the case that some diagonal element is smaller in B while B - A ≥ 0?

Thank you!

**Tags:** econometrics
> **Comments:**
> **username_1**: Hi: the  covariance of a random vector, $X$, is its covariance matrix along with the diagonal elements. But the scalar variance of a random vector, $X$, is $X^\prime A X$ which is a scalar. So, this is why they compare them that way. If one matrix is "greater", then the variance of a given random vector, $X$ is always less if it has $A$ as its covariance matrix rather than $B$ as its covariance matrix.  
> **username_2**: An estimator can be more efficient than another, not clear what it means to write "a covariance matrix A is more efficient"  
> **username_1**: Note that efficiency denotes whether the variance of some estimator reaches it's CR lower bound so I don't think the term "efficient" applies here, atleast the way you explained the notion of $B-A$ being greater than or equal to zero.  
> **username_0**: Thank you! Just to clarify, 1) Bertrand: indeed "a covariance matrix A is more efficient" makes little sense. I meant an estimator with covariance matrix A being more efficient than another estimator. 2) Mark, indeed I should have used the term "relative efficiency" .  

---

## 1 answers

### Answer by username_2 on 2019-01-22. Score: 1 | Accepted answer

Relative efficiency between two unbiased estimators $\widehat{\theta }\_{A}$
and $\widehat{\theta }\_{B}$ of an unknown parameter vector $\theta \_{0}\in
\mathbb{R}^{K}$ is usually defined as follow (see for instance Ruud, 2001).
The estimator $\widehat{\theta }\_{A}$ is said to be efficient relative to $
\widehat{\theta }\_{B}$ if we have:
$$ \mathrm{V}\left[ \widehat{\theta }\_{A}\right] \equiv \Omega \_{A}<<\Omega
\_{B}\equiv \mathrm{V}\left[ \widehat{\theta }\_{B}\right] .$$

If $\Omega \_{B}-\Omega \_{A}$ is positive definite, then the diagonal terms
of $\Omega \_{B}$ and $\Omega \_{A}$ are necessarily such that $\sigma
\_{Bii}^{2}>\sigma \_{Aii}^{2}.$ Indeed if $v^{T}\left( \Omega \_{B}-\Omega
\_{A}\right) v>0$ for any $v\neq 0$ then for $v=e\_{i}$ we catch the diagonal
terms out of $\Omega \_{B}-\Omega \_{A}$ and find that $\sigma
\_{Bii}^{2}>\sigma \_{Aii}^{2}.$ The converse, however, it is not true: the
mere conditions $\sigma \_{Bii}^{2}>\sigma \_{Aii}^{2}$ do not garantee that $
\Omega \_{B}-\Omega \_{A}$ be positive definite. Covariances matter.

* The answer to your second question is: no it never happens that $\sigma \_{Bii}^{2}<\sigma \_{Aii}^{2}$ when $\Omega \_{B}>>\Omega \_{A}$.
* The answer to your first question is: it is important to consider all covariance terms. If we want that the confidence ellipse (at any threshold) of $\widehat{\theta }\_{A}$ be nested within the confidence ellipse of $\widehat{\theta }\_{B}$ then we need $\Omega \_{A}<<\Omega \_{B}$ and not just $\sigma \_{Aii}^{2}<\sigma \_{Bii}^{2}$.

See Ruud, (2001, chapter 9) for a proof and detailed explanations. An example is provided here, illustrating that the confidence ellipses are nested when $\Omega \_{B}-\Omega \_{A}$ positive definite, and not nested if $\Omega \_{B}-\Omega \_{A}$ is not positive definite.

Example:
\begin{eqnarray\*}
\widehat{\theta }\_{A}\left( a\right) &\sim &\mathcal{N}\left( 0,\Omega
\_{A}\left( a\right) \right) ,\qquad \widehat{\theta }\_{B}\sim \mathcal{N}%
\left( 0,\Omega \_{B}\right) \\
\Omega \_{A}\left( a\right) &=&\left(
\begin{array}{cc}
4 & a \\
a & 9%
\end{array}%
\right) ,\qquad \Omega \_{B}=\left(
\begin{array}{cc}
5 & 0 \\
0 & 10%
\end{array}%
\right).
\end{eqnarray\*}

For $a=0$ the matrix $\Omega \_{B}-\Omega \_{A}\left( 0\right) $ is positive
definite and the $95\%$ threshold confidence ellipses are nested. The Figure
below (left panel) represents the iso-curves centered on $\theta \_{0}=0$ and
whose equations are given by $v^{T}\left( \Omega \_{A}\left( 0\right) \right)
^{-1}v=5.99$ and $x^{T}\Omega \_{B}^{-1}x=5.99$ and illustrates that for $a=0$
the later is nested within the former. This is no longer true for $a=-5$
(right panel) in which case $\Omega \_{B}-\Omega \_{A}\left( -5\right) $ is no
longer positive definite. In this case the probability that $\widehat{\theta
}\_{A}$ is further away than $\widehat{\theta }\_{B}$ from the true value $%
\theta \_{0}=0$ is positive, and $\widehat{\theta }\_{A}$ is no longer
efficient (case with $a=-5$) relatively to $\widehat{\theta }\_{B}$. Note
that the variances always satisfy $\sigma \_{Bii}^{2}>\sigma \_{Aii}^{2}$ but
this is not sufficient for the ellipses to be nested, the covariances must
also satisfy $\left( \sigma \_{B11}^{2}-\sigma \_{A11}^{2}\right) \left(
\sigma \_{B22}^{2}-\sigma \_{A22}^{2}\right) -\left( \sigma \_{B12}-\sigma
\_{A12}\right) ^{2}>0$, which is not the case in this example for $a=-5$.
[![enter image description here](https://i.sstatic.net/eGGuO.png)](https://i.sstatic.net/eGGuO.png)

> **Comments:**
> **username_1**: Betrand: That's interesting but I don't think OP is interested in efficiency. I think he's interested in what it means when one says that one covariance matrix $B$ is greater than another covariance matrix $A$, in the sense of $(B-A) > 0$. That's my take but I could be wrong.  Thanks for insights.  
> **username_2**: Mark, I tried in the answer to clarify the question and had to introduce two characters $\widehat{\theta }_{A}$ and $\Omega_A$ instead of using only $A$. Well, I will delete the answer when it is downvoted.  
> **username_0**: Bertrand, I have found your answer very helpful. Could you please clarify (if relation to my first question) what do you mean by the "confidence ellipse" (at any threshold) of an estimator be nested within the confidence ellipse of another? Thank you!  
> **username_1**: Bertrand: I think your answer was great. I just wasn't sure if that was the question the OP was asking. Sounds like it was so my mistake and thanks for great explanation.  
> **username_2**: econ86: I have added an example in my last edit, hope that it helps.  
> **username_2**: This issue is also discussed here: https://stats.stackexchange.com/questions/81285/appropriate-measure-to-find-smallest-covariance-matrix  

---

