<p>Assume that for each observation <span class="math-container">$i = 1,\ldots, N$</span>, we have <span class="math-container">$M$</span> equations:
<span class="math-container">$$
y_{i,j} = x_{i,j}\beta_j + \varepsilon_{i,j}
$$</span>
Where <span class="math-container">$i = 1,\ldots, N$</span> enumerates individuals and and <span class="math-container">$j = 1,\ldots, M$</span> enumerates the equations. here <span class="math-container">$x$</span> is of size <span class="math-container">$1 \times k_j$</span> and <span class="math-container">$\beta_j$</span> is of size <span class="math-container">$k_j \times 1$</span> and <span class="math-container">$k_j$</span> is the number of covariates for regression <span class="math-container">$j$</span>. Stacking over all <span class="math-container">$i = 1,\ldots N$</span>, we get <span class="math-container">$M$</span> equations:
<span class="math-container">$$
y_j = X_j \beta_j + \varepsilon_j
$$</span>
where now <span class="math-container">$X_j$</span> is of size <span class="math-container">$N \times k_j$</span>. For simplicity, assume that <span class="math-container">$X_j$</span> are non-stochastic. Next, assume that for all <span class="math-container">$i = 1,\ldots, N$</span> and <span class="math-container">$j = 1,\ldots, M$</span>:
<span class="math-container">$$
\begin{align*}
&amp;\mathbb{E}(\varepsilon_{i,j}) = 0,\\
&amp;\mathbb{E}(\varepsilon_{i,j}^2) = \sigma_{jj}
\end{align*}
$$</span>
For the covariance between equations, let for all <span class="math-container">$i = 1,\ldots, N$</span> and <span class="math-container">$j,\ell = 1,\ldots, M$</span>:
<span class="math-container">$$
\mathbb{E}(\varepsilon_{i,j} \varepsilon_{i,\ell}) = \sigma_{j,\ell}
$$</span>
while for all <span class="math-container">$j,\ell = 1,\ldots, M$</span> and <span class="math-container">$i,i' = 1,\ldots, N$</span> with <span class="math-container">$i \ne i'$</span>:
<span class="math-container">$$
\mathbb{E}(\varepsilon_{i,j}, \varepsilon_{i',k}) = 0
$$</span>
This means that errors for the same individual might be correlated across equations, while errors for different individuals are uncorrelated.</p>
<p>This can be expressed more compactly as:
<span class="math-container">$$
cov(\varepsilon_j, \varepsilon_{\ell}) = \sigma_{j,\ell}I_N
$$</span>
Now, let us stack the various equations, one on top of the other:
<span class="math-container">$$
y = Z\beta + \varepsilon,
$$</span>
where:
<span class="math-container">$$
y = \begin{bmatrix} y_1\\y_2\\ \vdots\\y_M\end{bmatrix},
\varepsilon = \begin{bmatrix} \varepsilon_1 \\ \vdots \\ \varepsilon_M \end{bmatrix},\\
Z = \begin{bmatrix} X_1 &amp; 0 &amp; \ldots &amp; 0\\
0 &amp; X_2 &amp; \ldots &amp; 0,\\
\vdots  &amp; \vdots &amp; \ddots &amp; \vdots\\
0 &amp; 0 &amp; \ldots &amp; X_M
\end{bmatrix}, \beta = \begin{bmatrix} \beta_1\\ \vdots \\ \beta_M\end{bmatrix}
$$</span>
The variance-covariance matrix of <span class="math-container">$\varepsilon$</span> takes the form:
<span class="math-container">$$
\mathbb{E}(\varepsilon \varepsilon') = V = \begin{bmatrix} \sigma_{11} I_N &amp; \sigma_{12}I_N &amp; \ldots &amp; \sigma_{1M} I_N\\
\sigma_{21} I_N &amp; \sigma_{22} I_N &amp; \ldots &amp; \sigma_{2N} I_N\\
\ldots  &amp; \ldots &amp; \ddots  &amp; \vdots\\
\sigma_{M1} I_N &amp; \ldots &amp; \ldots &amp; \sigma_{MM}I_N
\end{bmatrix} = \Sigma \otimes I_N
$$</span>
where <span class="math-container">$\otimes$</span> is the Kronecker product and:
<span class="math-container">$$
\Sigma = \begin{bmatrix}\sigma_{11} &amp; \sigma_{12} &amp; \ldots &amp; \sigma_{1M}\\
\sigma_{21} &amp; \sigma_{22} &amp; \ldots &amp; \sigma_{2M}\\
\vdots &amp; \vdots &amp; \ddots &amp; \vdots\\
\sigma_{M1} &amp; \sigma_{M2} &amp; \ldots &amp; \sigma_{MM}
\end{bmatrix}
$$</span>
<span class="math-container">$\Sigma$</span> gives the variance covariance matrix of the errors for a fixed individual.</p>
<p>For the Kronecker product, we have the rules: <span class="math-container">$(A \otimes B)^{-1} = A^{-1} \otimes B^{-1}$</span> and <span class="math-container">$(A \otimes B)(C \otimes D) = AC \otimes BD$</span> and <span class="math-container">$(A \otimes B)' = A' \otimes B'$</span> .</p>
<p>Let <span class="math-container">$\hat \Sigma$</span> be the estimate of <span class="math-container">$\Sigma$</span> based on an initial OLS estimation of <span class="math-container">$y_j$</span> on <span class="math-container">$X_j$</span> and let <span class="math-container">$\hat V = \hat \Sigma \otimes I_N$</span>. Then the feasible GLS estimator is given by:
<span class="math-container">$$
\begin{align*}
\hat \beta &amp;= (Z' \hat V^{-1} Z)^{-1} Z' \hat V^{-1} y,\\
&amp;=(Z'(\hat \Sigma \otimes I_N)^{-1}Z)^{-1}Z'(\hat \Sigma \otimes I_N)^{-1}y,\\
&amp;= (Z'(\hat \Sigma^{-1}\otimes I_N)Z)^{-1}Z'(\hat \Sigma^{-1}\otimes I_N)y,\\
&amp;= \beta + (Z'(\hat \Sigma^{-1}\otimes I_n)Z)^{-1}Z'y
\end{align*}
$$</span></p>
<p>Now, let us assume that all the <span class="math-container">$X_i$</span> are identical, say <span class="math-container">$X$</span>, then <span class="math-container">$Z = I_M \otimes X$</span> and we can further simplify:
<span class="math-container">$$
\begin{align*}
\hat \beta &amp;= (Z'(\hat \Sigma^{-1}\otimes I_N)Z)^{-1}Z'(\hat \Sigma^{-1}\otimes I_N)y,\\
&amp;= ((I_M \otimes X)'(\hat \Sigma^{-1}\otimes I_N)(I_M \otimes X))^{-1}(I_M \otimes X)'(\hat \Sigma^{-1}\otimes I_N)y,\\
&amp;= ((I_M \hat \Sigma^{-1}\otimes X'I_N)(I_M \otimes X))^{-1}(I_M \hat \Sigma^{-1} \otimes X' I_N)y,\\
&amp;= (\hat \Sigma^{-1} \otimes X'X)^{-1}(\hat \Sigma^{-1}\otimes X')y,\\
&amp;= (\hat \Sigma \otimes (X'X)^{-1})(\hat \Sigma^{-1}\otimes X')y,\\
&amp;= (\hat \Sigma\hat \Sigma^{-1} \otimes (X'X)^{-1}X')y\\
&amp;= (I_M \otimes (X'X)^{-1} X')y
\end{align*}
$$</span>
Notice that <span class="math-container">$\hat \Sigma$</span> disappeared from this equation. The last one equation can be written in the following way:
<span class="math-container">$$
\hat \beta = \begin{bmatrix} (X'X)^{-1}X'y_1\\
(X'X)^{-1} X'y_2\\
\vdots\\
(X'X)^{-1}X' y_1
\end{bmatrix} = \beta + \begin{bmatrix}(X'X)^{-1}X'\varepsilon_1,\\ (X'X)^{-1}X'\varepsilon_2\\\vdots \\ (X'X)^{-1}X'\varepsilon_M\end{bmatrix}
$$</span>
So the feasible GLS estimates are identical to the OLS estimates from an equation by equation estimation. Notice that this also means that the residuals <span class="math-container">$\hat \varepsilon_j$</span> will be identical to the residuals from an OLS estimation.</p>
<p>Now to estimate the variance covariance matrix, we take the product <span class="math-container">$(\hat \beta - \beta)(\hat \beta - \beta)'$</span> which gives a matrix with entries:
<span class="math-container">$$
\begin{align*}
(\hat \beta_{j} - \beta_j)(\hat \beta_j - \beta_j)' &amp;= [(X'X)^{-1}X' \varepsilon_j][(X'X)^{-1}X'\varepsilon_j]',\\
&amp;= (X'X)^{-1}X'\varepsilon_j \varepsilon_j'X(X'X)^{-1}
\end{align*}
$$</span>
Then for equation <span class="math-container">$j$</span>, we have the variance covariance matix:
<span class="math-container">$$
V(\hat \beta_j) =  \mathbb{E}((\hat \beta_j - \beta_j)(\hat \beta_j - \beta_j)) = \sigma_{jj}\left(X'X\right)^{-1},
$$</span></p>
<p>As <span class="math-container">$\sigma_{jj}$</span> is not known, it is usually estimated by <span class="math-container">$\hat \sigma_{jj} = \frac{1}{N}\sum_i \hat \varepsilon_{i,j}^2$</span> where <span class="math-container">$\hat \varepsilon_{i,j}$</span> are the residuals of the feasible GLS estimator. However, in this case, these will be identical to the residuals of an OLS estimator (as the estimators <span class="math-container">$\hat \beta$</span> are identical). As such, the estimates of the variances of <span class="math-container">$\hat \beta$</span> for the SUR will be identical to the variance estimates of the OLS estimates (equation by equation).</p>

CC BY-SA 4.0
2021-07-10T12:24:17.283Z
Another thorough answer from you – a joy to read!
Richard Hardy
CC BY-SA 4.0
2022-10-21T07:16:03.283Z
Very beautiful answer. Do you happen to know the sources of the answer which I can cite in a research paper?
Jason Goal
CC BY-SA 4.0
2021-07-09T08:26:06.370Z
2021-07-09T08:26:06.370Z
tdm
<p>In a SURE framework, if all X are the same in all regressions I was under the impression that there is no efficiency gain. Recently an assistant professor told me that the beta coefficients would be the same as OLS, but the standard errors would decrease due to the SURE framework even though all the X are the same.</p>
<p>Looking at the derivation in Greene's 7th edition, section 10.2.2, I believe I am correct.</p>
<p>Can anyone clarify further? Does SURE give an efficiency improvement if all X are the same in all regressions?</p>

CC BY-SA 4.0
2021-07-08T13:01:42.120Z
As far as I know, you are correct. I think chapter 3 or 4 in luktepohl's text might give a proof of your claim.
mark leeds
<econometrics><causal-inference>
Seemingly Unrelated Regression Estimation - Equivalent to OLS Standard errors?
Michael Gmeiner
# Seemingly Unrelated Regression Estimation - Equivalent to OLS Standard errors?

(Asked by: username_0 on 2021-07-08)

In a SURE framework, if all X are the same in all regressions I was under the impression that there is no efficiency gain. Recently an assistant professor told me that the beta coefficients would be the same as OLS, but the standard errors would decrease due to the SURE framework even though all the X are the same.

Looking at the derivation in Greene's 7th edition, section 10.2.2, I believe I am correct.

Can anyone clarify further? Does SURE give an efficiency improvement if all X are the same in all regressions?

**Tags:** econometrics, causal-inference
> **Comments:**
> **username_1**: As far as I know, you are correct. I think chapter 3 or 4 in luktepohl's text might give a proof of your claim.  

---

## 1 answers

### Answer by username_2 on 2021-07-09. Score: 6 | Accepted answer

Assume that for each observation $i = 1,\ldots, N$, we have $M$ equations:
$$
y\_{i,j} = x\_{i,j}\beta\_j + \varepsilon\_{i,j}
$$
Where $i = 1,\ldots, N$ enumerates individuals and and $j = 1,\ldots, M$ enumerates the equations. here $x$ is of size $1 \times k\_j$ and $\beta\_j$ is of size $k\_j \times 1$ and $k\_j$ is the number of covariates for regression $j$. Stacking over all $i = 1,\ldots N$, we get $M$ equations:
$$
y\_j = X\_j \beta\_j + \varepsilon\_j
$$
where now $X\_j$ is of size $N \times k\_j$. For simplicity, assume that $X\_j$ are non-stochastic. Next, assume that for all $i = 1,\ldots, N$ and $j = 1,\ldots, M$:
$$
\begin{align\*}
&\mathbb{E}(\varepsilon\_{i,j}) = 0,\\
&\mathbb{E}(\varepsilon\_{i,j}^2) = \sigma\_{jj}
\end{align\*}
$$
For the covariance between equations, let for all $i = 1,\ldots, N$ and $j,\ell = 1,\ldots, M$:
$$
\mathbb{E}(\varepsilon\_{i,j} \varepsilon\_{i,\ell}) = \sigma\_{j,\ell}
$$
while for all $j,\ell = 1,\ldots, M$ and $i,i' = 1,\ldots, N$ with $i \ne i'$:
$$
\mathbb{E}(\varepsilon\_{i,j}, \varepsilon\_{i',k}) = 0
$$
This means that errors for the same individual might be correlated across equations, while errors for different individuals are uncorrelated.

This can be expressed more compactly as:
$$
cov(\varepsilon\_j, \varepsilon\_{\ell}) = \sigma\_{j,\ell}I\_N
$$
Now, let us stack the various equations, one on top of the other:
$$
y = Z\beta + \varepsilon,
$$
where:
$$
y = \begin{bmatrix} y\_1\\y\_2\\ \vdots\\y\_M\end{bmatrix},
\varepsilon = \begin{bmatrix} \varepsilon\_1 \\ \vdots \\ \varepsilon\_M \end{bmatrix},\\
Z = \begin{bmatrix} X\_1 & 0 & \ldots & 0\\
0 & X\_2 & \ldots & 0,\\
\vdots & \vdots & \ddots & \vdots\\
0 & 0 & \ldots & X\_M
\end{bmatrix}, \beta = \begin{bmatrix} \beta\_1\\ \vdots \\ \beta\_M\end{bmatrix}
$$
The variance-covariance matrix of $\varepsilon$ takes the form:
$$
\mathbb{E}(\varepsilon \varepsilon') = V = \begin{bmatrix} \sigma\_{11} I\_N & \sigma\_{12}I\_N & \ldots & \sigma\_{1M} I\_N\\
\sigma\_{21} I\_N & \sigma\_{22} I\_N & \ldots & \sigma\_{2N} I\_N\\
\ldots & \ldots & \ddots & \vdots\\
\sigma\_{M1} I\_N & \ldots & \ldots & \sigma\_{MM}I\_N
\end{bmatrix} = \Sigma \otimes I\_N
$$
where $\otimes$ is the Kronecker product and:
$$
\Sigma = \begin{bmatrix}\sigma\_{11} & \sigma\_{12} & \ldots & \sigma\_{1M}\\
\sigma\_{21} & \sigma\_{22} & \ldots & \sigma\_{2M}\\
\vdots & \vdots & \ddots & \vdots\\
\sigma\_{M1} & \sigma\_{M2} & \ldots & \sigma\_{MM}
\end{bmatrix}
$$
$\Sigma$ gives the variance covariance matrix of the errors for a fixed individual.

For the Kronecker product, we have the rules: $(A \otimes B)^{-1} = A^{-1} \otimes B^{-1}$ and $(A \otimes B)(C \otimes D) = AC \otimes BD$ and $(A \otimes B)' = A' \otimes B'$ .

Let $\hat \Sigma$ be the estimate of $\Sigma$ based on an initial OLS estimation of $y\_j$ on $X\_j$ and let $\hat V = \hat \Sigma \otimes I\_N$. Then the feasible GLS estimator is given by:
$$
\begin{align\*}
\hat \beta &= (Z' \hat V^{-1} Z)^{-1} Z' \hat V^{-1} y,\\
&=(Z'(\hat \Sigma \otimes I\_N)^{-1}Z)^{-1}Z'(\hat \Sigma \otimes I\_N)^{-1}y,\\
&= (Z'(\hat \Sigma^{-1}\otimes I\_N)Z)^{-1}Z'(\hat \Sigma^{-1}\otimes I\_N)y,\\
&= \beta + (Z'(\hat \Sigma^{-1}\otimes I\_n)Z)^{-1}Z'y
\end{align\*}
$$

Now, let us assume that all the $X\_i$ are identical, say $X$, then $Z = I\_M \otimes X$ and we can further simplify:
$$
\begin{align\*}
\hat \beta &= (Z'(\hat \Sigma^{-1}\otimes I\_N)Z)^{-1}Z'(\hat \Sigma^{-1}\otimes I\_N)y,\\
&= ((I\_M \otimes X)'(\hat \Sigma^{-1}\otimes I\_N)(I\_M \otimes X))^{-1}(I\_M \otimes X)'(\hat \Sigma^{-1}\otimes I\_N)y,\\
&= ((I\_M \hat \Sigma^{-1}\otimes X'I\_N)(I\_M \otimes X))^{-1}(I\_M \hat \Sigma^{-1} \otimes X' I\_N)y,\\
&= (\hat \Sigma^{-1} \otimes X'X)^{-1}(\hat \Sigma^{-1}\otimes X')y,\\
&= (\hat \Sigma \otimes (X'X)^{-1})(\hat \Sigma^{-1}\otimes X')y,\\
&= (\hat \Sigma\hat \Sigma^{-1} \otimes (X'X)^{-1}X')y\\
&= (I\_M \otimes (X'X)^{-1} X')y
\end{align\*}
$$
Notice that $\hat \Sigma$ disappeared from this equation. The last one equation can be written in the following way:
$$
\hat \beta = \begin{bmatrix} (X'X)^{-1}X'y\_1\\
(X'X)^{-1} X'y\_2\\
\vdots\\
(X'X)^{-1}X' y\_1
\end{bmatrix} = \beta + \begin{bmatrix}(X'X)^{-1}X'\varepsilon\_1,\\ (X'X)^{-1}X'\varepsilon\_2\\\vdots \\ (X'X)^{-1}X'\varepsilon\_M\end{bmatrix}
$$
So the feasible GLS estimates are identical to the OLS estimates from an equation by equation estimation. Notice that this also means that the residuals $\hat \varepsilon\_j$ will be identical to the residuals from an OLS estimation.

Now to estimate the variance covariance matrix, we take the product $(\hat \beta - \beta)(\hat \beta - \beta)'$ which gives a matrix with entries:
$$
\begin{align\*}
(\hat \beta\_{j} - \beta\_j)(\hat \beta\_j - \beta\_j)' &= [(X'X)^{-1}X' \varepsilon\_j][(X'X)^{-1}X'\varepsilon\_j]',\\
&= (X'X)^{-1}X'\varepsilon\_j \varepsilon\_j'X(X'X)^{-1}
\end{align\*}
$$
Then for equation $j$, we have the variance covariance matix:
$$
V(\hat \beta\_j) = \mathbb{E}((\hat \beta\_j - \beta\_j)(\hat \beta\_j - \beta\_j)) = \sigma\_{jj}\left(X'X\right)^{-1},
$$

As $\sigma\_{jj}$ is not known, it is usually estimated by $\hat \sigma\_{jj} = \frac{1}{N}\sum\_i \hat \varepsilon\_{i,j}^2$ where $\hat \varepsilon\_{i,j}$ are the residuals of the feasible GLS estimator. However, in this case, these will be identical to the residuals of an OLS estimator (as the estimators $\hat \beta$ are identical). As such, the estimates of the variances of $\hat \beta$ for the SUR will be identical to the variance estimates of the OLS estimates (equation by equation).

> **Comments:**
> **username_3**: Another thorough answer from you – a joy to read!  
> **username_4**: Very beautiful answer. Do you happen to know the sources of the answer which I can cite in a research paper?  

---

