<p>Your notation is a bit all over the place, so I'm going to try and standardize it for a general case.</p>
<p>Let <span class="math-container">$X$</span> be the <em>regression matrix</em> (which includes a column of 1s for the intercept term) and <span class="math-container">$\beta$</span> be the vector of coefficients to be estimated via OLS (which includes the intercept term). <span class="math-container">$Var(u_i|X) = 10 = \sigma^2$</span>, for all <span class="math-container">$i$</span>. Thus the errors are homoscedastic (constant variance) and the variance-covariance matrix of the errors <span class="math-container">$\Omega$</span> has diagonal entries that are all equal to <span class="math-container">$\sigma^2$</span>. Furthermore, if you assume that the error terms are not serially correlated, then the off-diagonal covariance terms <span class="math-container">$Cov(u_i, u_j|X)$</span> for all <span class="math-container">$i \neq j$</span> in the variance-covariance matrix <span class="math-container">$\Omega$</span> are zero.</p>
<p>Those are two of the standard Gauss-Markov assumptions used to establish the BLUEness of the OLS estimator. Under these assumptions, <span class="math-container">$\Omega$</span> is</p>
<p><span class="math-container">$$
E[uu'|X] = 
    \left[
    \begin{array}{ccccc}
    \sigma^2                                    \\
      &amp; \sigma^2    &amp;   &amp; \huge0        \\
      &amp;               &amp;   \ddots                  \\
      &amp; \huge0 &amp;   &amp; \sigma^2        \\
      &amp;               &amp;   &amp;   &amp; \sigma^2
    \end{array}
    \right] = \sigma^2I.
$$</span></p>
<p>To get the variance of the OLS estimates <span class="math-container">$b = \hat{\beta}$</span>, first note that</p>
<p><span class="math-container">$$
\begin{align}
b &amp;= (X'X)^{-1}X'y \\
&amp;= (X'X)^{-1}X'(X\beta + u) \\
&amp;= \beta + (X'X)^{-1}X'u \\
\implies b - \beta &amp;= (X'X)^{-1}X'u,
\end{align}
$$</span></p>
<p>using <span class="math-container">$y = X\beta + u$</span>. Then,</p>
<p><span class="math-container">$$
\begin{align}
Var(b|X) &amp;= E[(b-\beta)(b-\beta)'|X] \\
&amp;= E[(X'X)^{-1}X'u((X'X)^{-1}X'u)'|X] \\
&amp;= E[(X'X)^{-1}X'uu'X(X'X)^{-1}|X] \\
&amp;= (X'X)^{-1}X'E[uu'|X]X(X'X)^{-1}.
\end{align}
$$</span></p>
<p>But recall that we derived <span class="math-container">$E[uu'|X] = \sigma^2 I$</span> via the Gauss-Markov assumption of <a href="https://en.wikipedia.org/wiki/Gauss%E2%80%93Markov_theorem#Spherical_errors" rel="noreferrer">spherical errors</a>. Thus, by substitution,</p>
<p><span class="math-container">$$
\begin{align}
Var(b|X) &amp;= (X'X)^{-1}X'E[uu'|X]X(X'X)^{-1} \\
&amp;= (X'X)^{-1}X'(\sigma^2 I)X(X'X)^{-1} \\
&amp;= \sigma^2 (X'X)^{-1} X'X(X'X)^{-1} \\
\implies Var(b) &amp;= \sigma^2 (X'X)^{-1}.
\end{align}
$$</span></p>
<p>The standard deviation of <span class="math-container">$b$</span> is just the square root of the variance, or</p>
<p><span class="math-container">$$
sd(b) = \sqrt{\sigma^2 (X'X)^{-1}}.
$$</span></p>
<p>To find the standard deviation of the <span class="math-container">$k^{th}$</span> estimated coefficient in <span class="math-container">$b$</span>, <span class="math-container">$b_{k}$</span>, we simply extract the <span class="math-container">$k^{th}$</span> diagonal element of <span class="math-container">$(X'X)^{-1}$</span>, denoted as <span class="math-container">$(X'X)_{kk}^{-1}$</span>:</p>
<p><span class="math-container">$$
sd(b_k) = \sqrt{\sigma^2 (X'X)_{kk}^{-1}}.
$$</span></p>
<hr />
<p><span class="math-container">$\sigma^2$</span> is the (common) variance of the errors and, unfortunately, this value is unobserved in our sample. In your question, however, it appears that this value is just given to you directly — it's <span class="math-container">$10$</span>. <em>In general, however, <span class="math-container">$\sigma^2$</span> must be estimated using the data</em>. It turns out that given homoscedastic errors, an unbiased estimator of <span class="math-container">$\sigma^2$</span> is</p>
<p><span class="math-container">$$
s^2 = \frac{e'e}{n-P},
$$</span></p>
<p>where <span class="math-container">$n$</span> is the number of observations, <span class="math-container">$P$</span> is the number of columns in <span class="math-container">$X$</span>, and <span class="math-container">$e = \hat{u}$</span>; that is, the vector of residuals <span class="math-container">$y - Xb$</span>. <span class="math-container">$s$</span>, incidentally, is called the &quot;standard error of the regression&quot;, not to be confused with the standard errors of the OLS estimates.</p>
<p>Thus, the <strong>standard error of <span class="math-container">$\mathbf{b_k}$</span></strong> is estimated as</p>
<p><span class="math-container">$$
\boldsymbol{\widehat{se}(b_k) = \sqrt{s^2 (X'X)_{kk}^{-1}}},
$$</span></p>
<p>which is Hayashi's result.</p>

CC BY-SA 4.0
2018-10-27T13:44:53.900Z
Thank you so much for the thorough explanation! Really cleared things up.
user11767
CC BY-SA 4.0
2018-10-27T04:05:33.590Z
2021-06-30T04:31:50.260Z
2021-06-30T04:31:50.260Z
Kenneth Rios
<p>Consider a regression model on the form:</p>

<p><span class="math-container">$y_{i} = \alpha + \beta_{1}X_{i} + u_{i}$</span> (1)</p>

<p>I am given <span class="math-container">$var(u_{i}|X_{i}) = 10$</span> and I know the OLS estimates for <span class="math-container">$\alpha$</span> and <span class="math-container">$\beta$</span> and using this I have to calculate the standard errors of the parameter estimates. </p>

<p>I am following Hayashi's book and I know that</p>

<p><span class="math-container">$se(b_{k}) = \sqrt(s^{2}(X'X)^{-1}_{kk})$</span></p>

<p>Where <span class="math-container">$b_{k}$</span> is the parameter estimate of interest. I know <span class="math-container">$(X'X)^{-1}$</span> but what is confusing me is going from the given model to expressing it in matrix notation and that this does not use the variance of the error term.</p>

<p>I can express (1) as <span class="math-container">$Y = X\beta + u$</span> and since <span class="math-container">$var(u_{i}|X_{i}) = var(y_{i} - \alpha - \beta_{2}X_{i}|X_{i})$</span> but is this equal to <span class="math-container">$var(Y - X\beta|X)$</span>?</p>

<p>If so, I can express <span class="math-container">$var(u_{i}|X_{i}) = 10$</span> as <span class="math-container">$\sigma^{2}I_{n}$</span> where <span class="math-container">$\sigma^{2} = 10$</span>. And then could I use the fact that</p>

<p><span class="math-container">$var(b_{k}|X) = (X'X)^{-1}X' \sigma^{2} I_{n} X(X'X)^{-1}$</span></p>

<p>To find the standard errors of interest.</p>

<econometrics>
Calculating standard error
user11767
# Calculating standard error

(Asked by: username_0 on 2018-10-26)

Consider a regression model on the form:

$y\_{i} = \alpha + \beta\_{1}X\_{i} + u\_{i}$ (1)

I am given $var(u\_{i}|X\_{i}) = 10$ and I know the OLS estimates for $\alpha$ and $\beta$ and using this I have to calculate the standard errors of the parameter estimates.

I am following Hayashi's book and I know that

$se(b\_{k}) = \sqrt(s^{2}(X'X)^{-1}\_{kk})$

Where $b\_{k}$ is the parameter estimate of interest. I know $(X'X)^{-1}$ but what is confusing me is going from the given model to expressing it in matrix notation and that this does not use the variance of the error term.

I can express (1) as $Y = X\beta + u$ and since $var(u\_{i}|X\_{i}) = var(y\_{i} - \alpha - \beta\_{2}X\_{i}|X\_{i})$ but is this equal to $var(Y - X\beta|X)$?

If so, I can express $var(u\_{i}|X\_{i}) = 10$ as $\sigma^{2}I\_{n}$ where $\sigma^{2} = 10$. And then could I use the fact that

$var(b\_{k}|X) = (X'X)^{-1}X' \sigma^{2} I\_{n} X(X'X)^{-1}$

To find the standard errors of interest.

**Tags:** econometrics

---

## 1 answers

### Answer by username_1 on 2018-10-27. Score: 10 | Accepted answer

Your notation is a bit all over the place, so I'm going to try and standardize it for a general case.

Let $X$ be the *regression matrix* (which includes a column of 1s for the intercept term) and $\beta$ be the vector of coefficients to be estimated via OLS (which includes the intercept term). $Var(u\_i|X) = 10 = \sigma^2$, for all $i$. Thus the errors are homoscedastic (constant variance) and the variance-covariance matrix of the errors $\Omega$ has diagonal entries that are all equal to $\sigma^2$. Furthermore, if you assume that the error terms are not serially correlated, then the off-diagonal covariance terms $Cov(u\_i, u\_j|X)$ for all $i \neq j$ in the variance-covariance matrix $\Omega$ are zero.

Those are two of the standard Gauss-Markov assumptions used to establish the BLUEness of the OLS estimator. Under these assumptions, $\Omega$ is

$$
E[uu'|X] =
\left[
\begin{array}{ccccc}
\sigma^2 \\
& \sigma^2 & & \huge0 \\
& & \ddots \\
& \huge0 & & \sigma^2 \\
& & & & \sigma^2
\end{array}
\right] = \sigma^2I.
$$

To get the variance of the OLS estimates $b = \hat{\beta}$, first note that

$$
\begin{align}
b &= (X'X)^{-1}X'y \\
&= (X'X)^{-1}X'(X\beta + u) \\
&= \beta + (X'X)^{-1}X'u \\
\implies b - \beta &= (X'X)^{-1}X'u,
\end{align}
$$

using $y = X\beta + u$. Then,

$$
\begin{align}
Var(b|X) &= E[(b-\beta)(b-\beta)'|X] \\
&= E[(X'X)^{-1}X'u((X'X)^{-1}X'u)'|X] \\
&= E[(X'X)^{-1}X'uu'X(X'X)^{-1}|X] \\
&= (X'X)^{-1}X'E[uu'|X]X(X'X)^{-1}.
\end{align}
$$

But recall that we derived $E[uu'|X] = \sigma^2 I$ via the Gauss-Markov assumption of [spherical errors](https://en.wikipedia.org/wiki/Gauss%E2%80%93Markov_theorem#Spherical_errors). Thus, by substitution,

$$
\begin{align}
Var(b|X) &= (X'X)^{-1}X'E[uu'|X]X(X'X)^{-1} \\
&= (X'X)^{-1}X'(\sigma^2 I)X(X'X)^{-1} \\
&= \sigma^2 (X'X)^{-1} X'X(X'X)^{-1} \\
\implies Var(b) &= \sigma^2 (X'X)^{-1}.
\end{align}
$$

The standard deviation of $b$ is just the square root of the variance, or

$$
sd(b) = \sqrt{\sigma^2 (X'X)^{-1}}.
$$

To find the standard deviation of the $k^{th}$ estimated coefficient in $b$, $b\_{k}$, we simply extract the $k^{th}$ diagonal element of $(X'X)^{-1}$, denoted as $(X'X)\_{kk}^{-1}$:

$$
sd(b\_k) = \sqrt{\sigma^2 (X'X)\_{kk}^{-1}}.
$$

---

$\sigma^2$ is the (common) variance of the errors and, unfortunately, this value is unobserved in our sample. In your question, however, it appears that this value is just given to you directly — it's $10$. *In general, however, $\sigma^2$ must be estimated using the data*. It turns out that given homoscedastic errors, an unbiased estimator of $\sigma^2$ is

$$
s^2 = \frac{e'e}{n-P},
$$

where $n$ is the number of observations, $P$ is the number of columns in $X$, and $e = \hat{u}$; that is, the vector of residuals $y - Xb$. $s$, incidentally, is called the "standard error of the regression", not to be confused with the standard errors of the OLS estimates.

Thus, the **standard error of $\mathbf{b\_k}$** is estimated as

$$
\boldsymbol{\widehat{se}(b\_k) = \sqrt{s^2 (X'X)\_{kk}^{-1}}},
$$

which is Hayashi's result.

> **Comments:**
> **username_0**: Thank you so much for the thorough explanation! Really cleared things up.  

---

