<p>I think I figured out 1). I am still interested in 2).</p>

<p>OLS sets $b_0$ and $b_1$ in such a way that</p>

<p>$$
(b_0 \ \  b_1) = \arg\min_{a_0, a_1} \sum_j \left(Y_j - a_0 - a_1 X_j\right)^2. 
$$
The first order conditions are
$$
(-2)\sum_j \left(Y_j - b_0 - b_1 X_j\right) = 0 
$$
and
$$
(-2)\sum_j X_j \left(Y_j - b_0 - b_1 X_j\right) = 0.
$$
If observation $i$ is on the regression line then
$$
\begin{align*}
\left(Y_i - b_0 - b_1 X_i\right) &amp; = 0 \\
\\
X_i\left(Y_i - b_0 - b_1 X_i\right) &amp; = 0.
\end{align*}
$$
It follows from this and the original first order conditions that
$$
\begin{align*}
(-2)\sum_{j\neq i} \left(Y_j - b_0 - b_1 X_j\right) &amp; = 0 \\
\\
(-2)\sum_{j\neq i} X_j\left(Y_j - b_0 - b_1 X_j\right) &amp; = 0,
\end{align*}
$$
which are the first order conditions of the subsample's OLS problem.</p>

CC BY-SA 4.0
2018-07-23T15:05:33.040Z
2018-07-23T17:12:17.517Z
2018-07-23T17:12:17.517Z
Giskard
<p>Since Q1 has been solved, I'll focus on Q2. Yes, it's possible to
remove a sample point that is not on the regression line but still
yields $b_{1}=b_{1}'$ and $b_{0}\ne b_{0}'$. Such points have the property $x_{j}=\bar{x}$,
where $\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_{i}$. These points are
known as points <strong>without leverage</strong>.</p>

<p>Suppose $\left(x_{j},y_{j}\right)$ is a point without leverage but
not on the regression line, that is, $x_{j}=\overline{x}$ and $y_{j}\ne\overline{y}$.
We know</p>

<p>\begin{align*}
b_{1} &amp; =\frac{\sum_{i}\left(x_{i}-\bar{x}\right)y_{i}}{\sum_{i}\left(x_{i}-\bar{x}\right)^{2}}\\
 &amp; =\frac{\sum_{-j}\left(x_{i}-\bar{x}\right)y_{i}+\left(x_{j}-\overline{x}\right)y_{j}}{\sum_{-j}\left(x_{i}-\bar{x}\right)^{2}+\left(x_{j}-\overline{x}\right)^{2}}\\
 &amp; =\frac{\sum_{-j}\left(x_{i}-\bar{x}\right)y_{i}}{\sum_{-j}\left(x_{i}-\bar{x}\right)^{2}}\\
 &amp; =\frac{\sum_{-j}\left(x_{i}-\bar{x}_{-j}\right)y_{i}}{\sum_{-j}\left(x_{i}-\bar{x}_{-j}\right)^{2}}\\
 &amp; =b_{1}'
\end{align*}
here we use the conditions $x_{j}-\overline{x}=0$ and $\overline{x}=\overline{x}_{-j}$.
Combined with the condition $\overline{y}\ne\overline{y}_{-j}$, we
have 
\begin{align*}
b_{0}' &amp; =\overline{y}_{-j}-b_{1}'\overline{x}_{-j}\\
 &amp; \ne\overline{y}-b_{1}\overline{x}\\
 &amp; =b_{0}
\end{align*}</p>

CC BY-SA 4.0
2018-07-24T04:15:43.677Z
2018-07-24T04:15:43.677Z
Paul
<p>There is a sample of $n$ observations, each element has a numeric $Y$ and $X$ characteristic. There is an OLS regression over the sample
$$
Y = b_0 + b_1 X + \textbf{u},
$$
$\textbf{u}$ being the vector of residuals. Suppose we remove observation $i$ from the sample, restricting it to $n-1$ observations, and run an OLS regression again, yielding
$$
Y_{-i} = b_0' + b_1' X_{-i} + \textbf{u}'.
$$
Subscript $_{-i}$ denotes that $Y$ and $X$ are not the same vectors as before, as observation $i$ is missing.</p>

<p>Seems to me that if the observation $i$ we removed is on the original sample's 'regression line', that is if
$$
Y_i = b_0 + b_1 X_i,
$$
then $b_0 = b_0'$ and $b_1 = b_1'$.</p>

<p>Example: (in R code)</p>

<pre><code>x = c(5,3,4,5,4,4)
y = c(20,15,14,21,10,25)

plot(x,y)
abline(coef(reg))
</code></pre>

<p><a href="https://i.sstatic.net/7Rg1G.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/7Rg1G.png" alt="regression line"></a></p>

<pre><code>plot(x[-1],y[-1])

reg1 = lm(y[-1] ~ x[-1])
abline(coef(reg1))
</code></pre>

<p><a href="https://i.sstatic.net/AECl5.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/AECl5.png" alt="subsample regression"></a></p>

<p>My questions are:</p>

<p>1) Is this true, and if yes, what is the proof? <br>
[I figured this one out in the meantime, but feel free to give a nicer solution.]</p>

<p>2) Is it possible that after removing an observation (not on the 'regression line') we have</p>

<p>($b_0 = b_0'$ and $b_1 \neq b_1'$)  or ($b_0 \neq b_0'$ and $b_1 = b_1'$)?</p>

<econometrics><regression>
When is an OLS parameter unchanged on a subsample?
Giskard
# When is an OLS parameter unchanged on a subsample?

(Asked by: username_0 on 2018-07-23)

There is a sample of $n$ observations, each element has a numeric $Y$ and $X$ characteristic. There is an OLS regression over the sample
$$
Y = b_0 + b_1 X + \textbf{u},
$$
$\textbf{u}$ being the vector of residuals. Suppose we remove observation $i$ from the sample, restricting it to $n-1$ observations, and run an OLS regression again, yielding
$$
Y_{-i} = b_0' + b_1' X_{-i} + \textbf{u}'.
$$
Subscript $_{-i}$ denotes that $Y$ and $X$ are not the same vectors as before, as observation $i$ is missing.
Seems to me that if the observation $i$ we removed is on the original sample's 'regression line', that is if
$$
Y_i = b_0 + b_1 X_i,
$$
then $b_0 = b_0'$ and $b_1 = b_1'$.
Example: (in R code)
x = c(5,3,4,5,4,4)
y = c(20,15,14,21,10,25)

plot(x,y)
abline(coef(reg))


plot(x[-1],y[-1])

reg1 = lm(y[-1] ~ x[-1])
abline(coef(reg1))


My questions are:
1) Is this true, and if yes, what is the proof? 
[I figured this one out in the meantime, but feel free to give a nicer solution.]
2) Is it possible that after removing an observation (not on the 'regression line') we have
($b_0 = b_0'$ and $b_1 \neq b_1'$)  or ($b_0 \neq b_0'$ and $b_1 = b_1'$)?

**Tags:** econometrics, regression

---

## 2 answers

### Answer by username_1 on 2018-07-24. Score: 2 | Accepted answer

Since Q1 has been solved, I'll focus on Q2. Yes, it's possible to
remove a sample point that is not on the regression line but still
yields $b\_{1}=b\_{1}'$ and $b\_{0}\ne b\_{0}'$. Such points have the property $x\_{j}=\bar{x}$,
where $\bar{x}=\frac{1}{n}\sum\_{i=1}^{n}x\_{i}$. These points are
known as points **without leverage**.

Suppose $\left(x\_{j},y\_{j}\right)$ is a point without leverage but
not on the regression line, that is, $x\_{j}=\overline{x}$ and $y\_{j}\ne\overline{y}$.
We know

\begin{align\*}
b\_{1} & =\frac{\sum\_{i}\left(x\_{i}-\bar{x}\right)y\_{i}}{\sum\_{i}\left(x\_{i}-\bar{x}\right)^{2}}\\
& =\frac{\sum\_{-j}\left(x\_{i}-\bar{x}\right)y\_{i}+\left(x\_{j}-\overline{x}\right)y\_{j}}{\sum\_{-j}\left(x\_{i}-\bar{x}\right)^{2}+\left(x\_{j}-\overline{x}\right)^{2}}\\
& =\frac{\sum\_{-j}\left(x\_{i}-\bar{x}\right)y\_{i}}{\sum\_{-j}\left(x\_{i}-\bar{x}\right)^{2}}\\
& =\frac{\sum\_{-j}\left(x\_{i}-\bar{x}\_{-j}\right)y\_{i}}{\sum\_{-j}\left(x\_{i}-\bar{x}\_{-j}\right)^{2}}\\
& =b\_{1}'
\end{align\*}
here we use the conditions $x\_{j}-\overline{x}=0$ and $\overline{x}=\overline{x}\_{-j}$.
Combined with the condition $\overline{y}\ne\overline{y}\_{-j}$, we
have
\begin{align\*}
b\_{0}' & =\overline{y}\_{-j}-b\_{1}'\overline{x}\_{-j}\\
& \ne\overline{y}-b\_{1}\overline{x}\\
& =b\_{0}
\end{align\*}


---

### Answer by username_0 on 2018-07-23. Score: 3

I think I figured out 1). I am still interested in 2).

OLS sets $b\_0$ and $b\_1$ in such a way that

$$
(b\_0 \ \ b\_1) = \arg\min\_{a\_0, a\_1} \sum\_j \left(Y\_j - a\_0 - a\_1 X\_j\right)^2.
$$
The first order conditions are
$$
(-2)\sum\_j \left(Y\_j - b\_0 - b\_1 X\_j\right) = 0
$$
and
$$
(-2)\sum\_j X\_j \left(Y\_j - b\_0 - b\_1 X\_j\right) = 0.
$$
If observation $i$ is on the regression line then
$$
\begin{align\*}
\left(Y\_i - b\_0 - b\_1 X\_i\right) & = 0 \\
\\
X\_i\left(Y\_i - b\_0 - b\_1 X\_i\right) & = 0.
\end{align\*}
$$
It follows from this and the original first order conditions that
$$
\begin{align\*}
(-2)\sum\_{j\neq i} \left(Y\_j - b\_0 - b\_1 X\_j\right) & = 0 \\
\\
(-2)\sum\_{j\neq i} X\_j\left(Y\_j - b\_0 - b\_1 X\_j\right) & = 0,
\end{align\*}
$$
which are the first order conditions of the subsample's OLS problem.


---

