<p>tldr:</p>
<ul>
<li>If <span class="math-container">$D_i$</span> is a dummy variable, then if <span class="math-container">$cov(L_i, D_i)&gt; 0$</span> then OLS will overestimate the mean effect of <span class="math-container">$D_i$</span> on <span class="math-container">$y_i$</span>. If <span class="math-container">$cov(L_i, D_i)&lt; 0$</span> the OLS estimate will underestimate this mean effect.</li>
<li>If <span class="math-container">$D_i$</span> is not a dummy variable then the effect also depends on the covariance between <span class="math-container">$L_i$</span> and <span class="math-container">$(D_i)^2$</span>.</li>
</ul>
<p>Assume that the true model is given by:
<span class="math-container">$$
y_i = \gamma + \dfrac{1}{1 + e(1-\alpha)}L_i D_i + \eta_i,
$$</span>
where <span class="math-container">$\eta_i$</span> is uncorrelated with <span class="math-container">$D_i$</span> and has mean zero.</p>
<p>For a variable <span class="math-container">$X_i$</span> write <span class="math-container">$\bar X = \mathbb{E}(X_i)$</span> then we can rewrite above equation as:
<span class="math-container">$$
y_i = \gamma + \dfrac{\bar L}{1 + e(1-\alpha)}D_i + \sigma_i,
$$</span>
where
<span class="math-container">$$
\sigma_i = \eta_i + \dfrac{(L_i - \bar L)}{1 + e(1-\alpha)} D_i.
$$</span>
Notice that <span class="math-container">$\sigma_i$</span> has mean:
<span class="math-container">$$
\begin{align*}
\mathbb{E}(\sigma_i) &amp;= \dfrac{\mathbb{E}(L_i D_i) - \bar L \bar D}{1 + e(1-\alpha)}\\
&amp;= \dfrac{cov(L_i,D_i)}{1 + e(1-\alpha)}.
\end{align*}
$$</span>
Which is not necessarily zero, but this is not a big problem as it will be captured by the constant term in the regression.</p>
<p>Assume we run the following specification:
<span class="math-container">$$
y_i = \delta + \beta D_i + \varepsilon_i,
$$</span>
Then the estimate of <span class="math-container">$\beta$</span> will asymptotically estimate:
<span class="math-container">$$
\dfrac{cov(y_i, D_i)}{var(D_i)}.
$$</span>
Let's compute the numerator:
<span class="math-container">$$
\begin{align*}
y_i - \bar y &amp;= \dfrac{\bar L}{1 + e(1-\alpha)}(D_i - \bar D) + \sigma_i - \mathbb{E}(\sigma_i)\\
&amp;= \dfrac{\bar L}{1 + e(1-\alpha)}(D_i - \bar D) + \eta_i + \dfrac{(L_i - L) D_i}{1 + e(1-\alpha)}- \dfrac{cov(L_i,D_i)}{1 + e(1-\alpha)}.
\end{align*}
$$</span>
Then:
<span class="math-container">$$
\begin{align*}
(y_i - \bar y)(D_i - \bar D) = \dfrac{\bar L}{1 + e(1-\alpha)}(D_i - \bar D)^2 + \eta_i(D_i - D) + \dfrac{(L_i - \bar L)D_i(D_i - \bar D)}{1 + e(1-\alpha)} - \dfrac{cov(L_i, D_i)}{1 + e(1-\alpha)}(D_i - \bar D)
\end{align*}
$$</span>
So taking the expectation of both sides gives:
<span class="math-container">$$
cov(y_i, D_i) = \dfrac{\bar L}{1 + e(1-\alpha)}var(D_i) + \dfrac{\mathbb{E}((L_i - \bar L)(D_i - \bar D)D_i)}{1 + e(1-\alpha)}
$$</span>
The last term gives:
<span class="math-container">$$
\begin{align*}
&amp;\dfrac{1}{1 + e(1-\alpha)}\left[\mathbb{E}(L_i D_i D_i) - \mathbb{E}(\bar L D_i D_i) - \mathbb{E}(L_i \bar D D_i) + \mathbb{E}(\bar L \bar D D_i) \right],\\
&amp;=\dfrac{1}{1 + e(1-\alpha)}\left[\mathbb{E}(L_i D_i D_i) - \bar L \mathbb{E}(D_i D_i) - \bar D \mathbb{E}(L_i D_i) + \bar L (\bar D)^2 \right],\\
&amp;=\dfrac{1}{1 + e(1-\alpha)}\left[\mathbb{E}(L_i (D_i)^2) - \bar L \overline{D^2} - \bar D\left[\mathbb{E}(L_i D_i) - \bar L \bar D\right] \right],\\
&amp;= \dfrac{1}{1 + e(1-\alpha)} \left[ cov(L_i, (D_i)^2) - \bar D cov(L_i, D_i)\right]
\end{align*}
$$</span>
So the OLS estimate will converge to:
<span class="math-container">$$
\dfrac{\bar L}{1 + e(1-\alpha)} + \dfrac{1}{1 + e(1-\alpha)} \dfrac{cov(L_i, (D_i)^2)}{var(D_i)} - \bar D \dfrac{1}{1 + e(1-\alpha)} \dfrac{cov(L_i, D_i)}{var(D_i)}.
$$</span>
If <span class="math-container">$D_i$</span> is a dummy variable then <span class="math-container">$(D_i)^2 = D_i$</span> so <span class="math-container">$cov(L_i,(D_i)^2) = cov(L_i, D_i)$</span> and <span class="math-container">$var(D_i) = \bar D(1-\bar D)$</span> so, we get:
<span class="math-container">$$
\begin{align*}
&amp;\dfrac{\bar L}{1 + e(1-\alpha)} + \dfrac{1}{1 + e(1-\alpha)}(1 - \bar D)\dfrac{cov(L_i, D_i)}{\bar D(1-\bar D)},\\
&amp;= \dfrac{\bar L}{1 + e(1-\alpha)} + \dfrac{1}{1 + e(1-\alpha)}\dfrac{cov(L_i, D_i)}{\bar D},
\end{align*}
$$</span>
If <span class="math-container">$cov(L_i, D_i) &gt; 0$</span> then the OLS estimate will tend to overestimate <span class="math-container">$\dfrac{\bar L}{1 + e(1-\alpha)}$</span> if <span class="math-container">$cov(L_i, D_i) &lt; 0$</span> then the OLS will underestimate this mean effect of <span class="math-container">$D$</span> on <span class="math-container">$y$</span>.</p>

CC BY-SA 4.0
2021-12-04T16:24:56.277Z
2021-12-04T16:24:56.277Z
tdm
<p>Use non-parametric regression. A particular version that comes to mind is <strong>local linear regression</strong>. This can be written as</p>
<p><span class="math-container">$$Y_i = \beta_{0i} + \beta_{1i}X_i + \epsilon_i,$$</span></p>
<p>thereby allowing for individual heterogeneity in <span class="math-container">$\partial Y_i/\partial D_i = \beta_{1i}$</span>. The model is fitted using weighted regression such that</p>
<p><span class="math-container">$$\hat \beta_i = (X^\top W_iX)^{-1}(X^\top W_iY),$$</span></p>
<p>where <span class="math-container">$W_i$</span> is an <span class="math-container">$N \times N$</span> diagonal matrix with the <span class="math-container">$n$</span>'th diagonal element being <span class="math-container">$K((x_i-x_n)/h)$</span> where <span class="math-container">$K$</span> is the chosen kernel and <span class="math-container">$h$</span> is the bandwidth (see for example Le and Racine 'Non-Parametric Econometrics' page 81).</p>

CC BY-SA 4.0
2021-12-03T09:31:31.883Z
This looks like weighted least squares. That is an estimation method while weighted regression is probably not the right way of phrasing that.
Richard Hardy
CC BY-SA 4.0
2021-12-03T10:30:04.430Z
Because the weight matrix is diagonal each observation is assigned one and only one weight as in weighted regression. Numerically you get $(\hat \beta_{i1})$ for $i=1,...,N$ by running a single weighted least squares estimation. But you have to do this procedure $N$ times to get all $\hat \beta_{i1}$. And hence it is different from WLS in that sense.
Jesper Hybel
CC BY-SA 4.0
2021-12-03T10:31:08.800Z
As for the choice of wording see https://en.wikipedia.org/wiki/Weighted_least_squares, where they say 'Weighted least squares (WLS), also known as weighted linear regression'. So the wording is not mine.
Jesper Hybel
CC BY-SA 4.0
2021-12-03T10:38:47.817Z
Wikipedia is not free of poor wording, so it is not a very convincing argument. I do not disagree with your broader point. My quibble is that using *regression* in the meaning of an estimation method is questionable. A lot of people have been unintentionally and unfortunately mislead by muddling the distinction between a model, an estimator and certain other elements. (Phrases like "an OLS model" are ubiquitous even if they do not make sense.) Perhaps WLS is not the right name here either, so we can look for something better.
Richard Hardy
CC BY-SA 4.0
2021-12-03T10:52:46.357Z
And don't get me started on the [notion of a regression model](https://stats.stackexchange.com/questions/173660) :)
Richard Hardy
CC BY-SA 4.0
2021-12-03T11:11:30.963Z
There is no reason to look for 'something better' the estimation method already has the name 'local linear regression'. That, however, does not change the fact that solving numerically for the coefficients in a local linear regression model involves a step equivalent to solving $N$ GLS problems and each of these can be performed by calling in R for example lm(y~d,weights=w). Personally, I have no quarrel referring to this as 'weighted regression' and it is certainly not a word I made up.
Jesper Hybel
CC BY-SA 4.0
2021-12-04T16:20:41.187Z
Thank you all for your comments!
hbkn
CC BY-SA 4.0
2021-12-03T07:06:52.057Z
2021-12-03T07:06:52.057Z
Jesper Hybel
<p>When I run the following regression, I find that <span class="math-container">$\beta&gt;0$</span></p>
<p><span class="math-container">$$ y_i=\beta D_i+\varepsilon_i$$</span></p>
<p>To explain the finding, I wrote down a conceptual framework and I obtained the following prediction</p>
<p><span class="math-container">$$\frac{\partial y_i}{\partial D_i}=L_i \frac{1}{1+e(1-\alpha)}$$</span></p>
<p>By assumption <span class="math-container">$\alpha \in (0,1)$</span>, <span class="math-container">$e&gt;0$</span>, and <span class="math-container">$L_i&gt;0$</span>. Therefore <span class="math-container">$\frac{\partial y_i}{\partial D_i}&gt;0$</span> is generally confirmed. But what worries me is that the <span class="math-container">$L_i$</span> term that varies by <span class="math-container">$i$</span>. FWIW, <span class="math-container">$\frac{\partial L_i}{\partial D_i}$</span> is a function of <span class="math-container">$D_i$</span>. How can I account for the <span class="math-container">$L_i$</span> in my econometric framework?</p>

<econometrics><applied-econometrics>
From conceptual framework to econometric estimation
hbkn
# From conceptual framework to econometric estimation

(Asked by: username_0 on 2021-12-02)

When I run the following regression, I find that $\beta>0$

$$ y\_i=\beta D\_i+\varepsilon\_i$$

To explain the finding, I wrote down a conceptual framework and I obtained the following prediction

$$\frac{\partial y\_i}{\partial D\_i}=L\_i \frac{1}{1+e(1-\alpha)}$$

By assumption $\alpha \in (0,1)$, $e>0$, and $L\_i>0$. Therefore $\frac{\partial y\_i}{\partial D\_i}>0$ is generally confirmed. But what worries me is that the $L\_i$ term that varies by $i$. FWIW, $\frac{\partial L\_i}{\partial D\_i}$ is a function of $D\_i$. How can I account for the $L\_i$ in my econometric framework?

**Tags:** econometrics, applied-econometrics

---

## 2 answers

### Answer by username_2 on 2021-12-03. Score: 6 | Accepted answer

Use non-parametric regression. A particular version that comes to mind is **local linear regression**. This can be written as

$$Y\_i = \beta\_{0i} + \beta\_{1i}X\_i + \epsilon\_i,$$

thereby allowing for individual heterogeneity in $\partial Y\_i/\partial D\_i = \beta\_{1i}$. The model is fitted using weighted regression such that

$$\hat \beta\_i = (X^\top W\_iX)^{-1}(X^\top W\_iY),$$

where $W\_i$ is an $N \times N$ diagonal matrix with the $n$'th diagonal element being $K((x\_i-x\_n)/h)$ where $K$ is the chosen kernel and $h$ is the bandwidth (see for example Le and Racine 'Non-Parametric Econometrics' page 81).

> **Comments:**
> **username_3**: This looks like weighted least squares. That is an estimation method while weighted regression is probably not the right way of phrasing that.  
> **username_2**: Because the weight matrix is diagonal each observation is assigned one and only one weight as in weighted regression. Numerically you get $(\hat \beta_{i1})$ for $i=1,...,N$ by running a single weighted least squares estimation. But you have to do this procedure $N$ times to get all $\hat \beta_{i1}$. And hence it is different from WLS in that sense.  
> **username_2**: As for the choice of wording see https://en.wikipedia.org/wiki/Weighted_least_squares, where they say 'Weighted least squares (WLS), also known as weighted linear regression'. So the wording is not mine.  
> **username_3**: Wikipedia is not free of poor wording, so it is not a very convincing argument. I do not disagree with your broader point. My quibble is that using *regression* in the meaning of an estimation method is questionable. A lot of people have been unintentionally and unfortunately mislead by muddling the distinction between a model, an estimator and certain other elements. (Phrases like "an OLS model" are ubiquitous even if they do not make sense.) Perhaps WLS is not the right name here either, so we can look for something better.  
> **username_3**: And don't get me started on the [notion of a regression model](https://stats.stackexchange.com/questions/173660) :)  
> **username_2**: There is no reason to look for 'something better' the estimation method already has the name 'local linear regression'. That, however, does not change the fact that solving numerically for the coefficients in a local linear regression model involves a step equivalent to solving $N$ GLS problems and each of these can be performed by calling in R for example lm(y~d,weights=w). Personally, I have no quarrel referring to this as 'weighted regression' and it is certainly not a word I made up.  
> **username_0**: Thank you all for your comments!  

---

### Answer by username_1 on 2021-12-04. Score: 2

tldr:

* If $D\_i$ is a dummy variable, then if $cov(L\_i, D\_i)> 0$ then OLS will overestimate the mean effect of $D\_i$ on $y\_i$. If $cov(L\_i, D\_i)< 0$ the OLS estimate will underestimate this mean effect.
* If $D\_i$ is not a dummy variable then the effect also depends on the covariance between $L\_i$ and $(D\_i)^2$.

Assume that the true model is given by:
$$
y\_i = \gamma + \dfrac{1}{1 + e(1-\alpha)}L\_i D\_i + \eta\_i,
$$
where $\eta\_i$ is uncorrelated with $D\_i$ and has mean zero.

For a variable $X\_i$ write $\bar X = \mathbb{E}(X\_i)$ then we can rewrite above equation as:
$$
y\_i = \gamma + \dfrac{\bar L}{1 + e(1-\alpha)}D\_i + \sigma\_i,
$$
where
$$
\sigma\_i = \eta\_i + \dfrac{(L\_i - \bar L)}{1 + e(1-\alpha)} D\_i.
$$
Notice that $\sigma\_i$ has mean:
$$
\begin{align\*}
\mathbb{E}(\sigma\_i) &= \dfrac{\mathbb{E}(L\_i D\_i) - \bar L \bar D}{1 + e(1-\alpha)}\\
&= \dfrac{cov(L\_i,D\_i)}{1 + e(1-\alpha)}.
\end{align\*}
$$
Which is not necessarily zero, but this is not a big problem as it will be captured by the constant term in the regression.

Assume we run the following specification:
$$
y\_i = \delta + \beta D\_i + \varepsilon\_i,
$$
Then the estimate of $\beta$ will asymptotically estimate:
$$
\dfrac{cov(y\_i, D\_i)}{var(D\_i)}.
$$
Let's compute the numerator:
$$
\begin{align\*}
y\_i - \bar y &= \dfrac{\bar L}{1 + e(1-\alpha)}(D\_i - \bar D) + \sigma\_i - \mathbb{E}(\sigma\_i)\\
&= \dfrac{\bar L}{1 + e(1-\alpha)}(D\_i - \bar D) + \eta\_i + \dfrac{(L\_i - L) D\_i}{1 + e(1-\alpha)}- \dfrac{cov(L\_i,D\_i)}{1 + e(1-\alpha)}.
\end{align\*}
$$
Then:
$$
\begin{align\*}
(y\_i - \bar y)(D\_i - \bar D) = \dfrac{\bar L}{1 + e(1-\alpha)}(D\_i - \bar D)^2 + \eta\_i(D\_i - D) + \dfrac{(L\_i - \bar L)D\_i(D\_i - \bar D)}{1 + e(1-\alpha)} - \dfrac{cov(L\_i, D\_i)}{1 + e(1-\alpha)}(D\_i - \bar D)
\end{align\*}
$$
So taking the expectation of both sides gives:
$$
cov(y\_i, D\_i) = \dfrac{\bar L}{1 + e(1-\alpha)}var(D\_i) + \dfrac{\mathbb{E}((L\_i - \bar L)(D\_i - \bar D)D\_i)}{1 + e(1-\alpha)}
$$
The last term gives:
$$
\begin{align\*}
&\dfrac{1}{1 + e(1-\alpha)}\left[\mathbb{E}(L\_i D\_i D\_i) - \mathbb{E}(\bar L D\_i D\_i) - \mathbb{E}(L\_i \bar D D\_i) + \mathbb{E}(\bar L \bar D D\_i) \right],\\
&=\dfrac{1}{1 + e(1-\alpha)}\left[\mathbb{E}(L\_i D\_i D\_i) - \bar L \mathbb{E}(D\_i D\_i) - \bar D \mathbb{E}(L\_i D\_i) + \bar L (\bar D)^2 \right],\\
&=\dfrac{1}{1 + e(1-\alpha)}\left[\mathbb{E}(L\_i (D\_i)^2) - \bar L \overline{D^2} - \bar D\left[\mathbb{E}(L\_i D\_i) - \bar L \bar D\right] \right],\\
&= \dfrac{1}{1 + e(1-\alpha)} \left[ cov(L\_i, (D\_i)^2) - \bar D cov(L\_i, D\_i)\right]
\end{align\*}
$$
So the OLS estimate will converge to:
$$
\dfrac{\bar L}{1 + e(1-\alpha)} + \dfrac{1}{1 + e(1-\alpha)} \dfrac{cov(L\_i, (D\_i)^2)}{var(D\_i)} - \bar D \dfrac{1}{1 + e(1-\alpha)} \dfrac{cov(L\_i, D\_i)}{var(D\_i)}.
$$
If $D\_i$ is a dummy variable then $(D\_i)^2 = D\_i$ so $cov(L\_i,(D\_i)^2) = cov(L\_i, D\_i)$ and $var(D\_i) = \bar D(1-\bar D)$ so, we get:
$$
\begin{align\*}
&\dfrac{\bar L}{1 + e(1-\alpha)} + \dfrac{1}{1 + e(1-\alpha)}(1 - \bar D)\dfrac{cov(L\_i, D\_i)}{\bar D(1-\bar D)},\\
&= \dfrac{\bar L}{1 + e(1-\alpha)} + \dfrac{1}{1 + e(1-\alpha)}\dfrac{cov(L\_i, D\_i)}{\bar D},
\end{align\*}
$$
If $cov(L\_i, D\_i) > 0$ then the OLS estimate will tend to overestimate $\dfrac{\bar L}{1 + e(1-\alpha)}$ if $cov(L\_i, D\_i) < 0$ then the OLS will underestimate this mean effect of $D$ on $y$.


---

