<p>The constant term in your final FE model has no specific meaning without further restrictions. For Stata, it is only chosen such that the (sample) mean of the estimated individual effects add up to 0. So your testing <span class="math-container">$\alpha&gt;0$</span> is in fact based on
<span class="math-container">$$\frac{1}{n} \sum_{i=1}^n \hat\alpha_i,\quad \hat\alpha_i = \frac{1}{T_i} \sum_{t=1}^{T_i} (x_{it} - y_{it}),$$</span>
where I am using the <span class="math-container">$T_i$</span> notation in order to allow for the panel data to be unbalanced.</p>

<p><strong>For balanced panel data</strong>, the test is (almost) identical to one based on OLS if you use a cluster standard error for OLS because then
<span class="math-container">$$\frac{1}{n} \sum_{i=1}^n \hat\alpha_i = \frac{1}{n} \sum_{i=1}^n \frac{1}{T} \sum_{t=1}^T (x_{it}-y_{it}) = \frac{1}{nT} \sum_{i=1}^n \sum_{t=1}^T (x_{it} - y_{it}),$$</span>
where the LHS is a statistic from the FE regression and the RHS is the corresponding one from the OLS regression. OLS, FE and RE (and BE) will all give an identical intercept estimate, I think. Only standard errors will be different. <strong>If the panel data set is unbalanced</strong>, the method using the FE regression and that using the OLS regression differ in how the average differences (<span class="math-container">$\hat\alpha_i$</span>) are weighted across different <span class="math-container">$i$</span>.</p>

<p>The test you mention is about whether the mean of <span class="math-container">$x_{it}-y_{it}$</span> is bigger than 0 or not. This looks OK. What else? Which technique to use (OLS or FE or RE or BE) doesn't seem to be critical here. I would just use OLS + cluster se, as it is simple to both the researcher and the reader. Try:</p>

<pre><code>xtset id year
gen d = x - y
reg d, vce(cl id)
xtreg d, fe
xtreg d, re
xtreg d, be
</code></pre>

CC BY-SA 4.0
2020-04-26T06:51:07.263Z
You are right about the interpretion of the intercept I think. I think I will simply go with the OLS model and the clustered SE, as you suggested. In a more detailed analysis, do you think it is possible to somehow incorporate the fact that not every value $x_{i,t}$ will differ from $y_{i,t}$ because not every data point that they are based on is adjusted? Because as far as I understand, my estimator of $\alpha$ is biased towards zero at the moment (i.e. if I dont incorporate that detail in my analysis).
shenflow
CC BY-SA 4.0
2020-04-26T15:41:58.450Z
If whether being adjusted is determined exogenously (nonrandom), then you can do the test using only adjusted observations. But that's the same as restricting the population to those adjusted. If you want to see the whole population, and if 99% don't adjust, then the test stat will be close to 0 naturally, but it does not mean a bias. Anyway, it's all a matter of how you _define_ your null hypothesis. Is it E(x)>E(y)? Then what's the meaning of E(), etc. Try to clarify that, before worrying about which techniques to use.
chan1142
CC BY-SA 4.0
2020-04-26T03:09:39.380Z
2020-04-26T03:09:39.380Z
chan1142
<p>I have two panels. One panel that consists of an economic indicator variable across firms and years. The second panel consists of the same economic indicator variable across the same firms and years. However, the indicator variable in the second panel is calculated on the basis of different data than the one in the first panel, I call that data "adjusted". Hence, the values of the variable in the second panel data differ from that in the first panel. Put differently, I have two panels of paired observations of a single variable.  </p>

<p>I now want to assess wether the indicators based on the adjusted data are <em>significantly</em> larger than the ones based on the unadjusted data. </p>

<p><strong>Or simply</strong>: <em>Are the values in one panel larger than the values in the other panel that describes an alternative set of values of the same variable for the same firms and years.</em> </p>

<p>One issue to keep in mind is that the indicator variables are autocorrelated by construction, since they are computed on a rolling (overlapping) window basis. What I have come up with is as follows:</p>

<p><strong>Step 1</strong>: I calculate the paired differences, i.e. <span class="math-container">$\delta_{i,t} = x_{i,t} - y_{i,t}$</span>, where <span class="math-container">$x_{i,t}$</span> is the economic indicator based on adjusted data and <span class="math-container">$y_{i,t}$</span> is the economic indicator based on unadjusted data.</p>

<p><strong>Step 2</strong>: I estimate a fixed effects model on a constant only, i.e. <span class="math-container">$\delta_{i,t} = \alpha + \gamma_{i} + \epsilon_{i,t}$</span>, where <span class="math-container">$\alpha$</span> is the constant and <span class="math-container">$\gamma_{i}$</span> is a firm fixed effect.</p>

<p>I test the hypothesis that <span class="math-container">$\alpha &gt; 0$</span> against the null hypothesis <span class="math-container">$\alpha = 0$</span> using the usual t-test. I employ robust standard errors to account for the autocorrelation that I expect in the <span class="math-container">$\delta_{i,t}$</span>. </p>

<p>If the <span class="math-container">$\alpha$</span> is significantly larger than <span class="math-container">$0$</span>, I assume that the <span class="math-container">$x_{i,t}$</span> is, on average, larger than <span class="math-container">$y_{i,t}$</span>.</p>

<p>I am not an expert in panel regression, so my question is: Is this model design reasonable or flawed? Moreover, are there "better" ways to assess my question of interest?</p>

<h2>--------------------------------------------------------------------------------</h2>

<p><strong><em>EDIT 1</em></strong>: What I mean by paired samples is that the observations are paired, i.e. they are somewhat linked because they refer to the same {i,t}. That is why I compute the differences of each "pair" (paired t-test).</p>

<p><strong><em>EDIT 2</em></strong>: I actually analyse the quality of reported earnings. The one panel consists of quality metrics that are calculated based on actual reported earnings (<span class="math-container">$y_{i,t}$</span>). The other panel consists of quality metrics that are based on "alternative" earnings values (<span class="math-container">$x_{i,t}$</span>). What I mean by that is: For example, one could hypothetically imagine a firm in the US applying different accounting standards (e.g. German accounting standards), which leads to different values of reported earnings of the same firm in the same period. Now simply imagine that the second panel are earnings quality metrics that are based on earnings quantities reported under a set of different accounting regulations than the one in the first panel, but for the same years and firms! </p>

<p><strong><em>EDIT 3</em></strong>: The panels consists of around 500 firms and 10 years each. I want to conduct the test for the entire sample. That is, I have 500 <span class="math-container">$\delta_{i,t}$</span> (500 firm-year differences).</p>

CC BY-SA 4.0
2020-04-25T11:03:15.957Z
I'm not sure what you mean by "panels are paired" and "adjusted versus unadjusted data". It might be useful to clarify it.
emeryville
CC BY-SA 4.0
2020-04-25T11:41:13.830Z
To devise a plan to solve the problem the following information is necessary: 1. How many countries do you have? 2. How many years do you have? 3. For many variables (y) do you want to do the test? Statistical power and p-value adjustments will be affected by all of the above.
Grada Gukovic
CC BY-SA 4.0
2020-04-25T11:44:03.523Z
In step 1 shouldn't it say $\delta_{i,j,t} = x_{i,jt} - y_{i,j,t}$. $i$ - country, $j$- variable, $t$ - year.
Grada Gukovic
CC BY-SA 4.0
2020-04-25T11:46:20.973Z
See the adjustments I just made to make it more clear. I made a mistake, its not countries I am referring to, its firms! So $i$ denots a firm and not a country - sorry.
shenflow
CC BY-SA 4.0
2020-04-25T11:46:24.157Z
@ EDIT 2 - the adjustments are not of statistical nature, but when you do multiple testing you have to adjust the p-value for the number of tests. The most appropriate method for controlling the family-wise error rate depends on the size of the data.
Grada Gukovic
CC BY-SA 4.0
2020-04-25T11:48:56.730Z
so you have only 1 variable (one y and one x)?
Grada Gukovic
CC BY-SA 4.0
2020-04-25T11:51:12.917Z
Yes I do only have one variable. More specifically, I have two "observations" of that single variable for each i and t. One that is based on unadjusted data and one that is based on adjusted data. That is how I end up with two panels that only consist of one variable for each i and t.
shenflow
CC BY-SA 4.0
2020-04-25T11:53:31.627Z
Regarding you comment in response to EDIT 2: I am aware of that, however, the number of adjustments per time series of a firm i is not linked to the number of comparisons I am making. I am only trying to test ONE single hypothesis, that is, I want to test wether the values in the panel based on adjusted data are larger than in the other panel, irrespective of how many adjustments have been made in the panel based on adjusted data (might be 1, might be 5000 aswell).
shenflow
CC BY-SA 4.0
2020-04-25T11:57:02.670Z
do you know when the adjustment occurs for each firm?
Grada Gukovic
CC BY-SA 4.0
2020-04-25T11:58:22.407Z
Yes I do, but there is no specific pattern. Firm 1 might adjust their earnings in period 2. Firm 2 might adjust their earnings in period 3 and 5 and so forth. MOREOVER, the quality metric is actually based on a rolling regression. So even if a firm adjust its earnings only in period 1, the actual value within the panel might differ for period 1,2,3,4,5 (if the window is of size 5). So I think it is easier to assume that the entire time series of a firm has been adjusted.
shenflow
CC BY-SA 4.0
2020-04-25T15:48:11.933Z
@GradaGukovic I edited the question. Now, it should be completely clear what the question is. I would appreciate feedback! Since it is a really pressing issue of mine, I will add a bounty as soon as I can and reward a proper answer with it.
shenflow
CC BY-SA 4.0
2020-04-25T15:48:28.573Z
@emeryville please see the changes I made to the question.
shenflow
CC BY-SA 4.0
2020-04-25T21:43:15.797Z
Ive created a chat room: https://chat.stackexchange.com/rooms/107200/model-design-fixed-effects-model-for-paired-differences
Grada Gukovic
<econometrics><regression><paneldata>
model design - fixed effects model for paired differences
shenflow
# model design - fixed effects model for paired differences

(Asked by: username_0 on 2020-04-25)

I have two panels. One panel that consists of an economic indicator variable across firms and years. The second panel consists of the same economic indicator variable across the same firms and years. However, the indicator variable in the second panel is calculated on the basis of different data than the one in the first panel, I call that data "adjusted". Hence, the values of the variable in the second panel data differ from that in the first panel. Put differently, I have two panels of paired observations of a single variable.

I now want to assess wether the indicators based on the adjusted data are *significantly* larger than the ones based on the unadjusted data.

**Or simply**: *Are the values in one panel larger than the values in the other panel that describes an alternative set of values of the same variable for the same firms and years.*

One issue to keep in mind is that the indicator variables are autocorrelated by construction, since they are computed on a rolling (overlapping) window basis. What I have come up with is as follows:

**Step 1**: I calculate the paired differences, i.e. $\delta\_{i,t} = x\_{i,t} - y\_{i,t}$, where $x\_{i,t}$ is the economic indicator based on adjusted data and $y\_{i,t}$ is the economic indicator based on unadjusted data.

**Step 2**: I estimate a fixed effects model on a constant only, i.e. $\delta\_{i,t} = \alpha + \gamma\_{i} + \epsilon\_{i,t}$, where $\alpha$ is the constant and $\gamma\_{i}$ is a firm fixed effect.

I test the hypothesis that $\alpha > 0$ against the null hypothesis $\alpha = 0$ using the usual t-test. I employ robust standard errors to account for the autocorrelation that I expect in the $\delta\_{i,t}$.

If the $\alpha$ is significantly larger than $0$, I assume that the $x\_{i,t}$ is, on average, larger than $y\_{i,t}$.

I am not an expert in panel regression, so my question is: Is this model design reasonable or flawed? Moreover, are there "better" ways to assess my question of interest?

## --------------------------------------------------------------------------------

***EDIT 1***: What I mean by paired samples is that the observations are paired, i.e. they are somewhat linked because they refer to the same {i,t}. That is why I compute the differences of each "pair" (paired t-test).

***EDIT 2***: I actually analyse the quality of reported earnings. The one panel consists of quality metrics that are calculated based on actual reported earnings ($y\_{i,t}$). The other panel consists of quality metrics that are based on "alternative" earnings values ($x\_{i,t}$). What I mean by that is: For example, one could hypothetically imagine a firm in the US applying different accounting standards (e.g. German accounting standards), which leads to different values of reported earnings of the same firm in the same period. Now simply imagine that the second panel are earnings quality metrics that are based on earnings quantities reported under a set of different accounting regulations than the one in the first panel, but for the same years and firms!

***EDIT 3***: The panels consists of around 500 firms and 10 years each. I want to conduct the test for the entire sample. That is, I have 500 $\delta\_{i,t}$ (500 firm-year differences).

**Tags:** econometrics, regression, paneldata
> **Comments:**
> **username_1**: I'm not sure what you mean by "panels are paired" and "adjusted versus unadjusted data". It might be useful to clarify it.  
> **username_2**: To devise a plan to solve the problem the following information is necessary: 1. How many countries do you have? 2. How many years do you have? 3. For many variables (y) do you want to do the test? Statistical power and p-value adjustments will be affected by all of the above.  
> **username_2**: In step 1 shouldn't it say $\delta_{i,j,t} = x_{i,jt} - y_{i,j,t}$. $i$ - country, $j$- variable, $t$ - year.  
> **username_0**: See the adjustments I just made to make it more clear. I made a mistake, its not countries I am referring to, its firms! So $i$ denots a firm and not a country - sorry.  
> **username_2**: @ EDIT 2 - the adjustments are not of statistical nature, but when you do multiple testing you have to adjust the p-value for the number of tests. The most appropriate method for controlling the family-wise error rate depends on the size of the data.  
> **username_2**: so you have only 1 variable (one y and one x)?  
> **username_0**: Yes I do only have one variable. More specifically, I have two "observations" of that single variable for each i and t. One that is based on unadjusted data and one that is based on adjusted data. That is how I end up with two panels that only consist of one variable for each i and t.  
> **username_0**: Regarding you comment in response to EDIT 2: I am aware of that, however, the number of adjustments per time series of a firm i is not linked to the number of comparisons I am making. I am only trying to test ONE single hypothesis, that is, I want to test wether the values in the panel based on adjusted data are larger than in the other panel, irrespective of how many adjustments have been made in the panel based on adjusted data (might be 1, might be 5000 aswell).  
> **username_2**: do you know when the adjustment occurs for each firm?  
> **username_0**: Yes I do, but there is no specific pattern. Firm 1 might adjust their earnings in period 2. Firm 2 might adjust their earnings in period 3 and 5 and so forth. MOREOVER, the quality metric is actually based on a rolling regression. So even if a firm adjust its earnings only in period 1, the actual value within the panel might differ for period 1,2,3,4,5 (if the window is of size 5). So I think it is easier to assume that the entire time series of a firm has been adjusted.  
> **username_0**: @GradaGukovic I edited the question. Now, it should be completely clear what the question is. I would appreciate feedback! Since it is a really pressing issue of mine, I will add a bounty as soon as I can and reward a proper answer with it.  
> **username_0**: @emeryville please see the changes I made to the question.  
> **username_2**: Ive created a chat room: https://chat.stackexchange.com/rooms/107200/model-design-fixed-effects-model-for-paired-differences  

---

## 1 answers

### Answer by username_3 on 2020-04-26. Score: 3 | Accepted answer

The constant term in your final FE model has no specific meaning without further restrictions. For Stata, it is only chosen such that the (sample) mean of the estimated individual effects add up to 0. So your testing $\alpha>0$ is in fact based on
$$\frac{1}{n} \sum_{i=1}^n \hat\alpha_i,\quad \hat\alpha_i = \frac{1}{T_i} \sum_{t=1}^{T_i} (x_{it} - y_{it}),$$
where I am using the $T_i$ notation in order to allow for the panel data to be unbalanced.
For balanced panel data, the test is (almost) identical to one based on OLS if you use a cluster standard error for OLS because then
$$\frac{1}{n} \sum_{i=1}^n \hat\alpha_i = \frac{1}{n} \sum_{i=1}^n \frac{1}{T} \sum_{t=1}^T (x_{it}-y_{it}) = \frac{1}{nT} \sum_{i=1}^n \sum_{t=1}^T (x_{it} - y_{it}),$$
where the LHS is a statistic from the FE regression and the RHS is the corresponding one from the OLS regression. OLS, FE and RE (and BE) will all give an identical intercept estimate, I think. Only standard errors will be different. If the panel data set is unbalanced, the method using the FE regression and that using the OLS regression differ in how the average differences ($\hat\alpha_i$) are weighted across different $i$.
The test you mention is about whether the mean of $x_{it}-y_{it}$ is bigger than 0 or not. This looks OK. What else? Which technique to use (OLS or FE or RE or BE) doesn't seem to be critical here. I would just use OLS + cluster se, as it is simple to both the researcher and the reader. Try:
xtset id year
gen d = x - y
reg d, vce(cl id)
xtreg d, fe
xtreg d, re
xtreg d, be

> **Comments:**
> **username_0**: You are right about the interpretion of the intercept I think. I think I will simply go with the OLS model and the clustered SE, as you suggested. In a more detailed analysis, do you think it is possible to somehow incorporate the fact that not every value $x_{i,t}$ will differ from $y_{i,t}$ because not every data point that they are based on is adjusted? Because as far as I understand, my estimator of $\alpha$ is biased towards zero at the moment (i.e. if I dont incorporate that detail in my analysis).  
> **username_3**: If whether being adjusted is determined exogenously (nonrandom), then you can do the test using only adjusted observations. But that's the same as restricting the population to those adjusted. If you want to see the whole population, and if 99% don't adjust, then the test stat will be close to 0 naturally, but it does not mean a bias. Anyway, it's all a matter of how you _define_ your null hypothesis. Is it E(x)>E(y)? Then what's the meaning of E(), etc. Try to clarify that, before worrying about which techniques to use.  

---

