<p>I am a last-year student at university, currently working on my Bachelor's (so still learning R), and I really hope that you would suggest a potential solution (even if to use Python).</p>
<p>So, the main model for my bachelor is as follows:
<a href="https://i.sstatic.net/l2eRU.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/l2eRU.png" alt="enter image description here" /></a></p>
<p>I need to estimate this model by using the GMM (preferably Arellano and Bond (1991)). I have been searching for the solution for almost 2 months, yet still not succeeded.</p>
<p>The data set could be found here:
<a href="https://drive.google.com/file/d/15NBcZ7TBfoJ7TnM3BYgtt5JW3T3CmqGJ/view?usp=sharing" rel="nofollow noreferrer">https://drive.google.com/file/d/15NBcZ7TBfoJ7TnM3BYgtt5JW3T3CmqGJ/view?usp=sharing</a></p>
<p>My code is the following:</p>
<pre><code>PDataA2 &lt;- pdata.frame(DataA2, index = c(&quot;ID&quot;,&quot;Year&quot;))

z1 &lt;- pgmm(domegaACF_A ~ lag(domegaACF_A, 1) + ddebt + ff1 + ff1:ddebt + Age + ta + dsales | lag(domegaACF_A, 2),
           data = PDataA2, effect = &quot;twoways&quot;, model = &quot;twosteps&quot;, index = c(&quot;ID&quot;, &quot;Year&quot;), transformation = &quot;ld&quot;)

summary(z1, robust = TRUE)

</code></pre>
<p>After I have created all the needed variables, I am trying to estimate coeffs by using <em>pgmm</em> function in <em>plm</em> package. And receive 2 errors:</p>
<p>In case I use <code>transformation = &quot;d&quot;</code>, errors are as follows:
<a href="https://i.sstatic.net/t0kcX.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/t0kcX.png" alt="enter image description here" /></a></p>
<p>In case I use <code>transformation = &quot;ld&quot;</code>, errors are as follows:
<a href="https://i.sstatic.net/QSSZc.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/QSSZc.png" alt="enter image description here" /></a></p>
<p>I would really appreciate any comments and suggestions provided because I do not know how to get away from this dead-end. Thank you in advance!)</p>
<p>UPD: I have created lags as separate variables, so here is a new correlation matrix:</p>
<pre><code>                  domegaACF_A        ddebt          ff1          Age          ta       dsales lag1_domegaACF_A lag2_domegaACF_A
domegaACF_A       1.000000000 -0.014777102 -0.002600866 -0.019160423  0.02456158  0.256801279     -0.379350157     -0.027687422
ddebt            -0.014777102  1.000000000  0.128264730  0.004706522  0.03878795  0.057488971      0.018800962     -0.003222902
ff1              -0.002600866  0.128264730  1.000000000 -0.048072279 -0.02868682  0.008979745     -0.002808377      0.008062733
Age              -0.019160423  0.004706522 -0.048072279  1.000000000  0.27884112 -0.021440539     -0.051829392     -0.048162046
ta                0.024561579  0.038787947 -0.028686815  0.278841116  1.00000000  0.078247618      0.015304990      0.015539599
dsales            0.256801279  0.057488971  0.008979745 -0.021440539  0.07824762  1.000000000     -0.049518811     -0.004134281
lag1_domegaACF_A -0.379350157  0.018800962 -0.002808377 -0.051829392  0.01530499 -0.049518811      1.000000000     -0.358671174
lag2_domegaACF_A -0.027687422 -0.003222902  0.008062733 -0.048162046  0.01553960 -0.004134281     -0.358671174      1.000000000
</code></pre>
<p>And I believe that correlation is not much trouble.</p>

CC BY-SA 4.0
2021-01-17T22:00:57.590Z
On the first look i would suggest that you look for perfect multicolinearity. Singularity can occur if one of the dependant variables is a linear combitnation of a other variable, like x1= 1000* x2. To control for this check correlation between independant variables.
Levittstyle
CC BY-SA 4.0
2021-01-18T12:03:40.147Z
@Armenthus, thank you for the response, I have checked the correlation between variables, and the biggest coefficient is 0.25. So, I believe there is no perfect multicollinearity. I do not know how to share the output with the correlation coefficients, but I am sure, that the reason is somewhere else. Still thank you for the thought, really appreciate that!
Wadim iLchuk
CC BY-SA 4.0
2021-01-19T02:22:48.617Z
`domega ~ lag(domega, 1) + ddebt + ff1 + ff1:ddebt + Age + ta + dsales | lag(domega, 2)` in this part of your specification, why is the lag put after the `|`? What is the use of it here?
Brennan
CC BY-SA 4.0
2021-01-19T12:19:00.590Z
@Brennan, I apologize for such a mass in the code, without comments. `domega` is TFP growth, `ddebt` - debt growth, `ta` - log of total assets, `dsales` - sales growth.

ff1, ff2, ff3, ff4, ff5 stands for financial frictions in the model. I put lag after `|` since it was an example in the `plm` package description. It means that, besides the first lag specified in the model, the estimator can also take the second lag. Without `|` sign, `pgmm` function says that a model without `|` is outdated.
Wadim iLchuk
CC BY-SA 4.0
2021-01-19T17:31:44.643Z
Hi: you might be getting the high correlation because of the lagging, particularly if the variable being lagged doesn't change much over time. If that's the case, ( check correlation of that variable and its lag )  don't use the lag and see if that helps.
mark leeds
CC BY-SA 4.0
2021-01-20T10:10:35.917Z
Well he told us the highest correlation between independent variables is 0.25 (this would naturallay include the lagged variables). Maybe you should add the whole correlation matrix into your answer.
Levittstyle
CC BY-SA 4.0
2021-01-20T17:29:43.743Z
@Armenthus: hopefully not but you're probably right.  In that case, the author of the R package is probably the best person to ask since the algorithm for computing the likelihood is usually quite complicated.
mark leeds
CC BY-SA 4.0
2021-01-21T14:20:29.083Z
@Armenthus, I have updated the question with a correlation matrix
Wadim iLchuk
CC BY-SA 4.0
2021-01-21T14:40:34.323Z
Well lag 1 and 2 from domegaAFC_A are missing. As @markleeds leeds already pointed out this is where the high correlation may come from
Levittstyle
CC BY-SA 4.0
2021-01-21T17:48:05.820Z
@Armenthus, check the updated correlation matrix
Wadim iLchuk
CC BY-SA 4.0
2021-01-21T17:51:23.420Z
Hi: lag1domegaACF looks pretty negative correlated with its non-lag and pretty negatively correlated with its second lag. I don't know what this means ( I don't even know what the variable is ) but it could be the cause of your problem. correlations of between predictor variables -0.4 are not small.
mark leeds
CC BY-SA 4.0
2021-01-21T18:01:09.387Z
@markleeds, my topic is about productivity, so omegaACF is Total Factor Productivity obtained with ACF method, so negative correlation is typical I believe. But I am a student, so I am also not quite sure.
Wadim iLchuk
CC BY-SA 4.0
2021-01-22T04:05:11.590Z
Hi: I'm not a student but not sure either. :). It doesn't seem correct to use an autocorrelation estimate as a predictor in a model but this ( panel models ) is not my field so hopefully someone can help. Generally, an acf estimate is going to be correlated with the estimate adjacent to it so  not surprising that you're getting large correlations. I think the more immediate issue is if such a predictor should be included in the model at all. By inclusion I mean, not just the lagged versions but the variable itself ? I don't know enough about panels or economics to say anything useful here.
mark leeds
CC BY-SA 4.0
2021-01-22T14:09:51.367Z
@markleeds, I have checked the example from the internet with EmplUK dataset, and there is a correlation coefficient of 0.8 between variables, so even higher than mine.
Wadim iLchuk
CC BY-SA 4.0
2021-01-22T17:34:38.947Z
but maybe the software they used is different so they may not obtain the same singular warnings. ?  -0.4 still seems high to me and, like I said, using the acf estimate as a predictor doesn't make sense ( atleast to me )  so I would take the acf predictor out and see what happens. Did the internet example also use that variable ?
mark leeds
CC BY-SA 4.0
2021-01-23T17:13:23.837Z
@markleeds, no the EmplUK dataset contains other variables. But the research paper I follow does not specify which instruments are GMM ones, so I was trying to use any other variables as GMM instruments, and still, I ended up with the same errors.
Wadim iLchuk
CC BY-SA 4.0
2021-01-24T19:10:14.230Z
hi: one thing I would do is start out small and then work your way to a larger number of variables sequentially. this way, you can tell better what variables are problematic.
mark leeds
CC BY-SA 4.0
2021-01-25T12:00:36.043Z
@markleeds, yes exactly I tried to do the same. However, I believe that the problem comes from the instruments - namely GMM instruments that are specified after `|` sign. I also tried to revise the source code of pgmm function. And with my current knowledge the problem is exactly in the section of computing the estimator with W variables - which are the instruments. Here is the code btw, https://github.com/cran/plm/blob/master/R/est_gmm.R try W[[i]] or crossprod(Wx, to find
Wadim iLchuk
CC BY-SA 4.0
2021-01-26T16:56:02.670Z
Wladim: If you've gone that far with the code, then I would explain to the author exactly what's happening and then see if he-she has anything useful to say. If I had more time, I wouldn't mind getting into it to understand panels more deeply but I don't have that type of time right now. Usually, package authors are pretty happy that people are using their package and are generous when asked to help with a problem regarding it.
mark leeds
CC BY-SA 4.0
2021-02-22T13:34:04.310Z
For a given firm there are many holes over the years. How are leaded and lagged values defined in this case? Are they replaced by 0? by NA? How many observations over N and T do you have when you restrict your sample to observations over contiguous years?
Bertrand
CC BY-SA 4.0
2021-03-05T15:49:21.937Z
@Bertrand, I tried several methods: the companies with missing lagged values are removed, substituted with NAs, 0, null. I leave only those companies that have at least 3 years of observation (I have read that for AB estimator it is enough), but also I tried leaving companies with at least 4,5,6 years of observations. Time period is 9 years, but I have not tried to filter firms with consequent reporting years, will try to do this. However, I believe the only solution in order not to spend a lot of time on that is Stata
Wadim iLchuk
<microeconomics><regression><productivity><r><firm>
Arellano and Bond (1991) GMM estimation in R by using plm package
Wadim iLchuk
# Arellano and Bond (1991) GMM estimation in R by using plm package

(Asked by: username_0 on 2021-01-17)

I am a last-year student at university, currently working on my Bachelor's (so still learning R), and I really hope that you would suggest a potential solution (even if to use Python).
So, the main model for my bachelor is as follows:

I need to estimate this model by using the GMM (preferably Arellano and Bond (1991)). I have been searching for the solution for almost 2 months, yet still not succeeded.
The data set could be found here:
https://drive.google.com/file/d/15NBcZ7TBfoJ7TnM3BYgtt5JW3T3CmqGJ/view?usp=sharing
My code is the following:
PDataA2 <- pdata.frame(DataA2, index = c("ID","Year"))

z1 <- pgmm(domegaACF_A ~ lag(domegaACF_A, 1) + ddebt + ff1 + ff1:ddebt + Age + ta + dsales | lag(domegaACF_A, 2),
           data = PDataA2, effect = "twoways", model = "twosteps", index = c("ID", "Year"), transformation = "ld")

summary(z1, robust = TRUE)


After I have created all the needed variables, I am trying to estimate coeffs by using pgmm function in plm package. And receive 2 errors:
In case I use transformation = "d", errors are as follows:

In case I use transformation = "ld", errors are as follows:

I would really appreciate any comments and suggestions provided because I do not know how to get away from this dead-end. Thank you in advance!)
UPD: I have created lags as separate variables, so here is a new correlation matrix:
                  domegaACF_A        ddebt          ff1          Age          ta       dsales lag1_domegaACF_A lag2_domegaACF_A
domegaACF_A       1.000000000 -0.014777102 -0.002600866 -0.019160423  0.02456158  0.256801279     -0.379350157     -0.027687422
ddebt            -0.014777102  1.000000000  0.128264730  0.004706522  0.03878795  0.057488971      0.018800962     -0.003222902
ff1              -0.002600866  0.128264730  1.000000000 -0.048072279 -0.02868682  0.008979745     -0.002808377      0.008062733
Age              -0.019160423  0.004706522 -0.048072279  1.000000000  0.27884112 -0.021440539     -0.051829392     -0.048162046
ta                0.024561579  0.038787947 -0.028686815  0.278841116  1.00000000  0.078247618      0.015304990      0.015539599
dsales            0.256801279  0.057488971  0.008979745 -0.021440539  0.07824762  1.000000000     -0.049518811     -0.004134281
lag1_domegaACF_A -0.379350157  0.018800962 -0.002808377 -0.051829392  0.01530499 -0.049518811      1.000000000     -0.358671174
lag2_domegaACF_A -0.027687422 -0.003222902  0.008062733 -0.048162046  0.01553960 -0.004134281     -0.358671174      1.000000000

And I believe that correlation is not much trouble.

**Tags:** microeconomics, regression, productivity, r, firm
> **Comments:**
> **username_1**: On the first look i would suggest that you look for perfect multicolinearity. Singularity can occur if one of the dependant variables is a linear combitnation of a other variable, like x1= 1000* x2. To control for this check correlation between independant variables.  
> **username_0**: @Armenthus, thank you for the response, I have checked the correlation between variables, and the biggest coefficient is 0.25. So, I believe there is no perfect multicollinearity. I do not know how to share the output with the correlation coefficients, but I am sure, that the reason is somewhere else. Still thank you for the thought, really appreciate that!  
> **username_2**: `domega ~ lag(domega, 1) + ddebt + ff1 + ff1:ddebt + Age + ta + dsales | lag(domega, 2)` in this part of your specification, why is the lag put after the `|`? What is the use of it here?  
> **username_0**: @Brennan, I apologize for such a mass in the code, without comments. `domega` is TFP growth, `ddebt` - debt growth, `ta` - log of total assets, `dsales` - sales growth.  ff1, ff2, ff3, ff4, ff5 stands for financial frictions in the model. I put lag after `|` since it was an example in the `plm` package description. It means that, besides the first lag specified in the model, the estimator can also take the second lag. Without `|` sign, `pgmm` function says that a model without `|` is outdated.  
> **username_3**: Hi: you might be getting the high correlation because of the lagging, particularly if the variable being lagged doesn't change much over time. If that's the case, ( check correlation of that variable and its lag )  don't use the lag and see if that helps.  
> **username_1**: Well he told us the highest correlation between independent variables is 0.25 (this would naturallay include the lagged variables). Maybe you should add the whole correlation matrix into your answer.  
> **username_3**: @Armenthus: hopefully not but you're probably right.  In that case, the author of the R package is probably the best person to ask since the algorithm for computing the likelihood is usually quite complicated.  
> **username_0**: @Armenthus, I have updated the question with a correlation matrix  
> **username_1**: Well lag 1 and 2 from domegaAFC_A are missing. As @markleeds leeds already pointed out this is where the high correlation may come from  
> **username_0**: @Armenthus, check the updated correlation matrix  
> **username_3**: Hi: lag1domegaACF looks pretty negative correlated with its non-lag and pretty negatively correlated with its second lag. I don't know what this means ( I don't even know what the variable is ) but it could be the cause of your problem. correlations of between predictor variables -0.4 are not small.  
> **username_0**: @markleeds, my topic is about productivity, so omegaACF is Total Factor Productivity obtained with ACF method, so negative correlation is typical I believe. But I am a student, so I am also not quite sure.  
> **username_3**: Hi: I'm not a student but not sure either. :). It doesn't seem correct to use an autocorrelation estimate as a predictor in a model but this ( panel models ) is not my field so hopefully someone can help. Generally, an acf estimate is going to be correlated with the estimate adjacent to it so  not surprising that you're getting large correlations. I think the more immediate issue is if such a predictor should be included in the model at all. By inclusion I mean, not just the lagged versions but the variable itself ? I don't know enough about panels or economics to say anything useful here.  
> **username_0**: @markleeds, I have checked the example from the internet with EmplUK dataset, and there is a correlation coefficient of 0.8 between variables, so even higher than mine.  
> **username_3**: but maybe the software they used is different so they may not obtain the same singular warnings. ?  -0.4 still seems high to me and, like I said, using the acf estimate as a predictor doesn't make sense ( atleast to me )  so I would take the acf predictor out and see what happens. Did the internet example also use that variable ?  
> **username_0**: @markleeds, no the EmplUK dataset contains other variables. But the research paper I follow does not specify which instruments are GMM ones, so I was trying to use any other variables as GMM instruments, and still, I ended up with the same errors.  
> **username_3**: hi: one thing I would do is start out small and then work your way to a larger number of variables sequentially. this way, you can tell better what variables are problematic.  
> **username_0**: @markleeds, yes exactly I tried to do the same. However, I believe that the problem comes from the instruments - namely GMM instruments that are specified after `|` sign. I also tried to revise the source code of pgmm function. And with my current knowledge the problem is exactly in the section of computing the estimator with W variables - which are the instruments. Here is the code btw, https://github.com/cran/plm/blob/master/R/est_gmm.R try W[[i]] or crossprod(Wx, to find  
> **username_3**: Wladim: If you've gone that far with the code, then I would explain to the author exactly what's happening and then see if he-she has anything useful to say. If I had more time, I wouldn't mind getting into it to understand panels more deeply but I don't have that type of time right now. Usually, package authors are pretty happy that people are using their package and are generous when asked to help with a problem regarding it.  
> **username_4**: For a given firm there are many holes over the years. How are leaded and lagged values defined in this case? Are they replaced by 0? by NA? How many observations over N and T do you have when you restrict your sample to observations over contiguous years?  
> **username_0**: @Bertrand, I tried several methods: the companies with missing lagged values are removed, substituted with NAs, 0, null. I leave only those companies that have at least 3 years of observation (I have read that for AB estimator it is enough), but also I tried leaving companies with at least 4,5,6 years of observations. Time period is 9 years, but I have not tried to filter firms with consequent reporting years, will try to do this. However, I believe the only solution in order not to spend a lot of time on that is Stata  

---

## 0 answers

