Crash when using xtabond and/or xtdpdsys

[Thomas Mitterling]

Hey,

I try to estimate a dynamic panel using xtabond (and I've also tried xtdpdsys). Unfortunately Stata (and a few seconds later the whole computer) crashes without giving an error message. Looking at the task manager shows that Stata uses 10GB+ in a few seconds. Before running the commands I used xtset (in case the commands make problems if this hasn't been done). Any suggestions what could be wrong? Thanks in advance!

Best,
Thomas

[Hua Peng (StataCorp)]

I would suggest first to make sure your Stata is fully up to date by typing

update all

then follow the instruction. If that does not solve the problem, please contact Stata tech support at [email protected]

It will help to expedite the process if you can send us a do-file and a dataset which reproduces the crash along with the output from the following Stata commands:

about
query compilenumber
query memory

[Thomas Mitterling]

Thanks for the quick reply!

Stata is up to date. My current dataset is pretty large, so I'll try to replicate the problem with a smaller one and contact the support then.

[Thomas Mitterling]

I deleted observations from my (somewhat sparse, since a lot of data is missing) panel (from 58 to 10 cross-sections and from 326 to 127 time periods) and now the command works. I suspect that I have to limit the memory Stata tries to use?

edit:
New problem. I deleted nearly half of my observations (since the early dates are mainly missing values), Stata now never uses more than above 3,x GB, but now it seems like it gets stuck, since it runs for several minutes without any result.

[Sebastian Kripfganz]

xtabond and xtdpdsys are designed for panels with a short time dimension. 326 or 127 time periods are both much too large for this kind of estimation method. These GMM estimators create instruments that increase in their number quadratically in the number of time periods which implies that a huge number of instruments is created (which corresponds to huge matrices that Stata needs to create and invert). Even if Stata was able to do the computations (and maybe it is when you give it sufficient time), what you do would still not make any sense from an econometric perspective. You might want to do a brief search on the internet about "too many instruments".

[Thomas Mitterling]

Thanks! While Stata was running I read some stuff about these methods and that's what came to my mind too.

Seems like it didn't get stuck, but just needed more time. At least now I know what caused the problem (even though I don't really understand it, since the data I deleted was more or less empty anyway).

[Gustavo Sanchez (StataCorp)]

Hello Thomas,

I agree with what Sebastian has to say. To elaborate on his post:

- The excessive amount of processing time is associated to the extremely large matrix of instruments that arises from the number of periods in your estimation sample. The number of columns of the matrix of instruments is around k+(T-2)*(T-1)/2, where k is the number of exogenous variables and T is the number of periods. The default option for -xtabond- uses all the available time periods to construct the matrix of instrument, which is a very large number in your case leading to the huge matrix of instruments that is demanding a large amount of memory and processing time.

-xtabond- does have a couple of options that can be used to restrict the number of instruments (maxldep(#) and maxlags(#)); however, I would very cautious to use them with your panel structure. This leads to my second comment.

- The panel structure for your dataset does not seem to be compatible with the assumptions of the Arellano-Bond and Blundell-Bond/Arellano-Bover estimators. Those estimators are theoretically designed for a large number of panels and a small and fixed number of time periods (which does not seem to be your case). I would suggest that you have a look at section 8.1 in Baltagi (2013), which has a discussion that could be helpful for the selection of the estimator that you should use.

If you contact our tech-support group we may be able to provide additional hints on potential alternatives for your estimation.

[Thomas Mitterling]

Hey Gustavo,

thank you
I've already tried to limit the number of instruments. To be honest, I'm not that familiar with dynamic panels, it was more of a "I want a model to get some good estimates and I'll try this"-approach (I need some estimates in order to get forecasts for another "exercise").

That would be great, but I think it might be better (in the sense of more useful for other people) if I just open a new thread here?