Very strange: same STATA versions produce different results?!

[Peter Winter]

Hi,

my colleague and I use the same STATA version (14). This time, my colleague has prepared some estimations (GMM) in the .do file that produce one set of results, when I run the same file, STATA won't let me. The error I get is:

variance-covariance matrix of the two-step estimator is not full rank
Two-step estimator is not available. One-step estimator is available and variance-covariance matrix provides
correct coverage.

I tried to google this issue but apparently, no one has asked about this error before. Essentially, STATA won't let me continue unless I remove the "twostep" option from the GMM command. But then, the results are vastly different.

Does anyone have a solution? I am using the same data set and the same do file as my colleague is. It is something in the way STATA handles the data.

Thanks!

P.S.: just to rule that one out - we both have original and licensed STATA versions

[William Lisowski]

Welcome to Statalist.

Identifying your Stata version as 14 isn't enough to ensure you are running the same code. Compare the results of the about command on both your systems and ensure they match. Better yet, make sure you are both fully up-to-date.

. about

Stata/SE 14.2 for Mac (64-bit Intel)
Revision 04 May 2017
Copyright 1985-2015 StataCorp LLC

[Jorrit Gosens]

Did you try comparing gmm versions? Type:

which gmm

[Carlo Lazzaro]

Peter:
welcome to the list.
Are your copies of Stata fully updated?
Just type -update query- from within Stata and compare the messages Stata gives you.

[Peter Winter]

My Stata has the most recent updates installed. My gmm version is: " version 2.1.3 10apr2017"

I also don't have problems with GMM in general, only with the twostep procedure. My colleague checked her onestep estimations and they are identical to my onestep estimations. When I run the twostep estimations, Stata won't let me giving me the aforementioned error. My colleague tried the code on three separate computers and it has worked. Is there a way to 'reset' the GMM protocol within Stata? I don't have my Stata cd with me, otherwise I'd try to reinstall it.

[Carlo Lazzaro]

Peter:
it seems a matter for Stata Tech Support (http://www.stata.com/support/faqs/te...-tech-support/).

[Peter Winter]

I just installed Stata v13 and there the code worked without problems.
So what now? Does this mean that Stata has potentially made changes that may render the 'old' results inadequate? Do I keep using the older version?
I cannot even find an explanation online to the error that Stata 14 gives me.

The tech support has not been useful so far, unfortunately.

[Bruce Weaver]

Can you share the (anonymized) data and code so that others can give it a try?

[Peter Winter]

@Bruce: at this point, I'm not sure whether I'm allowed to publicly share the data, so I'll try to find a workaround for now.

I'm coming closer to the issue: it is caused by one of the control variables that are being used as lagged controls. If I remove this variable, everything runs smoothly. About the variable:
- type: long
- 6-7 digit numbers
- there are some missings (but so have other variables that work perfectly fine in the estimations)
- the estimation command being used is xtdpdsys


Any ideas? Please recall that this is neither an issue in my old Stata nor on the computer of my colleague. So it must be specific to this variable and my computer.

[William Lisowski]

Now that we know the command you are using is not gmm but xtdpdsys, help whatsnew has some relevant information.

update 23jun2015

4. xtabond, xtdpd, and xtdpdsys with option twostep reported standard errors that provided poor coverage when the second-step GMM weighting matrix was not full rank. For example, this may have happened when the number of instruments was too large relative to the number of panels. When the weighting matrix is not full rank, these commands now issue an error message, and the user is advised to use the default one-step estimates.

So the results you are getting with the fully-updated version of Stata 14 are probably the appropriate results. You have neglected to mention what version of Stata your colleague is using, but presumably it is some version before the 23jun2015 update to Stata 14 - perhaps it is Stata 13, since you have chosen to attempt your program in that version.

[Peter Winter]

Now that we know the command you are using is not gmm but xtdpdsys, help whatsnew has some relevant information.



So the results you are getting with the fully-updated version of Stata 14 are probably the appropriate results. You have neglected to mention what version of Stata your colleague is using, but presumably it is some version before the 23jun2015 update to Stata 14 - perhaps it is Stata 13, since you have chosen to attempt your program in that version.

Thank you, this was helpful!
In the meantime, I found out that adjusting the effect size of this variable solves the problem. In the regression tables that my colleague provided, the effect sizes were 0.0000 with standard errors of 0.0000 - which obviously does not allow for meaningful interpretation. I divided the values of this variable by 1000 and now everything works just fine. It changes the interpretation while also solving the Stata problem. Would it make sense that this is also a potential reason why this error might come up?

[William Lisowski]

I cannot speak to whether or not what you are doing is likely to be the cause of the problem. All I know is what I reported in post #10: that StataCorp made changes in xtdpdsys that under certain circumstances cause it to issue the error message you received where beforehand it had not.

So, a few questions now for you, to benefit those participating in or following this thread, as well as those who find it later searching for information about, say, their problem with xtdpdsys.

a) Is it correct that with no changes other than scaling your independent variable you were able to successfully run the same data (other than that scaling) and the same do-file in your up-to-date copy of Stata 14 that you previously were unable to run?

b) How do your results with the scaled variable compare to those from your colleague's results from her version of Stata? That is, we expect the estimates regarding that variable to change by 3 orders of magnitude; what happened to other estimates, and to the overall measures reported by xtdpdsys?

c) What version of Stata did your colleague use to produce her results?

[Peter Winter]

I cannot speak to whether or not what you are doing is likely to be the cause of the problem. All I know is what I reported in post #10: that StataCorp made changes in xtdpdsys that under certain circumstances cause it to issue the error message you received where beforehand it had not.

So, a few questions now for you, to benefit those participating in or following this thread, as well as those who find it later searching for information about, say, their problem with xtdpdsys.

a) Is it correct that with no changes other than scaling your independent variable you were able to successfully run the same data (other than that scaling) and the same do-file in your up-to-date copy of Stata 14 that you previously were unable to run?

b) How do your results with the scaled variable compare to those from your colleague's results from her version of Stata? That is, we expect the estimates regarding that variable to change by 3 orders of magnitude; what happened to other estimates, and to the overall measures reported by xtdpdsys?

c) What version of Stata did your colleague use to produce her results?

a) Yes
b) All results remain the same, only the scaled variable changes (as expected)
c) don't know yet - I will update this as soon as I'll receive the information

[Peter Winter]

a) Yes
b) All results remain the same, only the scaled variable changes (as expected)
c) don't know yet - I will update this as soon as I'll receive the information

Because I cannot edit my old post:

a) Yes
b) All results remain the same, only the scaled variable changes (as expected)
c) V13, but the newest updates were installed. Hence, it is likely a difference between V14 and V13 and not so much the recurring updates.

[William Lisowski]

Thank you for the followup.

The output of help whatsnew13 tells us that the updates to Stata 13 did not include any updates to xtdpdsys, so the different outcomes were due to the 23jun2015 update to Stata 14 that changed the functioning of xtdpdsys.

This does not explain why xtdpdsys is sensitive to the scaling of the data in the way you observed.