Singleton dummy variable

[Ariel Karlinsky]

Hi all,

Running xtivreg2 I get the foloowing error message regaring the Hansen's J statstic:

 
 Warning: estimated covariance matrix of moment conditions not of full rank.        model tests should be interpreted with caution. Possible causes: singleton dummy variable (dummy with one 1 and N-1 0s or vice versa) partial option may address problem.

My question is: How can I check which variables in which observations are singleton dummies? I was unable to find anything useful by googling this... Any help would be greatly appreciated

[Carlo Lazzaro]

Ariel:
can't you simply -describe- or -xttab- your variables?

[Ariel Karlinsky]

I'll try that (my access to that station is limited) - but my question is: What do I expect to see in xttab when a variable is a singleton dummy? I'm not quite sure about that. of course I have variables that do not vary by panel unit such as gender and variables that do not vary by time such as year fixed effects.

Also, this error occues in xtivreg2\xtoverid only when the vce(robust) option is requested. I googled it and found several threads regarding the matter, but I still don't understand what can cause this...

[Nick Cox]

Singleton dummies have sum 1. That can be approached in several ways. In your case you should know which of your variables are dummies, but if all else fails you can also get Stata to find them. Here findname needs to be downloaded first from theStata Journal site. (You won't need to use it if you can specify your dummies directly.)

. sysuse auto, clear
(1978 Automobile Data)

. gen foo = _n == 42

. tabstat price-foo, s(sum) col(stat)

    variable |       sum
-------------+----------
       price |    456229
         mpg |      1576
       rep78 |       235
    headroom |     221.5
       trunk |      1018
      weight |    223440
      length |     13907
        turn |      2934
displacement |     14600
  gear_ratio |     223.1
     foreign |        22
         foo |         1
------------------------

. findname , all(inlist(@, 0, 1, .))
foreign  foo

. tabstat `r(varlist)' , s(sum) col(stat)

    variable |       sum
-------------+----------
     foreign |        22
         foo |         1
------------------------

.

[Mark Schaffer]

The best discussion about this is an oldie but a goodie by Vince Wiggins:

http://www.stata.com/statalist/archi.../msg00594.html

[Ariel Karlinsky]

Thanks Nick! I do know which variables are dummies (most of them are virtual dummies from factor variables), so I used tabstat, sum(on e(sample)) like you show and I think I found the culprit.

Thanks for the link Mark, it explains brilliantly why this is happening.
Of course I especially liked the ending:

You might wonder why I always use the word "indicator" to describe binary
regressors, even when the original poster called them "dummy" variables. I was once briefing a group of mainly military personal about the implications of some model. About the third time I referred to the "colonel dummy" everyone at the table broke out in laughter, everyone that is except the older gentleman at the head of the table with lots of bars on his label and little bird emblems on his shoulders. I long ago forgot the subject of the talk, but I never forgot the lesson.

since the culprit is part of a FE and also on just 1 observation I will re-estimate by omitting the observation and also by omitting the indicator, things should stay pretty much the same.

Mark also suggested to partial out the variables i'm not interested in their coefficients. unfortunately xtoverid does not have a partial() option like ivreg2/xtivreg2 and so I might need to manaully partial these out. the issue is that I'm not really sure how to partial these out in a Random Effects model. Say my model is:
xtreg y (x1 = z1 z2) x2 x3 i.year, re
and I wish to partial out the time FE of i.year . in a non IV, non RE model I would just regress all of the variables (including y) on i.year, get residuals and then run a regression on all residuals.
I tried experimenting with this on my data but was unable to get results that seem to be correct, I suspect it's because the first stage in a RE model is some combination of BE and FE and this makes the usual way of partialling out somehow different.