margins fails after xtreg, be with wls option

[George Hoffman]

scenario: large unbalanced panel dataset, multiple measures repeated at hourly intervals in hundreds of patients, with many potential grouping variables.
problem: group means reported by oneway or ols do not equal group means after xtreg, be and margins (because of imbalance). this was not unexpected, but seemed unintuitive to some reviewers.
attempted solution: use xtreg, be with wls option to weight individual means to replicate ols coefficients but maintain appropriate variance analysis.
problem encountered: margins command doess not work after xtreg,be with wls option.

this can be replicated using the dataset posted by Prof Michael Oakes, available at

https://drive.google.com/file/d/0B4YCIvR8blHtMFI0dEI5cjF4SHM/edit?usp=sharing

stata code:

use pre_post_c_anal.dta, clear

. xtset id time
       panel variable:  id (unbalanced)
        time variable:  time, 0 to 1
                delta:  1 unit

. xtsum math

Variable         |      Mean   Std. Dev.       Min        Max |    Observations
-----------------+--------------------------------------------+----------------
math     overall |  463.6777   73.92396        278        642 |     N =     602
         between |             51.29837        331        642 |     n =     337
         within  |             56.90244   343.6777   583.6777 | T-bar = 1.78635

. oneway math gender,tab

            |           Summary of math
     gender |        Mean   Std. Dev.       Freq.
------------+------------------------------------
          0 |      462.74       73.23         257
          1 |      464.38       74.53         345
------------+------------------------------------
      Total |      463.68       73.92         602

                        Analysis of Variance
    Source              SS         df      MS            F     Prob > F
------------------------------------------------------------------------
Between groups      394.934161      1   394.934161      0.07     0.7883
 Within groups      3283920.55    600   5473.20091
------------------------------------------------------------------------
    Total           3284315.48    601   5464.75122

Bartlett's test for equal variances:  chi2(1) =   0.0907  Prob>chi2 = 0.763

. xtreg math gender,be

Between regression (regression on group means)  Number of obs     =        602
Group variable: id                              Number of groups  =        337

R-sq:                                           Obs per group:
     within  =      .                                         min =          1
     between = 0.0024                                         avg =        1.8
     overall = 0.0001                                         max =          2

                                                F(1,335)          =       0.80
sd(u_i + avg(e_i.))=  51.31379                  Prob > F          =     0.3723

------------------------------------------------------------------------------
        math |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      gender |   5.035392   5.636553     0.89   0.372    -6.052105    16.12289
       _cons |   463.2041    4.23229   109.45   0.000     454.8789    471.5293
------------------------------------------------------------------------------

. margins, by(gender)

Predictive margins                              Number of obs     =        602
Model VCE    : Conventional

Expression   : Linear prediction, predict()
over         : gender

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      gender |
          0  |   463.2041    4.23229   109.45   0.000     454.9089    471.4992
          1  |   468.2395   3.722694   125.78   0.000     460.9431    475.5358
------------------------------------------------------------------------------

. xtreg math gender,be wls

Between regression (regression on group means)  Number of obs     =        602
Group variable: id                              Number of groups  =        337

R-sq:                                           Obs per group:
     within  =      .                                         min =          1
     between = 0.0003                                         avg =        1.8
     overall = 0.0001                                         max =          2

                                                F(1,335)          =       0.10
sd(u_i + avg(e_i.))=    47.284                  Prob > F          =     0.7534

------------------------------------------------------------------------------
        math |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      gender |   1.637512   5.207385     0.31   0.753    -8.605782    11.88081
       _cons |   462.7393    3.94213   117.38   0.000     454.9849    470.4937
------------------------------------------------------------------------------

. margins, by(gender)
__000005 not found
r(111);

.

two questions:
1. would you summarize the between-gender math score difference as 1.64 (ols/ be wls estimate) or 5.04 (xtreg between effect)?
2. why does the margins commadn fail after the wls option for xtreg, be?

thank you

[Ariel Karlinsky]

Seems to be some sort of bug in stata. I'm able to replicate it by using one of the example datasets:

webuse nlswork
xtset idcode
generate age2 = age^2
generate ttl_exp2 = ttl_exp^2
 generate tenure2 = tenure^2
 generate byte black = (race==2)
xtreg ln_w grade age* ttl_exp* tenure* black not_smsa south, be
margins, by(black)
xtreg ln_w grade age* ttl_exp* tenure* black not_smsa south, be wls
margins, by(black)

margins after be without wls works as expected. margins after be with wls gives me the same error it gave you of some temporary variable name:

__000005 not found

[Rafal Raciborski (StataCorp)]

George discovered a bug in xtreg. We will have it fixed in a future update, meanwhile here is a workaround,

xtreg math gender, be wls
margins [fw=1], by(gender)

[George Hoffman]

thank you for confirming and for immediatley addressing. !