xtreg - Statalist

Jeff Wooldridge

Join Date: Apr 2014

Posts: 2291
#16

10 Apr 2017, 14:32

Richard: Steve is certainly correct that you don't want to insert zero for the heterogeneity when using fixed effects logit, as the mean can be anything (and is not identified with small T). But in the CRE approach, provided one includes a constant, the heterogeneity is forced to have zero mean. So, we know zero is not a crazy value in that case. A key reason I don't prefer using xtlogit, fe is that the average marginal effects are not identified in general. Neither are the marginal effects at the average. Plus, xtlogit, fe requires serial independence in the innovations, which is very likely violated. The CRE approach allows any kind of serial correlation, provided the pooled method with "cluster(id)" is used.
Comment
Rose Simmons

Join Date: Feb 2017

Posts: 114
#17

10 Apr 2017, 14:53

Thank you Richard Williams and Jeff Wooldridge

in the CRE approach, provided one includes a constant, the heterogeneity is forced to have zero mean

Does the CRE approach removes endogeneity caused by unobserved heterogeneity?

A key reason I don't prefer using xtlogit, fe is that the average marginal effects are not identified in general.

Is there a test that can be used to justify using the CRE model for my data? As opposed to, for example, using -xtprobit, re- or -xtlogit, fe- or -xtlogit, re- ?

The CRE approach allows any kind of serial correlation, provided the pooled method with "cluster(id)" is used.

So, if I do -xtprobit, re vce(cluster id)- then this will account for any suspected serial correlation?

Many thanks
Comment
Jeff Wooldridge

Join Date: Apr 2014

Posts: 2291
#18

13 Apr 2017, 12:40

Rose: Subject to being okay with modeling the heterogeneity as a function of the covariates, and assuming a specific distribution -- homoskedastic normal -- the CRE approach accounts for "endogeneity" just like fixed effects logit.

The CRE approach can be applied by pooling, which I prefer (with clustering) or by using joint MLE -- which is xtprobit, re. Whether you use CRE or not, the random effects probit from xtprobit, re requires no serial correlation, and it can be badly biased if there is. Clustering simply gives you the proper standard errors for what might be a badly biased estimator. This is why I prefer pooling. So I didn't give you quite the correct answer before. The choices are

Code:

probit saving i.married marriedbar income incomebar risk riskbar i.female age i.year, vce(cluster id) margins, dydx(married income) xtprobit saving i.married marriedbar income incomebar risk riskbar i.female age i.year, re margins, dydx(married income) predict(pu0)

There are ways to get a somewhat better APE in the second case, but they're more complicated.
Comment

Rose Simmons

Join Date: Feb 2017
Posts: 114

#19

15 Apr 2017, 11:33

Thank you for your reply Jeff Wooldridge

I see. So with -xtprobit, re-, serial correlation would bias the estimator, and clustering would not resolve this bias.

If I wanted to test for serial correlation using the -xtprobit, re- CRE model, would this test be the correct way of doing so:

Code:

. xtprobit saving $xlist $controllist $xbarlist $controlbarlist i.year, re nolog

Random-effects probit regression                Number of obs     =      5,217
Group variable: hhid                            Number of groups  =      1,715

Random effects u_i ~ Gaussian                   Obs per group:
                                                              min =          1
                                                              avg =        3.0
                                                              max =         13

Integration method: mvaghermite                 Integration pts.  =         12

                                                Wald chi2(53)     =    1105.29
Log likelihood  = -2146.0342                    Prob > chi2       =     0.0000

--------------------------------------------------------------------------------
        saving |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
---------------+----------------------------------------------------------------
          prec |  -.0010931   .0131483    -0.08   0.934    -.0268633     .024677
      purchase |   -.026384   .0135935    -1.94   0.052    -.0530268    .0002587
        retire |   .0015985   .0112664     0.14   0.887    -.0204832    .0236802
       bequest |    .021824   .0109539     1.99   0.046     .0003547    .0432933
       mediumh |   .1020788   .0671701     1.52   0.129    -.0295722    .2337297
         longh |  -.0678172   .1821308    -0.37   0.710     -.424787    .2891527
          male |   .1687157   .0951233     1.77   0.076    -.0177226     .355154
           age |   -.005493   .0166836    -0.33   0.742    -.0381923    .0272063
               |
   c.age#c.age |   .0000783   .0001571     0.50   0.618    -.0002295    .0003862
               |
      incabove |   .9790311    .216036     4.53   0.000     .5556084    1.402454
      incbelow |  -.6898426   .1917699    -3.60   0.000    -1.065705   -.3139805
      employed |  -.1524566   .1785747    -0.85   0.393    -.5024565    .1975433
       retired |  -.1875233   .1907869    -0.98   0.326    -.5614588    .1864122
        health |   .0700966   .0698169     1.00   0.315     -.066742    .2069352
        income |   3.26e-06   1.42e-06     2.29   0.022     4.73e-07    6.04e-06
          risk |  -.0091811   .0075625    -1.21   0.225    -.0240032    .0056411
   selfcontrol |   .1566454    .030062     5.21   0.000      .097725    .2155658
         child |  -.1716487   .0803249    -2.14   0.033    -.3290826   -.0142148
    saving1exp |    1.13745   .0676842    16.81   0.000     1.004792    1.270109
       partner |  -.4209314   .2057002    -2.05   0.041    -.8240965   -.0177664
           uni |   2.155549   .7149156     3.02   0.003     .7543398    3.556758
         owner |  -.3056495   .2035796    -1.50   0.133    -.7046582    .0933593
       precbar |  -.0019713   .0203818    -0.10   0.923    -.0419188    .0379763
   purchasebar |   .0405058   .0196025     2.07   0.039     .0020856    .0789259
     retirebar |   .0100902   .0155144     0.65   0.515    -.0203175     .040498
    bequestbar |   -.028466   .0131068    -2.17   0.030    -.0541549   -.0027771
    mediumhbar |   .3150949   .1137132     2.77   0.006     .0922211    .5379687
      longhbar |   .4015058   .2998125     1.34   0.181     -.186116    .9891275
   incabovebar |  -.3513961   .4948465    -0.71   0.478    -1.321277    .6184852
   incbelowbar |   -.660045   .3147067    -2.10   0.036    -1.276859   -.0432312
   employedbar |    .051539   .2310028     0.22   0.823    -.4012182    .5042963
    retiredbar |   .1097085   .2357967     0.47   0.642    -.3524445    .5718616
     healthbar |  -.0388204   .0876784    -0.44   0.658    -.2106669    .1330262
     incomebar |  -2.57e-07   2.21e-06    -0.12   0.908    -4.59e-06    4.07e-06
       riskbar |   .0103123   .0096908     1.06   0.287    -.0086813     .029306
selfcontrolbar |   .1772171   .0417632     4.24   0.000     .0953628    .2590714
      childbar |   .1051414   .0897257     1.17   0.241    -.0707178    .2810006
 saving1expbar |   1.362141   .1109774    12.27   0.000      1.14463    1.579653
    partnerbar |   .3072452   .2209611     1.39   0.164    -.1258305    .7403209
        unibar |  -2.030746    .717485    -2.83   0.005    -3.436991   -.6245017
      ownerbar |   .3776915   .2175592     1.74   0.083    -.0487168    .8040997
               |
          year |
         2005  |  -.8472853   .1079302    -7.85   0.000    -1.058825    -.635746
         2006  |  -.9637202   .1120691    -8.60   0.000    -1.183372   -.7440688
         2007  |  -.9018653   .1139076    -7.92   0.000     -1.12512   -.6786105
         2008  |  -.8508618   .1084133    -7.85   0.000    -1.063348   -.6383757
         2009  |  -.8397674   .1080751    -7.77   0.000    -1.051591   -.6279441
         2010  |  -1.031509   .1105662    -9.33   0.000    -1.248215   -.8148032
         2011  |  -.9904344   .1386261    -7.14   0.000    -1.262137   -.7187322
         2012  |  -.8233528   .1383123    -5.95   0.000     -1.09444   -.5522657
         2013  |  -.9552002    .141568    -6.75   0.000    -1.232668   -.6777319
         2014  |  -1.002724   .1358045    -7.38   0.000    -1.268896   -.7365517
         2015  |  -.9422198   .1417819    -6.65   0.000    -1.220107   -.6643323
         2016  |  -1.101685   .1399005    -7.87   0.000    -1.375884   -.8274846
               |
         _cons |  -2.804562   .5909726    -4.75   0.000    -3.962847   -1.646277
---------------+----------------------------------------------------------------
      /lnsig2u |  -1.151784   .1759894                     -1.496717   -.8068511
---------------+----------------------------------------------------------------
       sigma_u |   .5622032   .0494709                      .4731426    .6680278
           rho |   .2401634   .0321154                      .1829157    .3085619
--------------------------------------------------------------------------------
LR test of rho=0: chibar2(01) = 80.80                  Prob >= chibar2 = 0.000

. xtserial saving prec purchase retire bequest mediumh longh male age incabove incbelow employed retir
> ed health income risk selfcontrol child saving1exp partner uni owner

Wooldridge test for autocorrelation in panel data
H0: no first-order autocorrelation
    F(  1,     525) =      3.079
           Prob > F =      0.0799

Question 1: As the p-value>0.05, would I conclude that there is no serial correlation present, so it is fine to use either "CRE Probit, MLE" or "CRE Probit, Pooled MLE"?
Question 2: And as there appears to be no serial correlation, I may use the "CRE Probit, MLE" clustered by id?
Question 3: Also, I have not used a pooled MLE before. Is pooled probit clustered by id equivalent to xtprobit, so it may be used with panel data?

Many thanks

Last edited by Rose Simmons; 15 Apr 2017, 11:46.

Comment

Rose Simmons

Join Date: Feb 2017
Posts: 114

#20

17 Apr 2017, 12:06

Hi Jeff Wooldridge

Below I have run the regression firstly as CRE Probit, Pooled MLE, and secondly as CRE Probit, MLE.
There are discrepancies between AMEs for both, as you alluded to in #19:

There are ways to get a somewhat better APE in the second case, but they're more complicated.

Code:

. probit saving $xlist $controllist $xbarlist $controlbarlist i.year, cluster(hhid)

Iteration 0:   log pseudolikelihood = -3595.0071  
Iteration 1:   log pseudolikelihood = -2253.1774  
Iteration 2:   log pseudolikelihood = -2238.3128  
Iteration 3:   log pseudolikelihood = -2238.2926  
Iteration 4:   log pseudolikelihood = -2238.2926  

Probit regression                               Number of obs     =      5,248
                                                Wald chi2(49)     =    1644.05
                                                Prob > chi2       =     0.0000
Log pseudolikelihood = -2238.2926               Pseudo R2         =     0.3774

                                 (Std. Err. adjusted for 1,721 clusters in hhid)
--------------------------------------------------------------------------------
               |               Robust
        saving |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
---------------+----------------------------------------------------------------
          prec |   .0020956   .0109996     0.19   0.849    -.0194632    .0236545
      purchase |  -.0165292   .0111011    -1.49   0.136    -.0382869    .0052285
        retire |    -.00003   .0098231    -0.00   0.998    -.0192828    .0192229
       bequest |   .0213176   .0094893     2.25   0.025      .002719    .0399162
       mediumh |   .1083739   .0549465     1.97   0.049     .0006806    .2160671
         longh |  -.0715218   .1481507    -0.48   0.629    -.3618919    .2188482
          male |   .1069454   .0867256     1.23   0.218    -.0630336    .2769244
           age |   .0016083   .0143292     0.11   0.911    -.0264763     .029693
               |
   c.age#c.age |    .000026   .0001317     0.20   0.843    -.0002322    .0002842
               |
      employed |  -.0457743   .1574617    -0.29   0.771    -.3543937     .262845
       retired |  -.1001639   .1700681    -0.59   0.556    -.4334912    .2331635
        health |   .0540725    .062706     0.86   0.389    -.0688289    .1769739
        income |   3.28e-06   1.34e-06     2.46   0.014     6.63e-07    5.90e-06
          risk |  -.0084715   .0063633    -1.33   0.183    -.0209433    .0040002
   selfcontrol |   .1429765   .0262985     5.44   0.000     .0914324    .1945206
         child |  -.1908073   .0820995    -2.32   0.020    -.3517194   -.0298951
    saving1exp |   1.000168    .060082    16.65   0.000     .8824095    1.117926
       partner |  -.3371603   .1672952    -2.02   0.044    -.6650529   -.0092678
           uni |   1.741463   .5838617     2.98   0.003     .5971149    2.885811
         owner |  -.2333272   .1705377    -1.37   0.171    -.5675749    .1009205
       precbar |   .0008937   .0168922     0.05   0.958    -.0322145    .0340019
   purchasebar |   .0298694   .0163961     1.82   0.068    -.0022663    .0620051
     retirebar |   .0059327   .0130191     0.46   0.649    -.0195842    .0314496
    bequestbar |  -.0285628    .011298    -2.53   0.011    -.0507065   -.0064192
    mediumhbar |    .225546   .0986669     2.29   0.022     .0321624    .4189297
      longhbar |     .51408   .2972899     1.73   0.084    -.0685976    1.096758
   employedbar |   .0582963   .1988377     0.29   0.769    -.3314183     .448011
    retiredbar |   .1180584    .201384     0.59   0.558     -.276647    .5127638
     healthbar |  -.0239702   .0735509    -0.33   0.745    -.1681274     .120187
     incomebar |   1.06e-08   2.47e-06     0.00   0.997    -4.82e-06    4.85e-06
       riskbar |   .0097983   .0081572     1.20   0.230    -.0061895    .0257861
selfcontrolbar |    .132431   .0366494     3.61   0.000     .0605996    .2042625
      childbar |   .1516533   .0920436     1.65   0.099    -.0287489    .3320555
 saving1expbar |   1.292793   .0969167    13.34   0.000      1.10284    1.482747
    partnerbar |   .2700214   .1807993     1.49   0.135    -.0843387    .6243814
        unibar |  -1.658964   .5865876    -2.83   0.005    -2.808655   -.5092738
      ownerbar |   .3172171   .1855677     1.71   0.087    -.0464889     .680923
               |
          year |
         2005  |  -.7703029   .0869453    -8.86   0.000    -.9407126   -.5998932
         2006  |   -.846976   .0966694    -8.76   0.000    -1.036445   -.6575074
         2007  |  -.8256925   .0950997    -8.68   0.000    -1.012085   -.6393004
         2008  |  -.7703135   .0930896    -8.27   0.000    -.9527658   -.5878612
         2009  |  -.7504317   .0948202    -7.91   0.000    -.9362759   -.5645874
         2010  |  -.9193557   .0960152    -9.58   0.000    -1.107542   -.7311694
         2011  |  -.9188734   .1182567    -7.77   0.000    -1.150652   -.6870945
         2012  |  -.7463333   .1179684    -6.33   0.000    -.9775471   -.5151195
         2013  |  -.9001852   .1300323    -6.92   0.000    -1.155044   -.6453265
         2014  |  -.9538584   .1146882    -8.32   0.000    -1.178643   -.7290736
         2015  |  -.8923384   .1231458    -7.25   0.000      -1.1337   -.6509771
         2016  |  -1.006153   .1205799    -8.34   0.000    -1.242485    -.769821
               |
         _cons |  -2.767711   .5090417    -5.44   0.000    -3.765415   -1.770008
--------------------------------------------------------------------------------

. 
. estimates store pooledprobitCREmodel

. margins, dydx($xlist)

Average marginal effects                        Number of obs     =      5,248
Model VCE    : Robust

Expression   : Pr(saving), predict()
dy/dx w.r.t. : prec purchase retire bequest mediumh longh

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        prec |   .0004999   .0026242     0.19   0.849    -.0046434    .0056433
    purchase |  -.0039433   .0026418    -1.49   0.136    -.0091211    .0012345
      retire |  -7.15e-06   .0023435    -0.00   0.998    -.0046002    .0045859
     bequest |   .0050857   .0022607     2.25   0.024     .0006547    .0095166
     mediumh |   .0258543   .0130912     1.97   0.048      .000196    .0515127
       longh |  -.0170627    .035368    -0.48   0.629    -.0863826    .0522573
------------------------------------------------------------------------------

. xtprobit saving $xlist $controllist $xbarlist $controlbarlist i.year, re nolog

Random-effects probit regression                Number of obs     =      5,248
Group variable: hhid                            Number of groups  =      1,721

Random effects u_i ~ Gaussian                   Obs per group:
                                                              min =          1
                                                              avg =        3.0
                                                              max =         13

Integration method: mvaghermite                 Integration pts.  =         12

                                                Wald chi2(49)     =    1114.11
Log likelihood  = -2193.3799                    Prob > chi2       =     0.0000

--------------------------------------------------------------------------------
        saving |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
---------------+----------------------------------------------------------------
          prec |  -.0003073   .0130227    -0.02   0.981    -.0258313    .0252166
      purchase |  -.0256225   .0134264    -1.91   0.056    -.0519378    .0006928
        retire |   .0008218   .0111472     0.07   0.941    -.0210263    .0226699
       bequest |   .0232112   .0107606     2.16   0.031     .0021208    .0443015
       mediumh |   .1056747   .0665145     1.59   0.112    -.0246914    .2360408
         longh |  -.0565685   .1807123    -0.31   0.754    -.4107581    .2976211
          male |   .1644311    .094838     1.73   0.083    -.0214479    .3503101
           age |  -.0035417   .0165184    -0.21   0.830    -.0359171    .0288337
               |
   c.age#c.age |   .0000682   .0001552     0.44   0.660    -.0002359    .0003724
               |
      employed |  -.0800609   .1727371    -0.46   0.643    -.4186193    .2584976
       retired |  -.0870453   .1861375    -0.47   0.640    -.4518681    .2777775
        health |   .0693955   .0685963     1.01   0.312    -.0650508    .2038418
        income |   3.49e-06   1.41e-06     2.48   0.013     7.26e-07    6.24e-06
          risk |  -.0087037   .0074777    -1.16   0.244    -.0233596    .0059523
   selfcontrol |   .1569987   .0296503     5.30   0.000     .0988851    .2151123
         child |  -.1715351   .0795586    -2.16   0.031    -.3274671   -.0156031
    saving1exp |    1.14052   .0669629    17.03   0.000     1.009275    1.271765
       partner |  -.4323744   .2024286    -2.14   0.033    -.8291273   -.0356216
           uni |   2.087761   .7122471     2.93   0.003      .691782    3.483739
         owner |  -.2856642   .2007181    -1.42   0.155    -.6790644     .107736
       precbar |   .0031423   .0202085     0.16   0.876    -.0364657    .0427503
   purchasebar |   .0383723   .0194416     1.97   0.048     .0002675    .0764772
     retirebar |   .0046464   .0154117     0.30   0.763      -.02556    .0348529
    bequestbar |  -.0309332   .0129303    -2.39   0.017    -.0562762   -.0055902
    mediumhbar |   .2929432   .1131135     2.59   0.010     .0712448    .5146416
      longhbar |    .396319   .2992194     1.32   0.185    -.1901402    .9827782
   employedbar |   .0791057   .2238125     0.35   0.724    -.3595586    .5177701
    retiredbar |   .0894296   .2303285     0.39   0.698     -.362006    .5408652
     healthbar |  -.0237382   .0863624    -0.27   0.783    -.1930054    .1455291
     incomebar |   1.55e-07   2.18e-06     0.07   0.943    -4.12e-06    4.43e-06
       riskbar |    .010057   .0096174     1.05   0.296    -.0087928    .0289068
selfcontrolbar |    .174106   .0414547     4.20   0.000     .0928563    .2553558
      childbar |   .1068124   .0889696     1.20   0.230    -.0675648    .2811895
 saving1expbar |   1.373991   .1101322    12.48   0.000     1.158136    1.589846
    partnerbar |   .3305887   .2179788     1.52   0.129    -.0966419    .7578193
        unibar |  -1.985154   .7148522    -2.78   0.005    -3.386238   -.5840694
      ownerbar |   .3962089   .2148184     1.84   0.065    -.0248274    .8172452
               |
          year |
         2005  |  -.8451907   .1069296    -7.90   0.000    -1.054769   -.6356125
         2006  |    -.95204   .1111817    -8.56   0.000    -1.169952   -.7341279
         2007  |  -.9037048   .1127384    -8.02   0.000    -1.124668   -.6827416
         2008  |  -.8537038   .1074086    -7.95   0.000    -1.064221   -.6431867
         2009  |   -.836815   .1069047    -7.83   0.000    -1.046344   -.6272856
         2010  |  -1.026241   .1094783    -9.37   0.000    -1.240815   -.8116678
         2011  |   -.984347   .1373051    -7.17   0.000     -1.25346    -.715234
         2012  |  -.8343642   .1372978    -6.08   0.000    -1.103463   -.5652655
         2013  |  -.9757297   .1397632    -6.98   0.000     -1.24966   -.7017989
         2014  |  -1.036579   .1347487    -7.69   0.000    -1.300681    -.772476
         2015  |  -.9736545   .1403654    -6.94   0.000    -1.248766   -.6985433
         2016  |  -1.082008   .1383626    -7.82   0.000    -1.353194   -.8108225
               |
         _cons |  -3.082254   .5882316    -5.24   0.000    -4.235167   -1.929341
---------------+----------------------------------------------------------------
      /lnsig2u |   -1.11613   .1692543                     -1.447863    -.784398
---------------+----------------------------------------------------------------
       sigma_u |   .5723153   .0484334                      .4848424    .6755696
           rho |   .2467298   .0314566                      .1903307    .3133728
--------------------------------------------------------------------------------
LR test of rho=0: chibar2(01) = 89.83                  Prob >= chibar2 = 0.000

. 
. estimates store probitCREmodel

. margins, dydx($xlist) predict(pu0)

Average marginal effects                        Number of obs     =      5,248
Model VCE    : OIM

Expression   : Pr(saving=1 | u_i=0), predict(pu0)
dy/dx w.r.t. : prec purchase retire bequest mediumh longh

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        prec |  -.0000665   .0028195    -0.02   0.981    -.0055926    .0054596
    purchase |  -.0055475   .0029012    -1.91   0.056    -.0112338    .0001388
      retire |   .0001779   .0024134     0.07   0.941    -.0045523    .0049082
     bequest |   .0050254   .0023278     2.16   0.031      .000463    .0095878
     mediumh |   .0228794   .0144001     1.59   0.112    -.0053443    .0511031
       longh |  -.0122475   .0391267    -0.31   0.754    -.0889345    .0644395
------------------------------------------------------------------------------

Are the AMEs for the CRE Probit, Pooled MLE the correct ones?
And is there a statistical reason why the AMEs in the second regression are not the same as the first one?

Many Thanks

Comment

Taulant Yzellari

Join Date: May 2022

Posts: 5
#21

13 May 2022, 04:29

Hello Everyone, Could someone tell me if it is possible to estimate this panel model with xtreg Yi,t = A + A(L)Yi,t + ei,t
Thank you in advance
Comment

Announcement

Comment

Comment

Comment

Comment

Comment

Comment