Are dynamic models really feasible tool in Stata??

[Rim Achour]

Hello,

After a month of work I have started to doubt the validity of using dynamic panel approach in Stata and even any software. I discovered that the generated results can vary tremendously depending on your lag assumptions and that the results in general are chaotic.
My understanding is that dynamic models shouldn't lead to huge differences in results from OLS and Fixed Effects models especially that I don't have a small sample issue.
My results for Fixed Effects and Pooled OLs are very powerful, they suit well my hypotheses and are in accordance with the theory. But when I applied the regressions in a dynamic model setting using xtabond2 (theoretically dynamic are the ones to be used when the dependent variable is explained by its value since the the lagged dependent is correlated with the error term). I have been trying for one month to generate feasible results, however I am receiving completely different coefficients that are very far from the OLS and Fixed Effects results and are not theoretically justified. This convinced me that either Dynamic panel results are chaotic and arbitrary or that I am overlooking a simple procedure to generate results that make sense.

Here are my results for the OLS and Fixed Effects regressions:

. //pooled OLS 
. regress roeavgw L.roeavgw lconw LConNegavgw lcoffw LCoffNegavgw DLL DLLNegavgw carw CARNegavgw Drg DrgNegavgw costincomew
>  costincomeNegavg eqcgtaw revdivw sizeassetsw gdpg gap hh inflation i.year

      Source |       SS           df       MS      Number of obs   =   256,215
-------------+----------------------------------   F(32, 256182)   =  16321.62
       Model |  6949199.83        32  217162.495   Prob > F        =    0.0000
    Residual |  3408554.45   256,182  13.3052067   R-squared       =    0.6709
-------------+----------------------------------   Adj R-squared   =    0.6709
       Total |  10357754.3   256,214  40.4261839   Root MSE        =    3.6476

-----------------------------------------------------------------------------------
          roeavgw |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
------------------+----------------------------------------------------------------
          roeavgw |
              L1. |   .3971417   .0015851   250.55   0.000     .3940349    .4002484
                  |
            lconw |   .0274215   .0006878    39.87   0.000     .0260734    .0287695
      LConNegavgw |   .0179526   .0022544     7.96   0.000     .0135341    .0223712
           lcoffw |  -.0154726   .0027088    -5.71   0.000    -.0207818   -.0101634
     LCoffNegavgw |   .1270667   .0079808    15.92   0.000     .1114245    .1427088
              DLL |  -.0168602   .0006222   -27.10   0.000    -.0180797   -.0156408
       DLLNegavgw |   -.113601   .0010032  -113.24   0.000    -.1155673   -.1116347
             carw |   .0160214   .0022626     7.08   0.000     .0115867    .0204561
       CARNegavgw |   .1614652   .0038218    42.25   0.000     .1539745    .1689559
              Drg |   1237.047    84.7893    14.59   0.000     1070.862    1403.231
       DrgNegavgw |   6894.579   248.1179    27.79   0.000     6408.275    7380.883
      costincomew |  -.1372356   .0008017  -171.18   0.000    -.1388069   -.1356642
costincomeNegavgw |  -.0360034    .001089   -33.06   0.000    -.0381378   -.0338689
          eqcgtaw |   -.161473   .0029454   -54.82   0.000    -.1672458   -.1557001
          revdivw |   .1374203   .0011855   115.92   0.000     .1350967    .1397439
      sizeassetsw |  -.3366174   .0107229   -31.39   0.000    -.3576339   -.3156008
             gdpg |   .2105167   .0058221    36.16   0.000     .1991055    .2219279
              gap |    .000367   .0001817     2.02   0.043     .0000108    .0007232
               hh |  -.0000786   5.38e-06   -14.61   0.000    -.0000891    -.000068
        inflation |   -1.12732    .003604  -312.80   0.000    -1.134383   -1.120256
                  |
             year |
            2002  |  -.7356892   .0484119   -15.20   0.000    -.8305753   -.6408032
            2003  |  -.6645746   .0514692   -12.91   0.000    -.7654528   -.5636964
            2004  |   -.208966   .0500894    -4.17   0.000    -.3071398   -.1107921
            2005  |   .1314741   .0506474     2.60   0.009     .0322066    .2307417
            2006  |   .0016818   .0515721     0.03   0.974     -.099398    .1027617
            2007  |  -1.141754     .05628   -20.29   0.000    -1.252062   -1.031447
            2008  |  -.1841005   .0701736    -2.62   0.009    -.3216388   -.0465621
            2009  |    -1.5462   .0637728   -24.25   0.000    -1.671193   -1.421208
            2010  |  -.3790909   .0665499    -5.70   0.000     -.509527   -.2486548
            2011  |  -.1778272    .066232    -2.68   0.007    -.3076401   -.0480143
            2012  |   .5179759   .0694484     7.46   0.000     .3818589    .6540929
            2013  |   .1009313   .0678153     1.49   0.137    -.0319849    .2338475
                  |
            _cons |   14.76652   .1427936   103.41   0.000     14.48665    15.04639
-----------------------------------------------------------------------------------

. 
. regress roeavgw L.roeavgw lconw LConNegavgw lcoffw LCoffNegavgw DLL DLLNegavgw carw CARNegavgw Drg DrgNegavgw costincomew
>  costincomeNegavg eqcgtaw revdivw sizeassetsw gdpg gap hh inflation i.year if sz_large == 1

      Source |       SS           df       MS      Number of obs   =     2,854
-------------+----------------------------------   F(32, 2821)     =    202.57
       Model |  119108.133        32  3722.12916   Prob > F        =    0.0000
    Residual |  51835.7242     2,821  18.3749465   R-squared       =    0.6968
-------------+----------------------------------   Adj R-squared   =    0.6933
       Total |  170943.857     2,853    59.91723   Root MSE        =    4.2866

-----------------------------------------------------------------------------------
          roeavgw |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
------------------+----------------------------------------------------------------
          roeavgw |
              L1. |   .4478967   .0143403    31.23   0.000     .4197781    .4760153
                  |
            lconw |  -.0112788   .0102626    -1.10   0.272    -.0314018    .0088441
      LConNegavgw |    .032871   .0129993     2.53   0.012     .0073819    .0583602
           lcoffw |   .0181506   .0267296     0.68   0.497    -.0342608    .0705621
     LCoffNegavgw |      .0601   .0353409     1.70   0.089    -.0091967    .1293967
              DLL |  -.0014191   .0070631    -0.20   0.841    -.0152685    .0124303
       DLLNegavgw |  -.0772503   .0078993    -9.78   0.000    -.0927393   -.0617613
             carw |   .0707043   .0366496     1.93   0.054    -.0011583     .142567
       CARNegavgw |   .0441851   .0258885     1.71   0.088    -.0065772    .0949473
              Drg |  -2844.248   1371.875    -2.07   0.038    -5534.227   -154.2693
       DrgNegavgw |   8238.426   1519.591     5.42   0.000     5258.803    11218.05
      costincomew |  -.1137821   .0110077   -10.34   0.000     -.135366   -.0921981
costincomeNegavgw |  -.0442549   .0118836    -3.72   0.000    -.0675564   -.0209534
          eqcgtaw |  -.1946006   .0276027    -7.05   0.000    -.2487241   -.1404771
          revdivw |   .1350576   .0101463    13.31   0.000     .1151626    .1549525
      sizeassetsw |  -.5806133   .2793035    -2.08   0.038    -1.128273   -.0329535
             gdpg |   .1978534   .0622663     3.18   0.002     .0757614    .3199454
              gap |  -.0000512   .0019546    -0.03   0.979    -.0038838    .0037814
               hh |  -.0000747   .0000461    -1.62   0.105    -.0001652    .0000157
        inflation |   -1.34687   .0372393   -36.17   0.000    -1.419889   -1.273851
                  |
             year |
            2002  |  -1.024305   .4651085    -2.20   0.028    -1.936292   -.1123175
            2003  |  -.9640165   .5134956    -1.88   0.061    -1.970881    .0428485
            2004  |   -.894301   .4736394    -1.89   0.059    -1.823016    .0344136
            2005  |   -.643246   .4720555    -1.36   0.173    -1.568855     .282363
            2006  |  -.4326105   .4677691    -0.92   0.355    -1.349815    .4845936
            2007  |  -2.533086   .4949623    -5.12   0.000     -3.50361   -1.562561
            2008  |  -3.201493   .6268909    -5.11   0.000    -4.430704   -1.972282
            2009  |  -4.378072   .6884812    -6.36   0.000     -5.72805   -3.028095
            2010  |  -2.830637    .690961    -4.10   0.000    -4.185477   -1.475797
            2011  |  -1.003674    .717223    -1.40   0.162    -2.410009    .4026601
            2012  |  -1.038119   .7168958    -1.45   0.148    -2.443812    .3675743
            2013  |  -1.736967   .7367133    -2.36   0.018    -3.181518   -.2924156
                  |
            _cons |   20.27332   4.231452     4.79   0.000     11.97626    28.57037
-----------------------------------------------------------------------------------

. 
. //Fixed Effect
. xtreg roeavgw L.roeavgw lconw LConNegavgw lcoffw LCoffNegavgw DLL DLLNegavgw carw CARNegavgw Drg DrgNegavgw costincomew c
> ostincomeNegavg eqcgtaw revdivw sizeassetsw gdpg gap hh inflation i.year, fe 

Fixed-effects (within) regression               Number of obs     =    256,215
Group variable: rssd9001                        Number of groups  =      8,463

R-sq:                                           Obs per group:
     within  = 0.5921                                         min =          1
     between = 0.7628                                         avg =       30.3
     overall = 0.6412                                         max =         52

                                                F(32,247720)      =   11236.87
corr(u_i, Xb)  = 0.0972                         Prob > F          =     0.0000

-----------------------------------------------------------------------------------
          roeavgw |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
------------------+----------------------------------------------------------------
          roeavgw |
              L1. |   .2181716   .0016662   130.94   0.000     .2149059    .2214374
                  |
            lconw |     .03776   .0013078    28.87   0.000     .0351969    .0403232
      LConNegavgw |   .0244018   .0024762     9.85   0.000     .0195485     .029255
           lcoffw |   .0889885   .0044246    20.11   0.000     .0803164    .0976607
     LCoffNegavgw |   .0467325   .0091096     5.13   0.000     .0288778    .0645871
              DLL |   .0012758   .0007983     1.60   0.110    -.0002888    .0028404
       DLLNegavgw |  -.1444623   .0010513  -137.42   0.000    -.1465228   -.1424018
             carw |   .0218343   .0023238     9.40   0.000     .0172798    .0263888
       CARNegavgw |   .1542742   .0040023    38.55   0.000     .1464298    .1621186
              Drg |   284.7925   91.50798     3.11   0.002     105.4393    464.1457
       DrgNegavgw |   6855.552   266.6159    25.71   0.000     6332.992    7378.112
      costincomew |   -.195189   .0013185  -148.04   0.000    -.1977732   -.1926049
costincomeNegavgw |  -.0237514   .0011927   -19.91   0.000    -.0260891   -.0214138
          eqcgtaw |  -.1005786   .0050358   -19.97   0.000    -.1104487   -.0907085
          revdivw |   .1454887   .0019641    74.08   0.000     .1416392    .1493382
      sizeassetsw |  -.7470183   .0419119   -17.82   0.000    -.8291645    -.664872
             gdpg |   .1440672   .0053648    26.85   0.000     .1335523     .154582
              gap |   .0014688    .000168     8.74   0.000     .0011396     .001798
               hh |  -2.80e-06   7.73e-06    -0.36   0.717    -.0000179    .0000123
        inflation |  -1.024179   .0037139  -275.77   0.000    -1.031458     -1.0169
                  |
             year |
            2002  |  -.5752657    .046052   -12.49   0.000    -.6655263   -.4850051
            2003  |   -.114553   .0507574    -2.26   0.024    -.2140361   -.0150699
            2004  |   .3188279   .0515795     6.18   0.000     .2177335    .4199224
            2005  |   .4477712    .051978     8.61   0.000     .3458957    .5496468
            2006  |  -.0036491   .0522223    -0.07   0.944    -.1060034    .0987052
            2007  |  -1.413015   .0563046   -25.10   0.000    -1.523371   -1.302659
            2008  |  -.8616926   .0700671   -12.30   0.000    -.9990223   -.7243628
            2009  |   -1.65588   .0682372   -24.27   0.000    -1.789623   -1.522137
            2010  |   -.457155   .0738302    -6.19   0.000    -.6018603   -.3124497
            2011  |   .1503186   .0753015     2.00   0.046     .0027298    .2979075
            2012  |   1.071721   .0802658    13.35   0.000     .9144027     1.22904
            2013  |   1.001874   .0814396    12.30   0.000     .8422547    1.161494
                  |
            _cons |   21.73754    .519171    41.87   0.000     20.71998     22.7551
------------------+----------------------------------------------------------------
          sigma_u |  2.3559056
          sigma_e |  3.3317545
              rho |  .33333323   (fraction of variance due to u_i)
-----------------------------------------------------------------------------------
F test that all u_i=0: F(8462, 247720) = 7.01                Prob > F = 0.0000

. 
. xtreg roeavgw L.roeavgw lconw LConNegavgw lcoffw LCoffNegavgw DLL DLLNegavgw carw CARNegavgw Drg DrgNegavgw costincomew c
> ostincomeNegavg eqcgtaw revdivw sizeassetsw gdpg gap hh inflation i.year if sz_large == 1, fe

Fixed-effects (within) regression               Number of obs     =      2,854
Group variable: rssd9001                        Number of groups  =        251

R-sq:                                           Obs per group:
     within  = 0.6100                                         min =          1
     between = 0.6969                                         avg =       11.4
     overall = 0.6606                                         max =         46

                                                F(32,2571)        =     125.65
corr(u_i, Xb)  = 0.0652                         Prob > F          =     0.0000

-----------------------------------------------------------------------------------
          roeavgw |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
------------------+----------------------------------------------------------------
          roeavgw |
              L1. |   .2522249   .0158255    15.94   0.000     .2211929    .2832569
                  |
            lconw |    -.00023   .0198824    -0.01   0.991    -.0392171     .038757
      LConNegavgw |   .0051098    .017249     0.30   0.767    -.0287136    .0389332
           lcoffw |   .0944378   .0554664     1.70   0.089    -.0143255    .2032011
     LCoffNegavgw |   .0981468   .0461399     2.13   0.034     .0076717     .188622
              DLL |  -.0391742   .0114804    -3.41   0.001     -.061686   -.0166624
       DLLNegavgw |  -.0835019   .0091809    -9.10   0.000    -.1015047   -.0654991
             carw |   .0330242   .0456547     0.72   0.470    -.0564995     .122548
       CARNegavgw |   .0486497   .0320658     1.52   0.129    -.0142277     .111527
              Drg |  -4816.351   1651.257    -2.92   0.004    -8054.281   -1578.422
       DrgNegavgw |   7752.787   1914.667     4.05   0.000     3998.342    11507.23
      costincomew |  -.1468866   .0178185    -8.24   0.000    -.1818268   -.1119465
costincomeNegavgw |  -.0375357   .0148414    -2.53   0.011     -.066638   -.0084335
          eqcgtaw |  -.1470841    .054429    -2.70   0.007    -.2538132   -.0403549
          revdivw |   .1900046   .0190765     9.96   0.000     .1525978    .2274115
      sizeassetsw |  -2.190867   .7146737    -3.07   0.002    -3.592262   -.7894731
             gdpg |   .1716367   .0592037     2.90   0.004     .0555449    .2877285
              gap |  -.0002034   .0018777    -0.11   0.914    -.0038853    .0034785
               hh |  -.0001164   .0000787    -1.48   0.139    -.0002708     .000038
        inflation |  -1.386672   .0417323   -33.23   0.000    -1.468505    -1.30484
                  |
             year |
            2002  |  -.9735441   .4713765    -2.07   0.039     -1.89786    -.049228
            2003  |  -1.277842   .5518724    -2.32   0.021    -2.360001   -.1956822
            2004  |  -1.653339   .5452736    -3.03   0.002    -2.722559   -.5841189
            2005  |  -1.679689   .5423525    -3.10   0.002    -2.743181   -.6161968
            2006  |  -1.372545   .5466321    -2.51   0.012    -2.444428   -.3006607
            2007  |  -3.615571   .5784606    -6.25   0.000    -4.749866   -2.481275
            2008  |  -3.451909   .7047984    -4.90   0.000    -4.833939   -2.069879
            2009  |  -3.676669   .8217015    -4.47   0.000    -5.287933   -2.065405
            2010  |  -2.309551   .8710174    -2.65   0.008    -4.017518   -.6015843
            2011  |  -.6592042   .9113234    -0.72   0.470    -2.446207    1.127798
            2012  |  -.7587979   .9294192    -0.82   0.414    -2.581284    1.063688
            2013  |  -2.225555   .9646078    -2.31   0.021    -4.117042   -.3340679
                  |
            _cons |   48.25691   10.95059     4.41   0.000     26.78404    69.72978
------------------+----------------------------------------------------------------
          sigma_u |  3.2985744
          sigma_e |  3.9477583
              rho |  .41112538   (fraction of variance due to u_i)
-----------------------------------------------------------------------------------
F test that all u_i=0: F(250, 2571) = 3.02                   Prob > F = 0.0000

.

I appreciate any feedback that could shed a light on this implausible issue,

[daniel klein]

Since you do not show the code and results you get from xtabond2 (from SJ or SSC?) or any other proper estimator for dynamic panel models, it will be very hard to comment on your question, which seems to center around the discrepancy between the POLS, FE and dynamic estimators.

You seem to be aware that the results you show here are biased by design, although, as you also seem to be aware of, the bias is supposed to decline with larger T. I am not an expert on dynamic models, but there are experts on the list. They will need to see the code and results you get from those models. If you are lucky there might also be people in your field of research, so providing a bit more details about the theory and why you think results you obtain are not plausible might also help.

As a small comment, I do not believe that your problems have anything to do with Stata or other software, as you seem to claim in your initial statement. It is much more likely that you doubt the statistic theory and/or assumptions underlying dynamic panel model estimation. This is a fine and informative topic to discuss here.

Best
Daniel

[Rim Achour]

Dear Daniel,


Thanks for your reply. The codes and the results for the PooledOLS and the Fixed Effects regressions are included in the 1st post . I am attaching the codes ( dynamic.do ) and results ( dynamic.txt )for the dynamic model using xtabond2 for your consideration. Please note that I tried to attach my data, however I couldn't attach .dta file neither .csv. Also, I saved my data as tab delimited and I couldn't attach it too.
I am doubting whether xtabond2 really is a good estimation tool. Probably there are other softwares that determine automatically the number of lags and the best combination of instruments that improves the estimation. I am looking forward for your input . I highly appreciate your help.

[JanDitzen]

I haven't worked with xtabond2 for a while, but hopefully the questions are the right ones to ask and help you. Have you read Roodman, D. (2009, Oxford Bulletin of Economics and Statistics) and Roodman (2009, Stata Journal)? Both papers explain and discuss the strength and weaknesses of xtabond2.

• Your setting (large N, small T) is fine for xtabond2. However it seems that in your unbalanced panel, for a few of the groups there are no observations left. Does this pose a problem? Intuitively I would say no, because you have homogeneous panel, but you use lags as instruments.
• Is there a reason you use the option nolevel? There is a literature (see the above papers or Blundell, Bond (1998, Journal of Econometrics) stating that the additional moment condition improve estimations). Interestingly you do this in your 2nd specification in the text file.
• Do I read iv(gdpg i.year) that you use year dummies as instruments? The transformations xtabond2 is doing, could explain btw. the omitted variables for the year dummies.
• How did you select the number of lags? Of course the lags have a huge influence, as they are used as instruments for the level or first differences. Did you run tests for exogeneity and weak instruments? (saying so, there is a literature questioning if lags are valid instruments, see for example Reed (2015, Oxford Bulletin of Economics and Statistics)). Also, the Hansen J statistic is rejected in most of the cases.

[Jesse Wursten]

My understanding is that dynamic models shouldn't lead to huge differences in results from OLS and Fixed Effects models especially that I don't have a small sample issue.

In OLS and FE, the coefficient on your autoregressive term (lagged Y) will be biased. This bias decreases with T, not with N. So the statement above isn't really correct - sure you have a large panel, but it is wide rather than long (large N small T), which is exactly where the GMM methods ("xtabond") can make a large difference.

As a second sidenote, once you include lagged variables, interpretation of coefficients becomes trickier. We are usually interested in the long run impact of some variable, which is no longer just the coefficient estimate, but rather (b1 + b2 + ... + bn)/(1-alpha1-...-alpnan), where the bs are the coefficients on the lags of the variable of interest and alpha the autoregressive terms. As a result, the size of your longrun effect is highly dependent on the size of the estimated alphas.

[Sebastian Kripfganz]

Daniel, Jan, and Jesse have all given some good remarks already. Let me add a couple more:

• You are not giving the GMM estimators a fair comparison to the OLS estimators (including the FE estimator which is an OLS estimator after a particular variable transformation). For the OLS procedures, all variables are assumed to be (strictly) exogenous with respect to the idiosyncratic error term. You are not doing the same with your GMM estimator. By instrumenting all the variables with their own lags, you are implicitly assuming that the right-hand side variables in your model are predetermined or even endogenous. Of course, this makes a huge difference and it has nothing to do with the dynamic nature of the model. Just think about a static model and imagine that you run an IV estimation assuming that all variables are endogenous. It is extremely difficult to find valid and in particular strong instruments for all of these endogenous variables. While own lags might be valid, they can easily become weak instruments if the variables are not strongly correlated over time.
• There is a severe bug in the xtabond2 command that computes incorrect degrees of freedom for the Sargan/Hansen and difference-in-Hansen tests when you specify time dummies with the factor notation. xtabond2 also counts all the omitted categories as if it had estimated parameters for them. As a consequence, the degrees of freedom are too small and the p-values are invalid. You can do a simple count by yourself: Count the number of actually estimated coefficients and subtract them from the number of instruments. This should be the degrees of freedom for the Hansen test. You will notice that this is not the case in the xtabond2 output. For more details, see my comment in the following topic and the links therein: xtabond2 and deeper lags
• When using a system-GMM estimator (i.e. xtabond2 without the noleveleq option), you should explicitly specify the equation() suboption for the iv() option. See my comments in the following topic: Hansen test is missed after xtabond2 (collapse)
• In addition to my previous two points, time dummies should not be specified as instruments for both of the equations. The implied instruments are (asymptotically) colinear and including them as instruments for both equations would imply again an incorrect number of degrees of freedom. See the following topic for details: System GMM - Time dummies
• Ignoring the technical difficulties in specifying the command syntax for xtabond2 appropriately, the differing results are primarily a consequence of the (implicit) assumptions that you make (see in particular my first point above). It would be too easy to blame the GMM estimator for not giving you the desired results. As mentioned by others before, regress and xtreg yield biased/inconsistent estimates for small-T dynamic models. Try to start with a specification of xtabond2 that changes as few assumptions as possible compared to the OLS estimators, i.e. use the respective lags as instruments for the lagged dependent variable but continue to assume that all the other variables are strictly exogenous and instrument them by themselves in the first-differenced equation with the iv(, eq(diff)) option.