Compare Moderation effect between panel and cross sectional data

[Xaver Binghaus]

Dear All,

I have a dataset which i split into a cross sectional and time-panel dataset.
Both are using the same IV and CV but with different DV.
I added a moderation variable to see if a variable has an effect on the strength of my dp in each dataset.

My question now is: How can I test if the effect of the moderator is stronger in the cross sectional dataset than the panel data.

Thank you very much!
As I am still new and inexperienced in Stata and this kind of modelling, please forgive me for the possibly wrong wording.
Best regards,
Xaver

[Carlo Lazzaro]

Xaver:
welcome to this forum.
Assuming that you're dealing with -fe- estimator in your panel dataset, an idea might be:

use "https://www.stata-press.com/data/r16/nlswork.dta"
. regress ln_wage c.age##c.age if year==70 & idcode<=6

      Source |       SS           df       MS      Number of obs   =         5
-------------+----------------------------------   F(2, 2)         =      0.51
       Model |  .181259038         2  .090629519   Prob > F        =    0.6602
    Residual |  .352124836         2  .176062418   R-squared       =    0.3398
-------------+----------------------------------   Adj R-squared   =   -0.3203
       Total |  .533383874         4  .133345969   Root MSE        =     .4196

------------------------------------------------------------------------------
     ln_wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |  -2.512487   3.949111    -0.64   0.590    -19.50414    14.47917
             |
 c.age#c.age |   .0614176   .0941718     0.65   0.581     -.343771    .4666062
             |
       _cons |   26.77668   40.66101     0.66   0.578    -148.1735    201.7269
------------------------------------------------------------------------------

. est store regress

. regress ln_wage c.age##c.age i.idcode if idcode<=6

      Source |       SS           df       MS      Number of obs   =        76
-------------+----------------------------------   F(7, 68)        =      9.00
       Model |  3.91986067         7  .559980096   Prob > F        =    0.0000
    Residual |  4.22971324        68  .062201665   R-squared       =    0.4810
-------------+----------------------------------   Adj R-squared   =    0.4276
       Total |  8.14957391        75  .108660985   Root MSE        =     .2494

------------------------------------------------------------------------------
     ln_wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |   .1925272   .0430317     4.47   0.000     .1066587    .2783957
             |
 c.age#c.age |  -.0028797   .0007055    -4.08   0.000    -.0042875   -.0014719
             |
      idcode |
          2  |  -.4066809   .1023149    -3.97   0.000    -.6108471   -.2025148
          3  |  -.5763484   .0995888    -5.79   0.000    -.7750748    -.377622
          4  |  -.1772529   .1085198    -1.63   0.107    -.3938008    .0392949
          5  |   -.308972   .1065197    -2.90   0.005    -.5215287   -.0964153
          6  |   -.352106   .0986372    -3.57   0.001    -.5489334   -.1552786
             |
       _cons |  -.9215169   .6265056    -1.47   0.146    -2.171689    .3286555
------------------------------------------------------------------------------

. est store regress_fe

. suest regress regress_fe, vce(cluster idcode)

Simultaneous results for regress, regress_fe

                                                Number of obs     =         76

                                     (Std. Err. adjusted for 6 clusters in idcode)
----------------------------------------------------------------------------------
                 |               Robust
                 |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
regress_mean     |
             age |  -2.512487   1.480643    -1.70   0.090    -5.414494    .3895195
                 |
     c.age#c.age |   .0614176   .0361132     1.70   0.089    -.0093631    .1321982
                 |
           _cons |   26.77668   14.95088     1.79   0.073    -2.526506    56.07987
-----------------+----------------------------------------------------------------
regress_lnvar    |
           _cons |  -1.736917   .3794733    -4.58   0.000    -2.480671   -.9931627
-----------------+----------------------------------------------------------------
regress_fe_mean  |
             age |   .1925272    .084127     2.29   0.022     .0276413    .3574131
                 |
     c.age#c.age |  -.0028797   .0012792    -2.25   0.024     -.005387   -.0003724
                 |
          idcode |
              2  |  -.4066809   .0236205   -17.22   0.000    -.4529762   -.3603857
              3  |  -.5763484   .0581783    -9.91   0.000    -.6903757   -.4623211
              4  |  -.1772529   .0679074    -2.61   0.009     -.310349   -.0441569
              5  |   -.308972   .0507559    -6.09   0.000    -.4084517   -.2094923
              6  |   -.352106   .0485282    -7.26   0.000    -.4472195   -.2569924
                 |
           _cons |  -.9215169    1.28545    -0.72   0.473    -3.440952    1.597918
-----------------+----------------------------------------------------------------
regress_fe_lnvar |
           _cons |  -2.777374   .4702366    -5.91   0.000     -3.69902   -1.855727
----------------------------------------------------------------------------------

. test [regress_mean]c.age#c.age=[regress_fe_mean]c.age#c.age

 ( 1)  [regress_mean]c.age#c.age - [regress_fe_mean]c.age#c.age = 0

           chi2(  1) =    3.16
         Prob > chi2 =    0.0757

.

[Xaver Binghaus]

Hi Carlo,
Thank you very much! You have already untied a knot in my head.

Now I'm just unsure what the test should say in the end.
I am trying to find out:
[regress_mean]c.age#c.age >[regress_fe_mean]c.age#c.age

The Chi2 test here only describes the independence or does that describe my case exactly?

Thanks for the help!
best regards
Xaver

[Carlo Lazzaro]

Xaver:
the -test compare the interaction coefficients of the cross-sectional and the fixed effect regression model.
It was necessary to avoid -xtreg.fe- as it is not supported by -suest-.

[Xaver Binghaus]

Thank you! and just to be sure. Would ou be so kind and interpret me your result from above?
. test [regress_mean]c.age#c.age=[regress_fe_mean]c.age#c.age ( 1) [regress_mean]c.age#c.age - [regress_fe_mean]c.age#c.age = 0 chi2( 1) = 3.16 Prob > chi2 = 0.0757 Thnak you very much!

[Carlo Lazzaro]

Xaver:
we cannot reject at a 0.05 arbitrary cut off the null hypothesis that the difference between the coefficients estimated in the two models is zero (or, put differently, that the two coefficients do not differ).