margins- marginal effect of time-variant variables in mixed effects models

[Sule Yaylaci]

Dear Statalist users,
I am using Stata 14 SE, and have a question about using margins for a time-variant variable in a mixed-effects model.
My dependent variable Y, and three independent variables (X) are measured at two time-points. While X1and X3 are measured in each district at time 1 and time 2(wave), X2 is measured at the province-level at time 1 and 2. Districts are nested in provinces. Basically I have multi-level repeated measures data. My control variables (Z) are time-invariant. Data example is below.

The code I use to model the effect of X1 X2 and X3 on Y is:

 mixed Y i.wave  i.X1 c.X3  c.X2   Z1 Z2   ||province:  ||district_no:

I look at the effect of the interaction between X1 and X2:

 mixed Y i.wave  i.X1##c.X3 c.X2   Z1 Z2   ||province:  ||district_no:

And what I would like to do is to plot how this interaction. X3 is measured in wave 1 and 2, and I just am not sure how to show the effect of the change in X3 in a graph. I tried:

margins X1, at (X3(-.15(.03).15) 
marginsplot

I am not confident this is capturing what I would like to show. I generated a variable called 'DifferenceinX3' where I subtracted the wave 1 values of X3 from wave 2 values.
When I use this variable in the model instead of X3, both the estimates and the marginsplot change completely. I am not sure why the coefficient of X3 almost doubles when I use the difference variable instead.
Should not their substantive effect be exactly the same? The direction of effect is the same but the coefficient almost doubles.
But more importantly, how does one go about plotting the marginal effects of two time-variant variables in a mixed model?

I appreciate your help.

* Example generated by -dataex-. To install: ssc install dataex
clear
input double Y byte wave float X1 double X3 float(X2 Z1 Z2 province) int district_no float DifferenceinX3
 .5008916001877053 1 0 .8836196467457053         0 0 0 1  1              0
 .5701738334858188 2 0  .891889725917615  .5954652 0 0 1  1      .00827006
.25843545684528696 1 0 .8517262864682598         0 0 0 1  2              0
.33096601673721704 2 0 .8558748788826122  .5954652 0 0 1  2    .0041485853
 .2314806361495061 1 0 .8671302188663923         0 0 0 1  3              0
.28064805600966664 2 0 .8749831214675643  .5954652 0 0 1  3     .007852902
 .4701195219123506 1 0 .8039459375485475         0 0 0 1  4              0
 .5468765275197967 2 0  .825685903500473  .5954652 0 0 1  4     .021739945
.45914704629798625 1 0 .8386024120303737         0 0 0 1  5              0
 .5308816595945309 2 0 .8622581288649511  .5954652 0 0 1  5     .023655705
 .5529871977240398 1 0 .9015807301467821         0 0 0 1  6              0
 .5972691441441441 2 0 .9105439383410197  .5954652 0 0 1  6     .008963187
.23223635003739715 1 0  .869795996674554         0 0 0 1  7              0
 .2716732739920712 2 0 .8649493081680801  .5954652 0 0 1  7     -.00484667
.42910860429108605 1 0 .8222902932232359         0 0 0 1  8              0
 .5013144023806029 2 0 .8456993069130976  .5954652 0 0 1  8     .023409005
 .4536984981126014 1 0 .8757469606429013         0 0 0 1  9              0
 .5270582609388699 2 0 .8802646998000965  .5954652 0 0 1  9    .0045177345
 .4836112708453134 1 0 .7833479404031551         0 0 0 1 10              0
 .5462431001464458 2 0 .8043728423475259  .5954652 0 0 1 10     .021024877
   .36644963615473 1 0 .8615101724805369         0 0 0 1 11              0
 .4484892121448794 2 0 .8800895139308706  .5954652 0 0 1 11     .018579356
 .2582014753593243 1 0 .8491196205853588         0 0 0 1 12              0
 .3228475641790513 2 1 .8528333602230218  .5954652 0 0 1 12    .0037137566
.35363741339491916 1 0 .8177992041628406         0 0 0 1 13              0
 .4731265652090156 2 0 .8134437035333794  .5954652 0 0 1 13    -.004355507
.32956786802940646 1 0 .8485232696897375         0 0 0 1 14              0
.37584912406149446 2 0 .8606675244077803  .5954652 0 0 1 14     .012144266
 .3377397403399747 1 0 .8314157170778479         0 0 0 1 15              0
.42843334243252795 2 0 .8406463605192804  .5954652 0 0 1 15     .009230647
 .5449415852219232 1 0 .8457498530453939         0 0 0 2 16              0
  .641944955764613 2 0 .8683048852266039  .3317993 0 0 2 16      .02255503
  .524244480400856 1 0 .7929118002416432         0 0 0 2 17              0
 .6770883478172743 2 0 .8163368642780467  .3317993 0 0 2 17     .023425037
 .5755950385517935 1 0 .8811443932411674         0 0 0 2 18              0
 .6560495938435229 2 1 .9165220744168112  .3317993 0 0 2 18      .03537767
 .7091292483254775 1 0 .6773241515002459         0 0 0 2 19              0
   .85121412803532 2 0 .7589862514493954  .3317993 0 0 2 19      .08166207
 .4849688681767829 1 0 .8204195205479452         0 0 0 2 20              0
 .5811804708578187 2 0 .8416793893129771  .3317993 0 0 2 20      .02125984
  .685989894350023 1 0 .7928177975148384         0 0 0 2 21              0
 .8277890608586036 2 0  .828457731311777  .3317993 0 0 2 21      .03563996
 .7121102248005802 1 0 .7838338895068595         0 0 0 2 22              0
 .8619329388560157 2 0  .872473077649726  .3317993 0 0 2 22      .08863921
 .9071300179748353 1 0 .8856816985436041         0 0 0 2 23              0
 .9593705293276109 2 0 .9326021581461171  .3317993 0 0 2 23      .04692047
 .5623674911660778 1 1 .8395382395382396         0 0 0 2 24              0
 .6620594333102972 2 0 .8542088516054382  .3317993 0 0 2 24     .014670635
 .5348067182412929 1 0 .9103541429696387         0 0 0 3 25              0
 .6653963139734789 2 0  .915298976671581 .28208148 0 0 3 25     .004944839
 .4125722543352601 1 0 .8892910634048926         0 0 0 3 26              0
 .4892944388561575 2 0  .900830606594513 .28208148 0 0 3 26      .01153956
 .5145569620253164 1 0  .868237347294939         0 0 0 3 27              0
   .62026913372582 2 0 .8601643069393463 .28208148 0 0 3 27    -.008073069
 .6256125821524903 1 0 .8756493401735875         0 0 0 3 28              0
 .7158580413297394 2 0 .8784253184098804 .28208148 0 0 3 28    .0027759855
 .4544952285283777 1 0 .8783187717363644         0 0 0 3 29              0
 .5412363492612542 2 1 .8783167145512929 .28208148 0 0 3 29 -2.0720697e-06
 .5304798962386511 1 0 .9205705009276438         0 0 0 3 30              0
 .6460984702403908 2 0 .9102711397058824 .28208148 0 0 3 30    -.010299353
.43370756482224004 1 0 .9102380952380953         0 0 0 3 31              0
 .5171763437963087 2 0 .9005827090022595 .28208148 0 0 3 31    -.009655378
 .4417435328386157 1 0 .8575885377549252         0 0 0 3 32              0
 .5016402405686168 2 0 .8635209235209235 .28208148 0 0 3 32     .005932394
 .4869785664899747 1 0  .778960223307746         0 0 0 3 33              0
 .5746084480303749 2 0 .7624526498389209 .28208148 0 0 3 33    -.016507579
 .4162415833503367 1 0 .8761123713139068         0 0 0 3 34              0
.47278770253427505 2 1 .8665964542741794 .28208148 0 0 3 34    -.009515888
  .559327566508895 1 0 .8404325464855598         0 0 0 3 35              0
  .660734327400994 2 0 .8594414893617022 .28208148 0 0 3 35     .019008964
 .5650262617035853 1 0 .9233479726279236         0 0 0 3 36              0
 .7066111111111111 2 0 .9333808336302102 .28208148 0 0 3 36     .010032884
 .6467490520994242 1 0 .9294468787705594         0 0 0 3 37              0
 .7727925586485193 2 0 .9328652917946467 .28208148 0 0 3 37     .003418416
.43885714285714283 1 0 .8626991565135895         0 0 0 3 38              0
 .4796839729119639 2 0 .8629751290473956 .28208148 0 0 3 38     .000275978
 .5349692529496572 1 0 .8505993873465352         0 0 0 3 39              0
 .6348095224320963 2 0 .8626212058616248 .28208148 0 0 3 39     .012021798
  .569474921630094 1 0 .8865721434528774         0 0 0 3 40              0
 .6965041965041965 2 0 .8881137465949106 .28208148 0 0 3 40     .001541624
 .5593326906149139 1 0 .8820998278829604         0 0 0 3 41              0
 .6688803780964798 2 0 .8722279220266751 .28208148 0 0 3 41    -.009871885
.24974731232197003 1 0 .8973921874433282         0 0 0 3 42              0
.30645011600928074 2 0 .9088443737344518 .28208148 0 0 3 42      .01145216
 .3044498656702591 1 0 .8084917045579219         0 0 0 4 43              0
.44136020068534704 2 0 .7534060943451509         0 0 0 4 43     -.05508561
.06365623500559792 1 0 .8332165995447383         0 0 0 4 44              0
.12957372298031639 2 1  .753286551785397         0 0 0 4 44     -.07993005
.06817504100193827 1 0 .8357064790727029         0 0 0 4 45              0
.11110193633623715 2 0  .765450680404053         0 0 0 4 45     -.07025579
 .2962797808638043 1 0 .7742774566473989         0 0 0 4 46              0
 .5701609574907139 2 0 .7253514252245217         0 0 0 4 46     -.04892602
.15349887133182843 1 0 .8223345320244921         0 0 0 4 47              0
 .3618196160925585 2 0 .7642843118005105         0 0 0 4 47     -.05805022
.07620412844036697 1 0 .8578691709844559         0 0 0 4 48              0
.16082134968218906 2 0 .7408955808864356         0 0 0 4 48     -.11697356
.12797713559860274 1 0 .8392732354996506         0 0 0 4 49              0
.20923184520340365 2 0 .7784266879037254         0 0 0 4 49     -.06084653
 .1355679965983984 1 0 .8142646558566731         0 0 0 4 50              0
.26414204902576993 2 0 .7482440990213011         0 0 0 4 50    -.066020556
end

[Clyde Schechter]

There is no reason to think that a regression of Y on X and a regression of Y on difference in X would have the same coefficient. If you try to derive that algebraically you will see that it fails. Without formal algebra, in most realistic situations the difference in X will be on a smaller scale than the values of X themselves. So if the typical difference in X is about half the typical value of X, then the coefficient of the difference in X would have to be about twice as large as the coefficient of the difference in X.

I think what's confusing you is this: if you were to regress the difference in Y on the difference in X, then you would get the same coefficient as the regression of Y on X.

Your -marginsplot- is what I typically do to show interaction effects. If you prefer to have a graph of the difference Y(X3 | X1 = 2) - Y(X3 | X1 = 1) then you can get that with

margins, dydx(X1) at(X3 = (-.15(.03).15))
marginsplot

There's no substantive difference between these two approaches: just two different ways of showing the same thing. It's, I would say, a matter of taste which is better.

[Sule Yaylaci]

Thanks much, Clyde. This is very helpful.
I have one more question. I realized that X3 may be endogenous to the model in that change in X3 may have stemmed from changes in X1 and X2. So, I fit the following model:

 gsem (Y<-i.wave X1 X2 X3 P[province] C[province>district_no]) (X3<- i.wave X1 X2 P[province] C[province>district_no]), nocapslatent latent (P C)

And then I looked at the interaction between X1 and X3 :

gsem (Y<-i.wave i.X1##c.X3 X2  P[province] C[province>district_no]) (X3<- i.wave X1 X2 P[province] C[province>district_no]), nocapslatent latent (P C)

The model runs but then margins does not:

margins X1, dydx(X3)

and gives the following error :"prediction is a function of possibly stochastic quantities other than e(b)"

I assume this has to do with X3 being an endogenous variable, and it seems that running interactions does not make sense.

I should just discard the interaction model, right?

[Clyde Schechter]

The error message you are getting does arise because of the endogeneity of X3. But it does not follow that running interactions does not make sense. The problem, rather is that -margins- is not set up to incorporate cross-equation random effects. (That is, it cannot use the random effects of the equation for X3 to calculate margins for the equation for Y.)

However, I will tell you what does not make sense about this model: using the same random intercepts for both Y and X3. While it is legal to constrain the random intercepts of those equations to be identical, it would be an extraordinary coincidence if that were not a gross mis-specification of the true data generating process herema.

If you correct that, Stata will still not be able to run -margins- afterward, but at least you will have a coherent model. But I would not know how to calculate a marginal effect for X3 in this model. In fact, I don't think a marginal effect of X3 would be meaningful because X3 itself depends on X1 and X2 and wave.

[Sule Yaylaci]

Thanks so much for pointing out the problem of constraining the random intercepts of both equations, Clyde. I really appreciate the insight! Do you think it makes a difference whether I constrain the random intercepts for the Y or X3? Although overall results somewhat similar, coefficient sizes change significantly. Because both Y and X3 are observed at district-level twice, I believe I should use the "C[province>district_no]" restriction in both, but then the results cease to make sense. Statistically in models with an endogenous covariates, should the random intercepts be placed for the main equation (Y) or the endogenous equation (X3)? Judging by the results I get, I would guess placing the random intercept constraints for the X3 and not constraining the Y would be the way to go, as those results are the only ones that make sense. However, the coefficients become super large and almost uninterpretable. I wonder about the statistical logic.

[Clyde Schechter]

I didn't mean that one of the equations should lack the random intercepts. I meant that you should not constrain them to be the same in both equations. Sorry I didn't make that clearer. So I would do this as

gsem (Y<-i.wave i.X1##c.X3 X2 PY[province] CY[province>district_no])  ///
    (X3<- i.wave X1 X2 PX3[province] CX3[province>district_no]), nocapslatent latent (P C)

[Sule Yaylaci]

That makes sense. Thanks for clarifying. I appreciate the help. Do you think I need to do anything about the covariances? I guess by default the model estimates covariances of PY and PX# even though they are "the same" random intercepts.