Interpreting results from a random effects model with interaction with time

[Rena Jenkins]

Hi Stata Forum

I have been playing with random effects modelling with interactions with time, as helped by members of this forum. My data now looks like this

. list if id<5

      +----------------------------------------------+
      | id   time   exposure    outcome   outcome_bl |
      |----------------------------------------------|
   1. |  1      1          2   34.14907     33.98133 |
   2. |  1      2          2   36.89132     33.98133 |
   3. |  1      3          2   37.09322     33.98133 |
   4. |  2      1          2   24.22013      21.2088 |
   5. |  2      2          2   25.23487      21.2088 |
      |----------------------------------------------|
   6. |  3      1          1   31.93662     32.69796 |
   7. |  3      2          1   37.16611     32.69796 |
   8. |  3      3          1   33.71714     32.69796 |
   9. |  4      1          2   25.69123      25.6259 |
  10. |  4      2          2   25.60167      25.6259 |
      |----------------------------------------------|
  11. |  4      3          2   22.32802      25.6259 |
      +----------------------------------------------+

I get a sigificant main effect of time 4, but not any of the interactions (just) - see output at the end of this message. I did a naive analysis of means to try to understand what was going on:

. gen diff=outcome-outcome_bl

. table exposure time, c(mean diff)

-------------------------------------------
          |              time              
 exposure |         1          2          3
----------+--------------------------------
        1 | -.0730373   .1984727   .0705481
        2 | -.4656163   .1813771   .1950545
        3 | -.5124291  -1.245556   .6545261
        4 | -1.534946  -.0874589    .847056
-------------------------------------------

Am I right in thinking that the "main effect" of time 4 should be interpreted as the effects of being in exposure group 4 and time 1?

Many thanks

Rena

. xtmixed outcome i.exposure##time outcome_bl || id:

Performing EM optimization:

Performing gradient-based optimization:

Iteration 0:   log likelihood = -3527.7256  
Iteration 1:   log likelihood = -3527.7182  
Iteration 2:   log likelihood = -3527.7182  

Computing standard errors:

Mixed-effects ML regression                     Number of obs      =      1405
Group variable: id                              Number of groups   =       688

                                                Obs per group: min =         1
                                                               avg =       2.0
                                                               max =         3


                                                Wald chi2(12)      =   2885.85
Log likelihood = -3527.7182                     Prob > chi2        =    0.0000

-------------------------------------------------------------------------------
      outcome |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
     exposure |
           2  |  -.3427129   .3312138    -1.03   0.301      -.99188    .3064542
           3  |  -.2236166   1.019724    -0.22   0.826    -2.222239    1.775006
           4  |  -1.291953   .5311281    -2.43   0.015    -2.332945   -.2509614
              |
         time |
           2  |   .3840982   .3167607     1.21   0.225    -.2367413    1.004938
           3  |   .1534453   .3200552     0.48   0.632    -.4738513    .7807419
              |
exposure#time |
         2 2  |   .2322221   .3692434     0.63   0.529    -.4914817    .9559259
         2 3  |   .3565169   .3904371     0.91   0.361    -.4087257     1.12176
         3 2  |  -1.295537   1.049529    -1.23   0.217    -3.352576    .7615025
         3 3  |   .2487972   1.249102     0.20   0.842    -2.199398    2.696992
         4 2  |   .5126085   .6681532     0.77   0.443    -.7969478    1.822165
         4 3  |   1.400178    .741296     1.89   0.059    -.0527351    2.853092
              |
   outcome_bl |    .972069   .0181939    53.43   0.000     .9364095    1.007728
        _cons |   .5444925   .5156786     1.06   0.291     -.466219    1.555204
-------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
id: Identity                 |
                   sd(_cons) |   2.194716   .1090333       1.99109    2.419168
-----------------------------+------------------------------------------------
                sd(Residual) |   2.335483   .0639805       2.21339     2.46431
------------------------------------------------------------------------------
LR test vs. linear regression: chibar2(01) =   162.80 Prob >= chibar2 = 0.0000

[Clyde Schechter]

Am I right in thinking that the "main effect" of time 4 should be interpreted as the effects of being in exposure group 4 and time 1?

I assume you mean main effect of exposure group 4, given that your time variable takes on values only up to 3.

Under that assumption, yes, given that time = 1 is the reference value (omitted category) for time, the main effect of exposure group 4 is the effect of exposure group 4 at time 1.

[Rena Jenkins]

So it would be fair when writing up the results to say that the only signficant effect was exposure group 4 at time 1?

[Clyde Schechter]

Not necessarily. For example, the effect of exposure group 3 at time 2 is not shown directly in your output: it is the sum of the coefficients of exposure group3 main effect and the exposure group 3 X time 2 interaction effect. (I just picked that to exemplify the principle: there is nothing about that particular combination that is distinctive.) Even though the main effect and the interaction effect are not separately statistically significantly different from zero, their sum might be. The simplest way to get all of this is the -margins- command.

[Rena Jenkins]

Thanks again Clyde. I have had a go--the output is below. Presumably the p-values are whether a prediction is signficantly different to 0, which of course they all are. How would you see whether they are different from each other in a manner analagous to Tukey's HSD for example? Sorry if that is a stupid question. I am a bit bewildered by the margins help.

Best wishes

Rena

. margins exposure##time

Predictive margins                                Number of obs   =       1405

Expression   : Linear prediction, fixed portion, predict()

-------------------------------------------------------------------------------
              |            Delta-method
              |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
     exposure |
           1  |   23.62793   .2074826   113.88   0.000     23.22127    24.03459
           2  |   23.45553   .1354463   173.17   0.000     23.19006      23.721
           3  |   23.01794   .7453677    30.88   0.000     21.55705    24.47883
           4  |   22.86644   .3984132    57.39   0.000     22.08556    23.64731
              |
         time |
           1  |   23.14367   .1343979   172.20   0.000     22.88025    23.40708
           2  |   23.68148   .1406854   168.33   0.000     23.40574    23.95722
           3  |   23.62656   .1625478   145.35   0.000     23.30797    23.94514
              |
exposure#time |
         1 1  |   23.45613   .2902542    80.81   0.000     22.88724    24.02502
         1 2  |   23.84023   .2586304    92.18   0.000     23.33332    24.34713
         1 3  |   23.60958   .2645268    89.25   0.000     23.09111    24.12804
         2 1  |   23.11342   .1595368   144.88   0.000     22.80073     23.4261
         2 2  |   23.72974   .1789291   132.62   0.000     23.37904    24.08043
         2 3  |   23.62338   .2153203   109.71   0.000     23.20136     24.0454
         3 1  |   23.23251   .9775309    23.77   0.000     21.31659    25.14844
         3 2  |   22.32108   .8565431    26.06   0.000     20.64228    23.99987
         3 3  |   23.63476    1.12517    21.01   0.000     21.42946    25.84005
         4 1  |   22.16418   .4447607    49.83   0.000     21.29246    23.03589
         4 2  |   23.06088   .5637893    40.90   0.000     21.95588    24.16589
         4 3  |    23.7178   .6440276    36.83   0.000     22.45553    24.98007
-------------------------------------------------------------------------------

[Clyde Schechter]

To contrast them all against each other, -pwcompare- is what you need.

[Rena Jenkins]

Thanks Clyde--you are a fount of knowledge! Would giving the significant pairwise margin differences (together with the model coefficients etc) be generally accepted in aiding interpretation of results? If not how should we do it?

[Clyde Schechter]

Thanks for your kind words.

A full presentation of the contrasts would include not just those differences that turned out to be statistically significant, but all of them, along with confidence intervals or standard errors and p-values, perhaps highlighting the statistically significant ones in some way. (Not to mention all of the descriptive information on the study sample, and the raw results before contrasts were calculated.) I would expect to see that in, say, a thesis or a full-length academic paper.

But what would be appropriate depends on the context (an article, an abstract, a poster, a working paper, an executive summary, a technical report, an oral presentation, etc.), the audience, the expectations that prevail in your discipline, etc. So I don't think I can really advise you on this--you know better than I your specific situation and the conventions that apply. If you are unsure, you would be better off seeking advice from one of your colleagues on this.