Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Multilevel model: how to determine whether there was a statistically significant change over time

    Hi all,

    I am running a simple multilevel model to determine whether a group of people had a statistically significant change in an outcome variable over time. Each person was assessed at 5 time points. The outcome variable is continuous.

    My idea is that I would run a multilevel model with a random intercept for the person and a random slope for time, using the following code:

    Code:
    . mixed dep time || person: time
    
    Performing EM optimization:
    
    Performing gradient-based optimization:
    
    Iteration 0:   log likelihood = -191.64297  
    Iteration 1:   log likelihood = -191.49105  
    Iteration 2:   log likelihood = -191.48975  
    Iteration 3:   log likelihood = -191.48974  
    
    Computing standard errors:
    
    Mixed-effects ML regression                     Number of obs     =         75
    Group variable: person                          Number of groups  =         15
    
                                                    Obs per group:
                                                                  min =          5
                                                                  avg =        5.0
                                                                  max =          5
    
                                                    Wald chi2(1)      =       0.02
    Log likelihood = -191.48974                     Prob > chi2       =     0.8878
    
    ------------------------------------------------------------------------------
             dep |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
    -------------+----------------------------------------------------------------
            time |   .0095594   .0677819     0.14   0.888    -.1232906    .1424094
           _cons |   78.29368   .7173816   109.14   0.000     76.88763    79.69972
    ------------------------------------------------------------------------------
    
    ------------------------------------------------------------------------------
      Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
    -----------------------------+------------------------------------------------
    person: Independent          |
                       var(time) |   4.50e-12   5.45e-11      2.21e-22    .0917611
                      var(_cons) |   5.107832   2.402147      2.032018    12.83943
    -----------------------------+------------------------------------------------
                   var(Residual) |   7.128148   1.301701      4.983509    10.19573
    ------------------------------------------------------------------------------
    LR test vs. linear model: chi2(2) = 17.59                 Prob > chi2 = 0.0002
    
    Note: LR test is conservative and provided only for reference.
    In this case, given the fixed variable for time was not statistically significant (p=0.89), I can conclude there was no (statistically) significant change over time in the outcome variable.


    Does this sound correct?


    Any comments much appreciated.


    Thanks!

    MJ


  • #2
    Originally posted by MJ Smith View Post
    Each person was assessed at 5 time points. . . . I can conclude there was no (statistically) significant change over time in the outcome variable. Does this sound correct?
    No.

    .ÿ
    .ÿversionÿ15.1

    .ÿ
    .ÿclearÿ*

    .ÿ
    .ÿsetÿseedÿ`=strreverse("1493599")'

    .ÿ
    .ÿquietlyÿsetÿobsÿ15

    .ÿgenerateÿbyteÿpidÿ=ÿ_n

    .ÿgenerateÿdoubleÿpid_uÿ=ÿrnormal()

    .ÿ
    .ÿquietlyÿexpandÿ5

    .ÿbysortÿpid:ÿgenerateÿbyteÿtimÿ=ÿ_n

    .ÿ
    .ÿgenerateÿdoubleÿrspÿ=ÿpid_uÿ+ÿinlist(tim,ÿ2,ÿ4)ÿ/ÿ2ÿ+ÿ(timÿ==ÿ3)ÿ+ÿrnormal()

    .ÿ
    .ÿmixedÿrspÿtimÿ||ÿpid:ÿtim,ÿnolrtestÿnolog

    Mixed-effectsÿMLÿregressionÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿÿÿÿÿ=ÿÿÿÿÿÿÿÿÿ75
    Groupÿvariable:ÿpidÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿgroupsÿÿ=ÿÿÿÿÿÿÿÿÿ15

    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿObsÿperÿgroup:
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿminÿ=ÿÿÿÿÿÿÿÿÿÿ5
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿavgÿ=ÿÿÿÿÿÿÿÿ5.0
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿmaxÿ=ÿÿÿÿÿÿÿÿÿÿ5

    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿWaldÿchi2(1)ÿÿÿÿÿÿ=ÿÿÿÿÿÿÿ0.89
    Logÿlikelihoodÿ=ÿ-124.86979ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿProbÿ>ÿchi2ÿÿÿÿÿÿÿ=ÿÿÿÿÿ0.3454

    ------------------------------------------------------------------------------
    ÿÿÿÿÿÿÿÿÿrspÿ|ÿÿÿÿÿÿCoef.ÿÿÿStd.ÿErr.ÿÿÿÿÿÿzÿÿÿÿP>|z|ÿÿÿÿÿ[95%ÿConf.ÿInterval]
    -------------+----------------------------------------------------------------
    ÿÿÿÿÿÿÿÿÿtimÿ|ÿÿÿ.0828814ÿÿÿ.0878326ÿÿÿÿÿ0.94ÿÿÿ0.345ÿÿÿÿ-.0892673ÿÿÿÿ.2550302
    ÿÿÿÿÿÿÿ_consÿ|ÿÿÿ.5938504ÿÿÿ.3955487ÿÿÿÿÿ1.50ÿÿÿ0.133ÿÿÿÿ-.1814107ÿÿÿÿ1.369112
    ------------------------------------------------------------------------------

    ------------------------------------------------------------------------------
    ÿÿRandom-effectsÿParametersÿÿ|ÿÿÿEstimateÿÿÿStd.ÿErr.ÿÿÿÿÿ[95%ÿConf.ÿInterval]
    -----------------------------+------------------------------------------------
    pid:ÿIndependentÿÿÿÿÿÿÿÿÿÿÿÿÿ|
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(tim)ÿ|ÿÿÿ3.03e-26ÿÿÿ4.02e-25ÿÿÿÿÿÿ1.57e-37ÿÿÿÿ5.85e-15
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(_cons)ÿ|ÿÿÿ1.073977ÿÿÿ.4785401ÿÿÿÿÿÿÿ.448455ÿÿÿÿ2.572002
    -----------------------------+------------------------------------------------
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(Residual)ÿ|ÿÿÿ1.157185ÿÿÿ.2112743ÿÿÿÿÿÿ.8090845ÿÿÿÿ1.655053
    ------------------------------------------------------------------------------

    .ÿ
    .ÿanovaÿrspÿtimÿpid

    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿ=ÿÿÿÿÿÿÿÿÿ75ÿÿÿÿR-squaredÿÿÿÿÿ=ÿÿ0.7095
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿRootÿMSEÿÿÿÿÿÿ=ÿÿÿÿ.934496ÿÿÿÿAdjÿR-squaredÿ=ÿÿ0.6162

    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿSourceÿ|ÿPartialÿSSÿÿÿÿÿÿÿÿÿdfÿÿÿÿÿÿÿÿÿMSÿÿÿÿÿÿÿÿFÿÿÿÿProb>F
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ-----------+----------------------------------------------------
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿModelÿ|ÿÿ119.46374ÿÿÿÿÿÿÿÿÿ18ÿÿÿ6.6368746ÿÿÿÿÿÿ7.60ÿÿ0.0000
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿtimÿ|ÿÿÿ21.55768ÿÿÿÿÿÿÿÿÿÿ4ÿÿÿÿÿ5.38942ÿÿÿÿÿÿ6.17ÿÿ0.0003
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿpidÿ|ÿÿ97.906064ÿÿÿÿÿÿÿÿÿ14ÿÿÿ6.9932903ÿÿÿÿÿÿ8.01ÿÿ0.0000
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿResidualÿ|ÿÿ48.903835ÿÿÿÿÿÿÿÿÿ56ÿÿÿ.87328277ÿÿ
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ-----------+----------------------------------------------------
    ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿTotalÿ|ÿÿ168.36758ÿÿÿÿÿÿÿÿÿ74ÿÿÿ2.2752375ÿÿ

    .ÿ
    .ÿcontrastÿq.tim,ÿnoeffects

    Contrastsÿofÿmarginalÿlinearÿpredictions

    Marginsÿÿÿÿÿÿ:ÿasbalanced

    ------------------------------------------------
    ÿÿÿÿÿÿÿÿÿÿÿÿÿ|ÿÿÿÿÿÿÿÿÿdfÿÿÿÿÿÿÿÿÿÿÿFÿÿÿÿÿÿÿÿP>F
    -------------+----------------------------------
    ÿÿÿÿÿÿÿÿÿtimÿ|
    ÿÿÿ(linear)ÿÿ|ÿÿÿÿÿÿÿÿÿÿ1ÿÿÿÿÿÿÿÿ1.18ÿÿÿÿÿ0.2820
    (quadratic)ÿÿ|ÿÿÿÿÿÿÿÿÿÿ1ÿÿÿÿÿÿÿ21.94ÿÿÿÿÿ0.0000
    ÿÿÿÿ(cubic)ÿÿ|ÿÿÿÿÿÿÿÿÿÿ1ÿÿÿÿÿÿÿÿ0.98ÿÿÿÿÿ0.3256
    ÿÿ(quartic)ÿÿ|ÿÿÿÿÿÿÿÿÿÿ1ÿÿÿÿÿÿÿÿ0.58ÿÿÿÿÿ0.4485
    ÿÿÿÿÿÿJointÿÿ|ÿÿÿÿÿÿÿÿÿÿ4ÿÿÿÿÿÿÿÿ6.17ÿÿÿÿÿ0.0003
    ÿÿÿÿÿÿÿÿÿÿÿÿÿ|
    ÿDenominatorÿ|ÿÿÿÿÿÿÿÿÿ56
    ------------------------------------------------

    .ÿ
    .ÿexit

    endÿofÿdo-file


    .


    There doesn't seem to be a linear trend over time, but that doesn't preclude other changes over the five time points.

    Comment


    • #3
      Wonderful! Thank you so much, very helpful!!

      Comment

      Working...
      X