Testing Simple Effects for mixed models

[Jason Levitt]

Hello!

I am using Stata version 14. I have a mixed model with the following code: mixed dependentvar i.time##diagnosis##condition || id: time.

I have been looking at the simple effects following the instructions here: http://stats.idre.ucla.edu/stata/faq...dels-stata-12/

Everything is looking clear until I get to the end, where the article states that "Test of simple effects are a type of post-hoc procedure and need to be adjusted. We won't go into the adjustment process on this page other than to state that there are at least four methods found in the literature (Dunn's procedure, Marascuilo & Levin, per family error rate or simultaneous test procedure)."

I have no idea how to account for the adjustments here. I would greatly appreciate input on how to do this. Thanks in advance!

[Joseph Coveney]

If your research question implies a specific hypothesis that is tested in the three-way interaction term, then don't adjust at all; it's not post hoc. If you're just sifting through the interaction terms, then you could do something like the following (start at the "Begin here" comment).

.ÿversionÿ14.2

.ÿ
.ÿclearÿ*

.ÿsetÿmoreÿoff

.ÿsetÿseedÿ1386466

.ÿ
.ÿquietlyÿsetÿobsÿ240

.ÿgenerateÿintÿpidÿ=ÿ_n

.ÿgenerateÿdoubleÿuÿ=ÿrnormal()

.ÿ
.ÿforvaluesÿtimÿ=ÿ1/3ÿ{
ÿÿ2.ÿÿÿÿÿÿÿÿÿlocalÿvarlist1ÿ`varlist1'ÿocm`tim'
ÿÿ3.ÿÿÿÿÿÿÿÿÿforvaluesÿdigÿ=ÿ0/1ÿ{
ÿÿ4.ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿlocalÿvarlist2ÿ`varlist2'ÿocm`tim'`dig'
ÿÿ5.ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿforvaluesÿcdtÿ=ÿ0/1ÿ{
ÿÿ6.ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿgenerateÿdoubleÿocm`tim'`dig'`cdt'ÿ=ÿuÿ+ÿrnormal()
ÿÿ7.ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ}
ÿÿ8.ÿÿÿÿÿÿÿÿÿ}
ÿÿ9.ÿ}

.ÿquietlyÿreshapeÿlongÿ`varlist2',ÿi(pid)ÿj(cdt)

.ÿquietlyÿreshapeÿlongÿ`varlist1',ÿi(pidÿcdt)ÿj(dig)

.ÿquietlyÿreshapeÿlongÿocm,ÿi(pidÿcdtÿdig)ÿj(tim)

.ÿ
.ÿlabelÿdefineÿDxÿ0ÿNegativeÿ1ÿPositive

.ÿlabelÿvaluesÿdigÿDx

.ÿ
.ÿlabelÿdefineÿConditionsÿ0ÿBadÿ1ÿGood

.ÿlabelÿvaluesÿcdtÿConditions

.ÿ
.ÿmixedÿocmÿi.(tim##dig##cdt)ÿ||ÿpid:ÿ,ÿnostderrÿnolrtestÿnolog

Mixed-effectsÿMLÿregressionÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿÿÿÿÿ=ÿÿÿÿÿÿ2,880
Groupÿvariable:ÿpidÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿgroupsÿÿ=ÿÿÿÿÿÿÿÿ240

ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿObsÿperÿgroup:
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿminÿ=ÿÿÿÿÿÿÿÿÿ12
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿavgÿ=ÿÿÿÿÿÿÿ12.0
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿmaxÿ=ÿÿÿÿÿÿÿÿÿ12

ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿWaldÿchi2(11)ÿÿÿÿÿ=ÿÿÿÿÿÿÿ9.72
Logÿlikelihoodÿ=ÿ-4342.7501ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿProbÿ>ÿchi2ÿÿÿÿÿÿÿ=ÿÿÿÿÿ0.5561

----------------------------------------------------------------------------------
ÿÿÿÿÿÿÿÿÿÿÿÿÿocmÿ|ÿÿÿÿÿÿCoef.ÿÿÿStd.ÿErr.ÿÿÿÿÿÿzÿÿÿÿP>|z|ÿÿÿÿÿ[95%ÿConf.ÿInterval]
-----------------+----------------------------------------------------------------
ÿÿÿÿÿÿÿÿÿÿÿÿÿtimÿ|
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ2ÿÿ|ÿÿÿ.0044072ÿÿÿ.0901786ÿÿÿÿÿ0.05ÿÿÿ0.961ÿÿÿÿ-.1723396ÿÿÿÿ.1811541
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ3ÿÿ|ÿÿ-.0000286ÿÿÿ.0901786ÿÿÿÿ-0.00ÿÿÿ1.000ÿÿÿÿ-.1767754ÿÿÿÿ.1767183
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿÿÿÿÿÿÿÿÿdigÿ|
ÿÿÿÿÿÿÿPositiveÿÿ|ÿÿÿ.0273655ÿÿÿ.0901786ÿÿÿÿÿ0.30ÿÿÿ0.762ÿÿÿÿ-.1493813ÿÿÿÿ.2041124
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿÿÿÿÿtim#digÿ|
ÿÿÿÿÿ2#Positiveÿÿ|ÿÿ-.0948393ÿÿÿ.1275318ÿÿÿÿ-0.74ÿÿÿ0.457ÿÿÿÿ-.3447971ÿÿÿÿ.1551185
ÿÿÿÿÿ3#Positiveÿÿ|ÿÿ-.0813853ÿÿÿ.1275318ÿÿÿÿ-0.64ÿÿÿ0.523ÿÿÿÿ-.3313431ÿÿÿÿ.1685725
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿÿÿÿÿÿÿÿÿcdtÿ|
ÿÿÿÿÿÿÿÿÿÿÿGoodÿÿ|ÿÿ-.1218816ÿÿÿ.0901786ÿÿÿÿ-1.35ÿÿÿ0.177ÿÿÿÿ-.2986285ÿÿÿÿ.0548653
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿÿÿÿÿtim#cdtÿ|
ÿÿÿÿÿÿÿÿÿ2#Goodÿÿ|ÿÿÿ.1126576ÿÿÿ.1275318ÿÿÿÿÿ0.88ÿÿÿ0.377ÿÿÿÿ-.1373002ÿÿÿÿ.3626154
ÿÿÿÿÿÿÿÿÿ3#Goodÿÿ|ÿÿÿ.1935147ÿÿÿ.1275318ÿÿÿÿÿ1.52ÿÿÿ0.129ÿÿÿÿ-.0564431ÿÿÿÿ.4434725
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿÿÿÿÿdig#cdtÿ|
ÿÿPositive#Goodÿÿ|ÿÿ-.0124939ÿÿÿ.1275318ÿÿÿÿ-0.10ÿÿÿ0.922ÿÿÿÿ-.2624517ÿÿÿÿ.2374639
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿtim#dig#cdtÿ|
2#Positive#Goodÿÿ|ÿÿÿ.0145943ÿÿÿ.1803573ÿÿÿÿÿ0.08ÿÿÿ0.936ÿÿÿÿ-.3388995ÿÿÿÿÿ.368088
3#Positive#Goodÿÿ|ÿÿÿ.0371053ÿÿÿ.1803573ÿÿÿÿÿ0.21ÿÿÿ0.837ÿÿÿÿ-.3163884ÿÿÿÿÿ.390599
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿÿÿÿÿÿÿ_consÿ|ÿÿ-.1008909ÿÿÿ.0870025ÿÿÿÿ-1.16ÿÿÿ0.246ÿÿÿÿ-.2714128ÿÿÿÿ.0696309
----------------------------------------------------------------------------------

------------------------------------------------------------------------------
ÿÿRandom-effectsÿParametersÿÿ|ÿÿÿEstimateÿÿÿStd.ÿErr.ÿÿÿÿÿ[95%ÿConf.ÿInterval]
-----------------------------+------------------------------------------------
pid:ÿIdentityÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(_cons)ÿ|ÿÿÿÿ.840804ÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿ.
-----------------------------+------------------------------------------------
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(Residual)ÿ|ÿÿÿ.9758621ÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿÿÿ.ÿÿÿÿÿÿÿÿÿÿÿ.
------------------------------------------------------------------------------

.ÿ
.ÿ*
.ÿ*ÿBeginÿhere
.ÿ*
.ÿcontrastÿtim#dig#cdtÿdig#cdtÿtim#digÿtim#cdt,ÿnoeffects

Contrastsÿofÿmarginalÿlinearÿpredictions

Marginsÿÿÿÿÿÿ:ÿasbalanced

------------------------------------------------
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|ÿÿÿÿÿÿÿÿÿdfÿÿÿÿÿÿÿÿchi2ÿÿÿÿÿP>chi2
-------------+----------------------------------
ocmÿÿÿÿÿÿÿÿÿÿ|
ÿtim#dig#cdtÿ|ÿÿÿÿÿÿÿÿÿÿ2ÿÿÿÿÿÿÿÿ0.04ÿÿÿÿÿ0.9787
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿdig#cdtÿ|ÿÿÿÿÿÿÿÿÿÿ1ÿÿÿÿÿÿÿÿ0.00ÿÿÿÿÿ0.9487
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿtim#digÿ|ÿÿÿÿÿÿÿÿÿÿ2ÿÿÿÿÿÿÿÿ1.00ÿÿÿÿÿ0.6059
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿtim#cdtÿ|ÿÿÿÿÿÿÿÿÿÿ2ÿÿÿÿÿÿÿÿ5.56ÿÿÿÿÿ0.0620
------------------------------------------------

.ÿ
.ÿ//ÿGetÿnamesÿofÿcontrasts
.ÿforvaluesÿiÿ=ÿ1/`:ÿwordÿcountÿ`r(cmdline)''ÿ{
ÿÿ2.ÿÿÿÿÿÿÿÿÿlocalÿwordÿ:ÿwordÿ`i'ÿofÿ`r(cmdline)'
ÿÿ3.ÿÿÿÿÿÿÿÿÿlocalÿwordÿ:ÿsubinstrÿlocalÿwordÿ","ÿ"",ÿall
ÿÿ4.ÿÿÿÿÿÿÿÿÿifÿstrpos("`word'",ÿ"#")ÿ>ÿ0ÿlocalÿcontrastsÿ`contrasts'ÿ`word'
ÿÿ5.ÿ}

.ÿ
.ÿ//ÿGetÿmatrixÿofÿp-values
.ÿmatrixÿdefineÿPÿ=ÿr(p)

.ÿ
.ÿ//ÿAdjustÿandÿdisplay
.ÿforvaluesÿiÿ=ÿ1/`=colsof(P)'ÿ{
ÿÿ2.ÿÿÿÿÿÿÿÿÿifÿ`i'ÿ==ÿ1ÿdisplayÿinÿsmclÿasÿtextÿ"Contrast"ÿ_column(15)ÿ"AdjustedÿP-value"
ÿÿ3.ÿÿÿÿÿÿÿÿÿdisplayÿinÿsmclÿasÿresultÿ"`:ÿwordÿ`i'ÿofÿ`contrasts''"ÿ_column(20)ÿ%04.2fÿmin(P[1,ÿ`i']ÿ*ÿ`=colsof(P)',ÿ1)
ÿÿ4.ÿ}
ContrastÿÿÿÿÿÿAdjustedÿP-value
tim#dig#cdtÿÿÿÿÿÿÿÿ1.00
dig#cdtÿÿÿÿÿÿÿÿÿÿÿÿ1.00
tim#digÿÿÿÿÿÿÿÿÿÿÿÿ1.00
tim#cdtÿÿÿÿÿÿÿÿÿÿÿÿ0.25

.ÿ
.ÿexit

endÿofÿdo-file


.

By the way, your model set up treats time as categorical in the fixed effects equation but at continuous in the random effects equation.

[Jason Levitt]

Joseph,

Thank you so much for the reply. Shortly after I posted I actually realized that time was a continous variable in the random effects equation. In this data set, time is indeed a factorial variable. However, when I changed it to R.time (so the model was "mixed dependentvar i.time##diagnosis##condition || id: R.time") I got the following error:


Performing gradient-based optimization:

Iteration 0: log likelihood = -1772.4501 (not concave)
could not calculate numerical derivatives -- discontinuous region with missing values encountered
could not calculate numerical derivatives -- discontinuous region with missing values encountered

Computing standard errors:
standard-error calculation failed
estimates post: matrix has missing values
r(504);


I have no idea what is going on. It worked fine with the random effect of time as a continuous variable, but not when I changed it to a categorical one. I would greatly appreciate any help.

[Joseph Coveney]

You could try

mixed dependentvar i.time##i.diagnosis##i.condition || _all: R.time || id:

The variance component for time could turn out to be nearly zero.

[Jason Levitt]

You could try

mixed dependentvar i.time##i.diagnosis##i.condition || _all: R.time || id:

The variance component for time could turn out to be nearly zero.

Oh my gosh. I have been trying to figure out that code for so long. Thank you so much for the help, it worked! You were right, the variance component for time was nearly zero. What do you think is the best way to interpret this?