Mixed model overfitting? 3 time points 3 random effects.
Hello!
I'm creating an exploratory model predicting minutes of moderate-to-vigorous physical activity (mvpa) over time. The goal is to see how the "type" of social network tie (3 types) might be related to mvpa and how this relationship might differ between treatment/control and change over time.
I have 3 unevenly spaced time points (baseline, 1 years, and 3 years), that can vary individually. Data is in "long" form and a1timeyears is a variable that is 0 for everyone at baseline, but can take values for each individual participant that are roughly around the values of 1 and 3 at the subsequent time points. I plan to treat time as continuous. There are roughly 500 participants. Here is one of our proposed models:
I know I'm overusing the "c." specification, but it helps me to ensure that I've intentionally specified how every term is to be treated (given how easy it is to overlook this).
As you can see, there are random effects for the intercept, linear and quadratic growth parameters, and the random effects are allowed to covary (6 parameters). But, again, there are only 3 time points. This model successfully runs in Stata without any warnings and marginsplot produces nice quadratic mvpa curves for the treatment and control groups over time:
However, I suspect this model is overfit, even though I'm treating time as continuous.
Is there any way to confirm whether the model is overfit in Stata? Should we not present this model? If not, does anyone have recommendations on alternative approaches to take?
I have also run a more simple linear model:
This is fine, and tells a similar story, but of course the marginsplot does not capture the quadratic nature of mvpa over time.
Thank you for taking the time to read this, and for any advice you can provide!
I'm creating an exploratory model predicting minutes of moderate-to-vigorous physical activity (mvpa) over time. The goal is to see how the "type" of social network tie (3 types) might be related to mvpa and how this relationship might differ between treatment/control and change over time.
I have 3 unevenly spaced time points (baseline, 1 years, and 3 years), that can vary individually. Data is in "long" form and a1timeyears is a variable that is 0 for everyone at baseline, but can take values for each individual participant that are roughly around the values of 1 and 3 at the subsequent time points. I plan to treat time as continuous. There are roughly 500 participants. Here is one of our proposed models:
Code:
mixed a1mvpamin i.networktietype##i.trt##c.a1timeyears##c.a1timeyears c.a0age i.a1female c.a1avgtotalmin c.a1bmi /// || study_id: c.a1timeyears c.a1timeyears2, covariance(unstructured)
As you can see, there are random effects for the intercept, linear and quadratic growth parameters, and the random effects are allowed to covary (6 parameters). But, again, there are only 3 time points. This model successfully runs in Stata without any warnings and marginsplot produces nice quadratic mvpa curves for the treatment and control groups over time:
Code:
quietly margins, at(trt=(0) networktietype=(0) a1timeyears=(0(.1)3)) at(trt=(1) networktietype=(2) a1timeyears=(0(.1)3))
marginsplot, recast(line) recastci(rarea) name(typeXtrtXtime, replace) title(, size(medium))xtitle("Years in Study", size(small)) ///
ci1opts(color(midblue%50)) ci2opts(color(green%50)) plot1opts(lcolor(midblue)) plot2opts(lcolor(green)) ///
legend(order(3 "Control - No Tie" 4 "Intervention - Intervention Tie")) ytitle("Model-Estimated Adult MVPA (min)", size(small))
Is there any way to confirm whether the model is overfit in Stata? Should we not present this model? If not, does anyone have recommendations on alternative approaches to take?
I have also run a more simple linear model:
Code:
mixed a1mvpamin i.networktietype##i.trt##c.a1timeyears c.a0age i.a1female c.a1avgtotalmin c.a1bmi /// || study_id: c.a1timeyears, covariance(unstructured)
Thank you for taking the time to read this, and for any advice you can provide!
