Repeated measure ANOVA

[Cloudy Tech]

Hi,

I have cross-sectional datasets for 3 years: 2015, 2018, 2021.

I calculated an Index for 3 years for each individuals. The index is a continuous variable.

My question is can I do repeated measure ANOVA to compare among age groups - use age group as a repeated measure in repeated measure ANOVA analysis. The reviewer of my paper said that I can only do this with same individuals over years. But my data is only pooled cross sectional over 3 years. My repeated measures is not individuals, I used repeated demographic groups – such as age, gender, race, education, marital status, income, and employment status acrosss over different years, and difference states. For example, age group would be the subject or unit of analysis, and I tracked the Index scores of age groups over different years or different states. Since my research focused on demographic differences. This method helps me to detect whether changes in FHI scores over time are consistent across different age groups and provided insights into temporal patterns in the index across the population segments.

Can I do it? Or I only can use repeated measure ANOVA when I have longitudinal data for individuals? Can I treated age group as a subject here?

Thanks,

[Erik Ruzek]

You do have repeated measures, but it is repeated measures of different individuals from the same states. The ANOVA model the reviewer is calling you out on is not appropriate for this data. Instead, you need to account for the state an individual belongs to in your modeling. This may also involve using weights provided by the database creator.

From your description, it is not clear what you mean by repeated demographic groups in regards to age. Is it that you have categorized individuals to age buckets (20-30, 31-40, 41-50) or something different? Assuming that you have age buckets (cohorts), and that you have questions about trends for the different demographic groups, I would utilize a panel data model in this analysis, whether fixed or random effects.

Below, I simulate a toy dataset that I assume your data looks like, and I show the model I would use to investigate trends.

version 16.1
clear
set seed 63451

set obs 50
gen state = _n
gen u_i = rnormal(0,2)    // random intercept w/ associated variability (SD=2)

* three time points per
expand 3
bysort state: gen year = _n
sort state year

* create 10 individuals per state year
expand 10
sort state year
gen id = _n
gen random = rnormal(0,1) // for purposes of creating age categories
gen age_cohort = 0 if random < -1.31
replace age_cohort = 1 if random >= -1.31 & random < 1.21
replace age_cohort = 2 if random >= 1.21
drop random
gen female = rbinomial(1,0.6)    // assume more females than males


* create outcome as function of year, female, age_cohort, random state intercept, and residual
gen fem_year = female*year
gen age_year = age_cohort*year
gen FHI = 100 + 2.5*year + -1*female + -3*fem_year + ///
    3*age_cohort + .5*age_year + u_i + rnormal(0, 15)

* Analytical model - random effects of states
mixed FHI i.female##i.year i.age_cohort##i.year || state:, stddev

* Model-based predictions of the female and age_cohort trends over the three years of data
margins female#year
marginsplot, xdimension(year)

margins age_cohort#year
marginsplot, xdimension(year)

[Cloudy Tech]

Hi Erik,

Thanks so much! Yes my age groups are categorized in buckets (18- 25, 26 - 30, 31 - 35,....) for 3 years. I have same variables for 3 cross-sectional datasets: age groups, gender, ...state, year. I want to compare the differences among different groups. I used one-way ANOVA and the Kruskal-Wallis’s test and then did the repeated measure ANOVA for robustness. Thanks for the suggestion about the trend analysis.

My best,

[Cloudy Tech]

mixed FHI_PPCA i.gender_numeric##i.tracknew i.A3Ar_w##i.tracknew || stateq:, stddev

Performing EM optimization ...

Performing gradient-based optimization:
Iteration 0: Log likelihood = 1197.2382
Iteration 1: Log likelihood = 1197.2382

Computing standard errors ...

Mixed-effects ML regression Number of obs = 68,171
Group variable: stateq Number of groups = 51
Obs per group:
min = 1,199
avg = 1,336.7
max = 2,511
Wald chi2(20) = 5409.03
Log likelihood = 1197.2382 Prob > chi2 = 0.0000

-----------------------------------------------------------------------------------------
FHI_PPCA | Coefficient Std. err. z P>|z| [95% conf. interval]
------------------------+----------------------------------------------------------------
gender_numeric |
Female | -.0417793 .0031319 -13.34 0.000 -.0479177 -.0356409
|
tracknew |
2 | .0038118 .0084359 0.45 0.651 -.0127221 .0203458
3 | -.00096 .0082969 -0.12 0.908 -.0172217 .0153018
|
gender_numeric#tracknew |
Female#2 | -.0050051 .0044601 -1.12 0.262 -.0137468 .0037366
Female#3 | -.0109561 .00447 -2.45 0.014 -.0197171 -.0021952
|
A3Ar_w |
25-34 | .0190869 .0066019 2.89 0.004 .0061474 .0320264
35-44 | .0343145 .0066618 5.15 0.000 .0212577 .0473713
45-54 | .0494749 .0065611 7.54 0.000 .0366154 .0623343
55-64 | .0839686 .0065908 12.74 0.000 .0710508 .0968863
65+ | .1436994 .0065209 22.04 0.000 .1309186 .1564803
|
A3Ar_w#tracknew |
25-34#2 | -.0306321 .0095986 -3.19 0.001 -.049445 -.0118191
25-34#3 | -.0182994 .0095492 -1.92 0.055 -.0370155 .0004167
35-44#2 | -.0230918 .0096234 -2.40 0.016 -.0419534 -.0042303
35-44#3 | -.0033359 .0095627 -0.35 0.727 -.0220784 .0154067
45-54#2 | -.006617 .0095123 -0.70 0.487 -.0252608 .0120269
45-54#3 | .0120903 .0094539 1.28 0.201 -.0064391 .0306196
55-64#2 | .0306509 .0094786 3.23 0.001 .0120732 .0492287
55-64#3 | .0405903 .0094431 4.30 0.000 .0220822 .0590984
65+#2 | .0147212 .00935 1.57 0.115 -.0036044 .0330468
65+#3 | .0501554 .0092749 5.41 0.000 .0319769 .068334
|
_cons | .6322283 .0067659 93.44 0.000 .6189674 .6454891
-----------------------------------------------------------------------------------------

------------------------------------------------------------------------------
Random-effects parameters | Estimate Std. err. [95% conf. interval]
-----------------------------+------------------------------------------------
stateq: Identity |
sd(_cons) | .0246065 .0026157 .0199787 .0303062
-----------------------------+------------------------------------------------
sd(Residual) | .2375165 .0006435 .2362586 .2387811
------------------------------------------------------------------------------
LR test vs. linear model: chibar2(01) = 556.35 Prob >= chibar2 = 0.0000

. margins gender_numeric#tracknew

Predictive margins Number of obs = 68,171

Expression: Linear prediction, fixed portion, predict()

-----------------------------------------------------------------------------------------
| Delta-method
| Margin std. err. z P>|z| [95% conf. interval]
------------------------+----------------------------------------------------------------
gender_numeric#tracknew |
Male#1 | .6965923 .0041526 167.75 0.000 .6884534 .7047312
Male#2 | .6989493 .0041795 167.23 0.000 .6907576 .7071409
Male#3 | .7124283 .0041548 171.47 0.000 .704285 .7205717
Female#1 | .654813 .0040455 161.86 0.000 .646884 .662742
Female#2 | .6521649 .0040558 160.80 0.000 .6442156 .6601142
Female#3 | .6596929 .0040915 161.23 0.000 .6516737 .6677122
-----------------------------------------------------------------------------------------

I got this results - tracknew (year), A3Ar_w (age groups).