Repeated Measures ANOVA

[Victor Smith]

Hi, I would like to know how to do the repeated measures ANOVA analysis with the following data - I did a longitudinal experiment on several students. I first divided them into two groups - one is the treatment group (treatment = 1), the other is the control group (treatment = 0). Then I collected their writing scores in 4 waves (meaning, I measured their writing scores four times in the longitudinal experiment which lasted around 1 week).

I want to know if the treatment influences their scores. But I don't know how to do the repeated measures ANOVA analysis in stata? Can anyone help me out?

clear
input student treatment wave score
1 0 1 12
1 0 2 4
1 0 3 1
1 0 4 6
2 0 1 7
2 0 2 1
2 0 3 6
2 0 4 12
3 0 1 6
3 0 2 12
3 0 3 4
3 0 4 2
4 0 1 2
4 0 2 8
4 0 3 0
4 0 4 4
5 0 1 0
5 0 2 6
5 0 3 0
5 0 4 4
6 0 1 3
6 0 2 4
6 0 3 4
6 0 4 2
7 0 1 3
7 0 2 4
7 0 3 0
7 0 4 3
8 0 1 4
8 0 2 4
8 0 3 2
8 0 4 4
9 0 1 6
9 0 2 3
9 0 3 3
9 0 4 2
10 0 1 3
10 0 2 3
10 0 3 4
10 0 4 4
11 0 1 1
11 0 2 4
11 0 3 0
11 0 4 4
12 1 1 2
12 1 2 9
12 1 3 0
12 1 4 7
13 1 1 4
13 1 2 0
13 1 3 4
13 1 4 9
14 1 1 0
14 1 2 7
14 1 3 0
14 1 4 9
15 1 1 8
15 1 2 6
15 1 3 3
15 1 4 9
16 1 1 9
16 1 2 0
16 1 3 0
16 1 4 0
17 1 1 0
17 1 2 1
17 1 3 4
17 1 4 0
18 1 1 0
18 1 2 8
18 1 3 9
18 1 4 9
19 1 1 9
19 1 2 0
19 1 3 2
19 1 4 0
20 1 1 7
20 1 2 0
20 1 3 7
20 1 4 7
21 1 1 9
21 1 2 0
21 1 3 0
21 1 4 1
22 1 1 4
22 1 2 1
22 1 3 7
22 1 4 10
23 1 1 2
23 1 2 1
23 1 3 2
23 1 4 4
24 1 1 1
24 1 2 3
24 1 3 3
24 1 4 9
end

[Carlo Lazzaro]

Victor:
you can give -xtreg- a go:

. xtset student wave
       panel variable:  student (strongly balanced)
        time variable:  wave, 1 to 4
                delta:  1 unit

. xtreg score i.treatment i.wave, fe
note: 1.treatment omitted because of collinearity

Fixed-effects (within) regression               Number of obs     =         96
Group variable: student                         Number of groups  =         24

R-sq:                                           Obs per group:
     within  = 0.0838                                         min =          4
     between = 0.0000                                         avg =        4.0
     overall = 0.0658                                         max =          4

                                                F(3,69)           =       2.10
corr(u_i, Xb)  = -0.0000                        Prob > F          =     0.1077

------------------------------------------------------------------------------
       score |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
 1.treatment |          0  (omitted)
             |
        wave |
          2  |  -.5416667   .9553027    -0.57   0.573    -2.447443     1.36411
          3  |  -1.541667   .9553027    -1.61   0.111    -3.447443      .36411
          4  |   .7916667   .9553027     0.83   0.410     -1.11411    2.697443
             |
       _cons |       4.25    .675501     6.29   0.000     2.902412    5.597588
-------------+----------------------------------------------------------------
     sigma_u |  1.5664262
     sigma_e |  3.3092655
         rho |  .18304395   (fraction of variance due to u_i)
------------------------------------------------------------------------------
F test that all u_i=0: F(23, 69) = 0.90                      Prob > F = 0.6025

. xtreg score i.treatment i.wave, re

Random-effects GLS regression                   Number of obs     =         96
Group variable: student                         Number of groups  =         24

R-sq:                                           Obs per group:
     within  = 0.0838                                         min =          4
     between = 0.0006                                         avg =        4.0
     overall = 0.0659                                         max =          4

                                                Wald chi2(4)      =       6.42
corr(u_i, X)   = 0 (assumed)                    Prob > chi2       =     0.1698

------------------------------------------------------------------------------
       score |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
 1.treatment |   .0751748   .6726275     0.11   0.911    -1.243151      1.3935
             |
        wave |
          2  |  -.5416667   .9479302    -0.57   0.568    -2.399576    1.316242
          3  |  -1.541667   .9479302    -1.63   0.104    -3.399576    .3162424
          4  |   .7916667   .9479302     0.84   0.404    -1.066242    2.649576
             |
       _cons |    4.20928   .7629085     5.52   0.000     2.714007    5.704554
-------------+----------------------------------------------------------------
     sigma_u |          0
     sigma_e |  3.3092655
         rho |          0   (fraction of variance due to u_i)
------------------------------------------------------------------------------

. xttest0

Breusch and Pagan Lagrangian multiplier test for random effects

        score[student,t] = Xb + u[student] + e[student,t]

        Estimated results:
                         |       Var     sd = sqrt(Var)
                ---------+-----------------------------
                   score |   11.05779       3.325325
                       e |   10.95124       3.309265
                       u |          0              0

        Test:   Var(u) = 0
                             chibar2(01) =     0.00
                          Prob > chibar2 =   1.0000

.

Unfortunately, due to the absence of evidence of panel-wise effect, both -fe- and -re- specification points you to a pooled OLS:

. regress score i.treatment i.wave, vce(cluster student)

Linear regression                               Number of obs     =         96
                                                F(4, 23)          =       3.28
                                                Prob > F          =     0.0287
                                                R-squared         =     0.0659
                                                Root MSE          =     3.2837

                               (Std. Err. adjusted for 24 clusters in student)
------------------------------------------------------------------------------
             |               Robust
       score |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
 1.treatment |   .0751748   .6485081     0.12   0.909    -1.266366    1.416716
             |
        wave |
          2  |  -.5416667   1.158745    -0.47   0.645    -2.938714    1.855381
          3  |  -1.541667   .9204292    -1.67   0.107    -3.445719    .3623862
          4  |   .7916667   1.110405     0.71   0.483    -1.505382    3.088715
             |
       _cons |    4.20928   .9186782     4.58   0.000      2.30885    6.109711
------------------------------------------------------------------------------

In conclusion, according to the sample you (laudably) provided there's no evidence of treatment effect or panel-wise effect

[Joseph Coveney]

But I don't know how to do the repeated measures ANOVA analysis in stata? Can anyone help me out?

In addition to Carlo's approach with -xtreg-, you can use Stata's -anova- command and fit the repeated-measures ANOVA model in a single command.

anova score treatment / student|treatment wave treatment#wave, repeated(wave)

You can also fit the model using -mixed-.

mixed score i.treatment##i.wave || student: , reml dfmethod(satterthwaite) nolog
contrast treatment wave treatment#wave, small

The likelihood-ratio test result there comes to the same conclusion that Carlo's analysis does.

Because you have more than two waves, you might want to allow for a more flexible residual covariance structure, such as you would get with MANOVA, which you can fit also with -mixed-.

mixed score i.treatment##i.wave || student: , reml dfmethod(satterthwaite) ///
    noconstant residuals(unstructured, t(wave)) nolrtest nolog
contrast treatment wave treatment#wave, small

Exploration of the residual covariance matrix would itself seem to be substantively interesting in this regard. You will also notice that now there is a statistically significant main effect of time (wave).

[Victor Smith]

Thanks Carlo and Joseph!