Alternatives for Repeated Measures ANOVA

[Albert Henry]

Dear Statalisters,

I am fairly new to STATA and I am currently working on a large dataset with around 500 variables and 450,000 observations. From this dataset, I need to extract two variables, let's say, X₁ (N = ~450,000) which contains information on 1^st test results and X₂ (N = ~20,000) which contains information on 2^nd (repeated) test results. I want to run an unadjusted Repeated Measures ANOVA using these two variables to estimate the standard error measurement, which equals to the square root of mean square residual (RMSE) from the ANOVA statistics. I had a look at STATA documentation on ANOVA ( http://www.stata.com/manuals13/ranova.pdf ), but from my understanding, it involves reshaping the data into a long format, which I think may not be effective, given the size of the dataset I have.

My question is: Is there any other alternatives to do this more efficiently? I was thinking of using perhaps MANOVA or linear regression, but is it possible to obtain the RMSE value equivalent to the one from repeated measures ANOVA output?

Thanks in advance for your help.

Ps: I am using STATA 13.0

[Phil Bromiley]

You didn't get a quick answer. You'll increase your chances of a useful answer by following the FAQ on asking questions - provide Stata code in code delimiters, Stata output, and sample data using dataex.

I don't fully understand the structure of you data or the details of what you want to run. However, if reshaping the data to be long makes things easier, I'd do it regardless of the number of observations. [450,000 is not that many. If you have time issues, then drop the variables you don't need before reshaping.] Work with a small portion of the data when you're figuring out how to do the reshaping. Once you've got it right, just run it on the larger dataset. Speed of course would depend on your machine and version of Stata, but I can't imagine it would take that long. Even if it did, just run it overnight.

When I have a question like "is it possible to obtain the RMSE value equivalent to the one from repeated measures ANOVA output?" I'd try it out on a sample data set and see what I get. Create a 30 observation data set and see if the RMSEs are identical. There are regressions and panel regressions that are identical to most ANOVA routines.

[Albert Henry]

Dear Phil,

Thanks for your advice. I didn't include the code, output, and sample data because I only have access to STATA through remote desktop which blocks all internet access for confidentiality and security reason (apologies for this). However, I managed to try it on smaller dataset as you suggested and I have now figured out the problem.

Many thanks,

Albert