Can (should) the confidence interval on the RMSEA be changed?

[Karen Jackson]

Hi all,

For a recent paper, I ran a pretty standard SEM model in Stata 14 and used the estat gof, stats(all) command to get the fit indices. I reported (along with other fit indices) the RMSEA with the 90% confidence intervals. The paper has received a revise and resubmit, but the editor asked that I give the 95% confidence intervals instead. Is there a way to get this through Stata? Or should I just calculate myself? I am also wondering about the appropriateness of this, considering that when I see the RMSEA CI's given in papers, the usual is for 90% and the references I've found (Kline, 2005; Kenny, Kaniskan, & McCoach, 2015) all use 90%, not 95%. Advice?

Thanks.

[Bruce Weaver]

It sounds as though the editor may be just thinking that 95% CIs are always standard, and is not aware of the convention you find in the literature for this particular situation. If the journal accepts footnotes, I think I would report the 90% CI in the body of the text, but also report the 95% CI in the footnote, citing the relevant literature, and explaining briefly that even though 95% CIs are routine for many situations, in this particular situation, it is more conventional to set the confidence level to 90%. And if they don't buy that approach, I'd try it the other way around (95% in the body, 90% in the footnote!)

HTH.

[Rich Goldstein]

you can get this via Stata through the "level" option to your command (alternatively, you can use "set level")

[Karen Jackson]

Thanks for the responses. I think I might have to go with the footnote idea, Bruce. The level command option only changes the CI levels for the parameters from the SEM command, but not for the RMSEA CI estimates in the estat gof command, Rich. Maybe I'm missing something there.

I just never see a 95% CI for RMSEA. I wonder why RMSEA uses the 90% CI when 95% is more common. Something to do with a fit statistic versus a parameter estimate?

I just found this in case anyone else has this problem:
Curran, Bollen, Chen, Paxton & Kirby (2003) ""Widespread use of the 90 percent CI is based on the ability to link this CI to the use of the usual likelihood ratio test T . Specifically, if the lower bound of the 90 percent CI is equal to zero, this implies that the probability level of the test statistic T is greater than .05. Thus, the CI provides information about the likelihood ratio test statistic but also provides much more information about model fit beyond the standard p value (for further details, see MacCallum et al. 1996)." pg. 243.

For future reference, in an MPlus discussion, someone recommended using R and the MBESS R package with the ci.rmsea function. I might have to try that, but R is not my strength. I would rather stay in Stata and get the numbers.

Thanks.