Problem with comparing first order CFA model with a second order CFA model

[Anna Skubel]

I am having an issue comparing a first order factor model with a second order factor model. I was able to make each model in the SEM builder, but there is no difference in chi-square df between the models, thus indicating that something is not working correctly, as the higher order model should have one less df. The goodness of fit indices are the exact same between the first order model (where the latent factors are allowed to covary freely) and a second order model (where a higher order latent factor is introduced that might explain the significant covariances between the first level latent factors). Has this happened to anyone- if so, how did you troubleshoot?

[Joseph Coveney]

Wouldn't an equivalent second-order CFA—where you just substitute the second-order factor for the covariance term between the two-first order factors—have the same model degrees of freedom and the same log-likelihood as the first-order CFA?

You can run the following in order to see what I mean.

version 17.0

clear *

// seedem
set seed 1580253197

tempname On Off Corr
matrix define `On' = J(3, 3, 0.75) + I(3) * 0.25
matrix define `Off' = J(3, 3, 0.25)
matrix define `Corr' = (`On', `Off') \ (`Off', `On')

forvalues i = 1/6 {
    local varlist `varlist' y`i'
}
quietly drawnorm `varlist', double corr(`Corr') n(250)

program define showEm
    version 17.0
    syntax

    display in smcl as text "df = " as result e(df_m), ///
        as text "log-likelihood = " as result e(ll)
end

*
* Begin here
*
sem ///
    (y1-y3 <- F1) ///
    (y4-y6 <- F2), ///
        covariance(F1*F2) ///
            nocnsreport nodescribe nolog
showEm

sem ///
    (y1-y3 <- F1) ///
    (y4-y6 <- F2) ///
        (F1@1 F2@1 <- F3), ///
            nocnsreport nodescribe nolog
showEm

exit

That's with two first-order latent factors. I don't get much more complicated than that in practice, but I think that it would generalize.