[SEM] How to obtain post-estimation GOF stats, MLMV method

[Michael Chen]

Dear colleagues,

I'm trying to estimate the following SEM using the MLMV (full information maximum likelihood) in Stata 13.1:

#delimit ;
sem
 
(cash inkind age raceblack raceother edu_hs edu_somecol edu_colgrad -> pressure)
               
(pressure raceblack raceother edu_hs edu_somecol edu_colgrad income -> depress)
 
(depress age edu_hs edu_somecol edu_colgrad income -> Withdrawal)
(depress age edu_hs edu_somecol edu_colgrad income -> Hostility)
 
(depress age edu_hs edu_somecol edu_colgrad income -> HarshDiscipline)
(depress age edu_hs edu_somecol edu_colgrad income -> Engagement)
 
(Withdrawal age edu_hs edu_somecol edu_colgrad income -> HarshDiscipline)
(Withdrawal age edu_hs edu_somecol edu_colgrad income -> Engagement)
 
(Hostility age edu_hs edu_somecol edu_colgrad income -> HarshDiscipline)
(Hostility age edu_hs edu_somecol edu_colgrad income -> Engagement)
 
(HarshDiscipline age raceblack raceother edu_hs edu_somecol edu_colgrad income childage boy -> Childwb)
 
(Engagement age raceblack raceother edu_hs edu_somecol edu_colgrad income childage boy -> Childwb)
 
(Withdrawal -> encourage listen understand)
(Hostility -> friends prevent money)                          
(HarshDiscipline -> shout spank slap)
(Engagement -> play read tell)
(Childwb -> cogscore internalize externalize)
 
, method(mlmv)
latent(Withdrawal Hostility HarshDiscipline Engagement Childwb)
cov(e.HarshDiscipline*e.Engagement)
 
nocapslatent
from(b);              
               
#delimit cr

(Note: the line "from(b)" was included because this full model couldn't be estimated in one shot. I broke it down into 3 parts in order to obtain good starting values.)

After taking quite a while, the model does converge. I also did not notice any glaringly strange, out-of-whack, or missing SEs, statistics, or CIs in the unstandardized as well as standardized estimates.

However, at the end of the estimation procedure, there is this message shown:

Note: The LR test of model vs. saturated is not reported because the
saturated model is not full rank.

When I try to examine the post-estimation goodness-of-fit statistics, here's what Stata displays:

. estat gof, stats(all)

----------------------------------------------------------------------------
Fit statistic        |      Value   Description
---------------------+------------------------------------------------------
Likelihood ratio     |
          chi2_ms(.) |          .   model vs. saturated
            p > chi2 |          .
          chi2_bs(.) |          .   baseline vs. saturated
            p > chi2 |          .
---------------------+------------------------------------------------------
Population error     |
               RMSEA |          .   Root mean squared error of approximation
 90% CI, lower bound |      0.000
         upper bound |          .
              pclose |          .   Probability RMSEA <= 0.05
---------------------+------------------------------------------------------
Information criteria |
                 AIC | 115381.285   Akaike's information criterion
                 BIC | 115893.733   Bayesian information criterion
---------------------+------------------------------------------------------
Baseline comparison  |
                 CFI |          .   Comparative fit index
                 TLI |          .   Tucker-Lewis index
---------------------+------------------------------------------------------
Size of residuals    |
                  CD |      0.134   Coefficient of determination
----------------------------------------------------------------------------
Note: SRMR is not reported because of missing values.

How do I go about obtaining GOF statistics after using the MLMV method to estimate?

Thanks in advance for taking the time to review my query.

michael

[Richard Williams]

I don't think the problem is with mlmv. I've run other models using mlmv and estat gof and they worked fine. I think the problem is with your model, but it is kind of a huge model so it isn't obvious to me what the problem is.

You might try running it without mlmv and see what happens.

Other than that, I might suggest simplifying the model and gradually building it back up. Maybe you can isolate the problem that way.

Also a lot of improvements to sem have been made to sem since 13.1. In particular, models using mlmv run must faster now. Before I wore myself out too much trying to get this to work, I would try to find a machine with 15.1 and try it out. When students using ancient versions of Stata (13, 14) come to me with a problem, one of the first things I like to do is run it with my machine and Stata 15.1. I don't like to spend a lot of time solving a problem that Stata fixed years ago.

[Michael Chen]

Hi Prof Williams,

I re-ran the model containing just the 1st path, which are all observed variables, and it quickly converged under both method(ml) and method(mlmv):

disp "cash, inkind ---> presure"
 
#delimit ;
 
sem
(cash inkind
m3age mrace_black mrace_other
medu_hs medu_somecol medu_colgrad
fedu_hs fedu_somecol fedu_colgrad -> pressure)

, method(mlmv)
nocapslatent ;
 
#delimit cr

estat gof, stats(all)

Interestingly, Stata notified that because the 2 exogenous variables have missing, it automatically applied "noxconditional" to the estimation (ML as well as MLMV)

note: Missing values found in observed exogenous variables.
Using the noxconditional behavior.
Specify the forcexconditional option to override this behavior.

And despite a very parsimonious model (pressure = cash, inkind, SES covariates), estat gof, stats(all) again is not informative:

----------------------------------------------------------------------------
Fit statistic        |      Value   Description
---------------------+------------------------------------------------------
Likelihood ratio     |
          chi2_ms(.) |          .   model vs. saturated
            p > chi2 |          .
          chi2_bs(.) |          .   baseline vs. saturated
            p > chi2 |          .
---------------------+------------------------------------------------------
Population error     |
               RMSEA |          .   Root mean squared error of approximation
 90% CI, lower bound |      0.000
         upper bound |          .
              pclose |          .   Probability RMSEA <= 0.05
---------------------+------------------------------------------------------
Information criteria |
                 AIC |  45628.181   Akaike's information criterion
                 BIC |  45832.967   Bayesian information criterion
---------------------+------------------------------------------------------
Baseline comparison  |
                 CFI |          .   Comparative fit index
                 TLI |          .   Tucker-Lewis index
---------------------+------------------------------------------------------
Size of residuals    |
                  CD |      0.035   Coefficient of determination
----------------------------------------------------------------------------
Note: SRMR is not reported because of missing values.

I will try to find a colleague or faculty with Stata 15.1 and see if perhaps I'm just being limited by software (I'm using Stata 13.1).

If you have any other suggestions, please let me know.

Thank you very much,

Michael