sem model--poor overall goodness of fit

[Taka Sakamoto]

Dear Statalisters:

I have a sem model where the directions of coefficients and statistical significance look good. But when I check the overall goodness of fit, it is a poorly fit model. I browsed the general reasons why you may get a poorly fit model, but I don't know what to do with the situation, as I don't want to abandon the whole model. I would be grateful if you could teach me how I can get over the situation.

My sem model looks like this:



The results of overall goodness of fit are like this:

. estat gof, stats(all)

    
Fit statistic              Value    Description
    
Likelihood ratio    
chi2_ms(2)     196.872    model vs. saturated
p > chi2       0.000
chi2_bs(10)    1457.854    baseline vs. saturated
p > chi2       0.000
    
Population error    
RMSEA       0.254    Root mean squared error of approximation
90% CI, lower bound       0.225
upper bound       0.285
pclose       0.000    Probability RMSEA <= 0.05
    
Information criteria
AIC   31922.766    Akaike's information criterion
BIC   32007.905    Bayesian information criterion
    
Baseline comparison  
CFI       0.865    Comparative fit index
TLI       0.327    Tucker–Lewis index
    
Size of residuals    
CD       0.754    Coefficient of determination
    
Note: SRMR is not reported because    of missing values.

Thank you in advance for your help.

Best wishes,

Taka Sakamoto

[Richard Williams]

You have 2 more paths you can add to the model.

Let socialmarketecon directly affect GDPGrowth. Unless your theory emphatically argues that socialmarketecon should have no direct effect once other variables are controlled, this would seem perfectly reasonable to me.

Let consumption directly affect investment, or investment affect consumption, or let their residuals be correlated. Correlated residuals would seem like a safe bet to me, especially if causal direction is unclear or you don't think there is a causal connection between the two.

In short, the model seems easily fixable, unless you have strong theoretical reasons for excluding the possible paths that you did.

Incidentally, I would tweak the graph a bit. The line from eta1 to poverty and the line from poverty to GDPGrowth overlap and appear to be a nonsensical double-headed arrow. If you move eta1 up or down in the graph the lines will be distinct.

[Richard Williams]

If you want to use raw empiricism to decide how to modify a model, after sem you can run the postestimation command

estat mindices

"estat mindices reports modification indices for path coefficients and covariances that were constrained or omitted in the fitted model. Modification indices are score tests (Lagrange multiplier tests) for the statistical significance of the constrained parameters."

So, for example, it might show you that adding a path from socialmarketecon to GDPGrowth would substantially improve model fit.

You should use your judgment here. The model changes that would most improve model fit are not necessarily changes that make theoretical sense. Your model is pretty simple though, and I doubt that either of the two changes I suggested would do terrible violence to your theory.

For a good book on sem, see

https://www.stata.com/bookstore/disc...g-using-stata/

[Richard Williams]

Also, I give various Sem examples in

https://www3.nd.edu/~rwilliam/stats2/l95.pdf

For an added bonus, it includes a discussion of the classic 1982 paper, “Beyond Wives Family Sociology: A Method for Analyzing Couple Data,” by Thomson & Williams (classic to me, anyway, since it was my first published piece.)

[Taka Sakamoto]

Dear Richard:

Thank you very much for your much needed advice. I'm going to implement your suggestions and come back with results.

Thank you!

Taka

[Taka Sakamoto]

Dear Richard:

I correlated the errors of consumption and investment and added a direct effect path from socialmarketecon (sme) to GDP growth. The model now looks like this:



The model fit tests results are like this:

. estat gof, stats(all)

    
Fit statistic              Value    Description
    
Likelihood ratio    
chi2_ms(0)       0.000    model vs. saturated
p > chi2           .
chi2_bs(10)    1457.854    baseline vs. saturated
p > chi2       0.000
    
Population error    
RMSEA       0.000    Root mean squared error of approximation
90% CI, lower bound       0.000
upper bound       0.000
pclose       1.000    Probability RMSEA <= 0.05
    
Information criteria
AIC   31729.895    Akaike's information criterion
BIC   31825.676    Bayesian information criterion
    
Baseline comparison  
CFI       1.000    Comparative fit index
TLI       1.000    Tucker–Lewis index
    
Size of residuals    
CD       0.693    Coefficient of determination
    
Note: SRMR is not reported because    of missing values.

Do the test results look okay? It looks like they have improved much after implementing your suggestions. Please let me know.

Taka

[Taka Sakamoto]

Dear Richard:

I correlated the errors of consumption and investment and added a direct effect path from socialmarketecon (sme) to GDP growth. The model now looks like this:



The model fit tests results are like this:

. estat gof, stats(all)

    
Fit statistic              Value    Description
    
Likelihood ratio    
chi2_ms(0)       0.000    model vs. saturated
p > chi2           .
chi2_bs(10)    1457.854    baseline vs. saturated
p > chi2       0.000
    
Population error    
RMSEA       0.000    Root mean squared error of approximation
90% CI, lower bound       0.000
upper bound       0.000
pclose       1.000    Probability RMSEA <= 0.05
    
Information criteria
AIC   31729.895    Akaike's information criterion
BIC   31825.676    Bayesian information criterion
    
Baseline comparison  
CFI       1.000    Comparative fit index
TLI       1.000    Tucker–Lewis index
    
Size of residuals    
CD       0.693    Coefficient of determination
    
Note: SRMR is not reported because    of missing values.

Do the test results look okay? It looks like they have improved much after implementing your suggestions. Please let me know.

Taka

[Richard Williams]

The model is just identified, so it has to fit perfectly. That is, once you added those two paths, the fit was a fore-ordained conclusion.

It is possible that some paths aren’t necessary and could be eliminated, which would make the model more parsimonious. Unless you’ve got some strong theory about paths that are unnecessary though, a just identified model may be fine for you.

I’m not sure what your goal is in estimating this model. Perhaps you you want to explain why poverty and GDPGrowth are related. If so, your model claims it is because poverty directly affects gdp, poverty also indirectly affects gdp, and poverty and gdp share common causes. That sort of breakdown can be interesting and informative, but your own reasons and goals may differ. If you aren’t that familiar with path analysis and structural equation modeling you may want to read up on it a bit.

[Taka Sakamoto]

Thank you, Richard. I'll read the materials you suggested.

What if we got these test results instead?:

. estat gof, stats(all)

    
Fit statistic              Value    Description
    
Likelihood ratio     
chi2_ms(1)       4.035    model vs. saturated
p > chi2       0.045
chi2_bs(10)    1457.854    baseline vs. saturated
p > chi2       0.000
    
Population error     
RMSEA       0.045    Root mean squared error of approximation
90% CI, lower bound       0.006
upper bound       0.094
pclose       0.474    Probability RMSEA <= 0.05
    
Information criteria 
AIC   31731.930    Akaike's information criterion
BIC   31822.390    Bayesian information criterion
    
Baseline comparison  
CFI       0.998    Comparative fit index
TLI       0.979    Tucker–Lewis index
    
Size of residuals    
CD       0.684    Coefficient of determination
    
Note: SRMR is not reported because    of missing values.

Do these look reasonably good fit? I see that chi-squared does not suggest a good fit.

I'm a political scientist, and realized that social market economies (SMEs) of Europe achieve lower inequality and poverty. One of the characteristics of these SMEs (particularly Nordic countries) is that they make large investments in public education and R&D, and I thought SMEs might also be positively associated with GDP (or producitivity) growth. In equations I have done, poverty was negatively associated with GDP growth, which makes sense from a new Keynesian perspective. The sem results I got seem to be supportive of the idea.

Taka

[Taka Sakamoto]

Poverty is negatively associated with GDP growth when there's no direct effect from SMEs to GDP growth (which is consistent with a Keynesian view). But when I create a direct path from SMEs to GDP growth, poverty becomes positive and insignificant, which means that poverty is good for GDP growth which is not consistent with a Keynesian view.

[Richard Williams]

I would interpret that as saying poverty has no direct effect once SME is controlled for (the estimated effect may be positive but does not even come close to significantly differing from zero.). Poverty still has negative indirect effects on gdp -- poverty negatively affects consumption and investment, which in turn positively affect gdp. So, according to your model, poverty is not good for gdp.

Indirect effects are still effects -- you shouldn't just look at the direct effect of a variable when assessing its impact. Even if the direct effect of poverty was positive and statistically significant, its total effect (direct + indirect) could still be negative.

So, I see no conflict with Keynes. A zero direct effect does not mean poverty is irrelevant for gdp growth. Rather, its effects are indirect. Also, part of the correlation between poverty and gdp is due to the fact that they share a common cause, sme.

Basically, all I know about this topic is what you have just told me. But to me, it seems like your inclusion of SMEs could be very important, and if no one has done that before, you may be on to something important.

Here is my handout on path analysis, which take about things like direct effects, indirect effects, and common causes. It may not be the best source, but one way or another, based on what you are saying, a good understanding of direct effects, indirect effects, and common causes might help you make some important points.

https://www3.nd.edu/~rwilliam/stats2/l62.pdf

[Taka Sakamoto]

Yes. Thank you for all your generous help. I'll read the materials and book you suggested.

Thank you so much!

Best wishes,

Taka

[Taka Sakamoto]

Dear Richard:

May I ask you a question? This is the model you helped me with, with consumption growth (hfcegrow) and gross fixed capital formation growth removed (gfcfgrow). In this simple model, is there a way to check if there's reverse causation from GDP growth to poverty (koenpov50disposable)? I've been checking your handout and other materials, but could not tell by myself.

Thank you for your help in advance.

Taka

[Taka Sakamoto]

Dear Professor Williams:

I hope you'll see this post. You helped me with my sem model before. I have another related question. Attached is the model I have. I want to reduce autocorrelation concerns and want to include a lagged dependent variable. Including a lagged dependent variable as easy as creating its box and connect it to gdpgrow like I have done in the model? Or do I need to do other things as well?

I thank you for your help in advance.

Best wishes,

Taka

Richard Williams


"glag" is the lagged dependent variable, and gdpgrow is the dependent variable.

SEM_1.stsem
SEM_1.stsem

[Taka Sakamoto]

I didn't do it right in including the image file. I'll keep trying. Thank you for your patience.