Measurement invariance with multigroup CFA: two different methods

[Renske Gaag]

Hi Statalist members,

I have a question about testing for measurement invariance in a two latent variable CFA with multiple groups using -sem- in stata 14.2. I found two methods to check for measurement invariance. The first method fixes loadings of a latent variable for item1 at 1 and intercept of item1 at 0 across all groups (https://stats.idre.ucla.edu/stata/fa...mand-stata-12/).

*Method 1
*Configural
sem (L1-> v1 v2 v3 v4) (L2 -> v5 v6 v7 v8)       ///
            (v1 <- L1 _cons@0)                               ///
            (v5  <- L2 _cons@0),                             ///
            group(grp1)  mean(L1 L2)                     ///
            ginvariant(none)                      
 
*Metric
sem (L1-> v1 v2 v3 v4) (L2 -> v5 v6 v7 v8)       ///
           (v1 <- L1 _cons@0)                                ///
           (v5   <- L2 _cons@0),                             ///
            group(grp1)  mean(L1 L2)                     ///
            ginvariant(mcoef)

The second method sets the mean of the latent variable to 0 and the variances to 1 in all groups (see https://blog.stata.com/2016/08/23/gr...nt-invariance/).

*Method 2
*Configural
sem (L1-> v1 v2 v3 v4) (L2 -> v5 v6 v7 v8),       ///
           group(grp1)  variance(L1@1 L2@1)      ///
           mean(L1@0 L2@0)                                ///
           ginvariant(none)
 
*Metric
sem (L1-> v1 v2 v3 v4) (L2 -> v5 v6 v7 v8),       ///
          group(grp1)  variance(L1@1 L2@1)       ///
          mean(L1@0 L2@0)                                ///
          ginvariant(mcoef)

I tried both methods. For the configural model both methods produced the same results for model fit. However, for the metric model there is a difference in model fit. Method 2 produces higher chi-square and has more degrees of freedom (a difference of 2 df when comparing two groups) as such resulting in somewhat different conclusions about metric invariance. I was wondering how to explain this difference and also what method is preferred if this difference in results is not related to my syntax.

I hope you can help me.
Thanks a lot!
Renske

[Weiwen Ng]

I don't see anything obvious in your syntax that should lead to the difference in df you described. Can you post the results, also in code delimiters?

[Renske Gaag]

Dear Weiwen,
Thanks for your quick response. Below are the results for the test of metric invariance (with indeed two additional constraints imposed for the second method). Hope you can help! (And I used code delimiters, but code window at least in preview is still rather big, so if there is an easy way to change this to a smaller window with scrolling bar also let me know)
Best regards,
Renske

 
------------------------------------------------------------------------------------------------

Method 1

------------------------------------------------------------------------------------------------



Structural equation model                       Number of obs     =     14.246

Grouping variable  = grp                     Number of groups  =          2

Estimation method  = mlmv

Log likelihood     = -143374,14



( 1)  [v1]0bn.grp#c.L1 = 1

( 2)  [v2]0bn.grp#c.L1 - [v2]1.grp#c.L1 = 0

( 3)  [v3]0bn.grp#c.L1 - [v3]1.grp#c.L1 = 0

( 4)  [v4]0bn.grp#c.L1 - [v4]1.grp#c.L1 = 0

( 5)  [v5]0bn.grp#c.L2 = 1

( 6)  [v6]0bn.grp#c.L2 - [v6]1.grp#c.L2 = 0

( 7)  [v7]0bn.grp#c.L2 - [v7]1.grp#c.L2 = 0

( 8)  [v8]0bn.grp#c.L2 - [v8]1.grp#c.L2 = 0

( 9)  [v1]0bn.grp = 0

(10)  [v5]0bn.grp = 0

(11)  [v1]1.grp#c.L1 = 1

(12)  [v5]1.grp#c.L2 = 1

(13)  [v1]1.grp = 0

(14)  [v5]1.grp = 0

--------------------------------------------------------------------------------

              |                 OIM

              |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]

---------------+----------------------------------------------------------------

Measurement    |

 v1 <-  |

          L1 |

[*] |          1  (constrained)

        _cons |

[*] |          0  (constrained)

 -------------+----------------------------------------------------------------

 v2 <-  |

          L1 |

[*] |   1,018263   ,0119713    85,06   0,000     ,9947997    1,041726

        _cons |

      grp0  |   ,0037138   ,0082256     0,45   0,652     -,012408    ,0198356

     grp1  |   ,0222782   ,0201428     1,11   0,269     -,017201    ,0617573

 -------------+----------------------------------------------------------------

 v3 <-  |

          L1 |

[*] |   1,040196   ,0113604    91,56   0,000      1,01793    1,062462

        _cons |

      grp0  |   ,0815293   ,0078452    10,39   0,000     ,0661529    ,0969057

     grp1  |   ,0757561   ,0190875     3,97   0,000     ,0383452     ,113167

 -------------+----------------------------------------------------------------

 v4 <-   |

          L1 |

[*] |   ,8227086    ,011538    71,30   0,000     ,8000945    ,8453227

        _cons |

      grp0  |   ,0210906    ,008193     2,57   0,010     ,0050326    ,0371486

     grp1  |   ,0147129   ,0203111     0,72   0,469    -,0250961    ,0545219

 -------------+----------------------------------------------------------------

 v5 <-    |

         L2 |

[*] |          1  (constrained)

        _cons |

[*] |          0  (constrained)

 -------------+----------------------------------------------------------------

 v6 <-    |

         L2 |

[*] |   1,144594   ,0319756    35,80   0,000     1,081923    1,207265

        _cons |

      grp0  |   ,3172666   ,0134544    23,58   0,000     ,2908965    ,3436368

     grp1  |    ,170937   ,0295832     5,78   0,000     ,1129551    ,2289189

 -------------+----------------------------------------------------------------

 v7 <-    |

         L2 |

[*] |   1,259114   ,0349859    35,99   0,000     1,190543    1,327685

        _cons |

      grp0  |   ,3115106   ,0143274    21,74   0,000     ,2834294    ,3395918

     grp1  |   ,1113909   ,0310276     3,59   0,000     ,0505779    ,1722039

 -------------+----------------------------------------------------------------

 v8 <-    |

         L2 |

[*] |   ,9170568   ,0279774    32,78   0,000     ,8622221    ,9718915

        _cons |

      grp0  |    ,070133   ,0121078     5,79   0,000     ,0464021    ,0938639

     grp1  |  -,0262003   ,0263016    -1,00   0,319    -,0777505    ,0253499

---------------+----------------------------------------------------------------

     mean(L1)|

      grp0  |  -,0155526   ,0087551    -1,78   0,076    -,0327123     ,001607

     grp1  |  -,0960347   ,0222418    -4,32   0,000    -,1396278   -,0524416

    mean(L2)|

      grp0  |  -,1985625   ,0091792   -21,63   0,000    -,2165534   -,1805716

     grp1  |  -,1057349    ,022392    -4,72   0,000    -,1496224   -,0618473

---------------+----------------------------------------------------------------

var(e.v1)|

      grp0  |   ,3875468    ,006696                      ,3746426    ,4008954

     grp1  |   ,3904593   ,0158579                      ,3605832    ,4228107

var(e.v2)|

      grp0  |   ,4196909    ,007187                      ,4058384    ,4340163

     grp1  |   ,4121586   ,0167265                      ,3806451    ,4462811

var(e.v3)|

      grp0  |   ,3291189   ,0062827                      ,3170326    ,3416659

     grp1  |   ,3122469     ,01418                      ,2856555    ,3413136

var(e.v4)|

      grp0  |   ,5541117    ,008079                      ,5385012    ,5701747

     grp1  |    ,567734   ,0199404                      ,5299663    ,6081931

 var(e.v5)|

      grp0  |   ,8158944   ,0119464                      ,7928126    ,8396482

     grp1  |   ,7929687   ,0276826                      ,7405263    ,8491249

 var(e.v6)|

      grp0  |   ,6393996   ,0108647                      ,6184559    ,6610525

     grp1  |   ,7016899   ,0265592                      ,6515187    ,7557246

 var(e.v7)|

      grp0  |   ,6144612   ,0110269                      ,5932245    ,6364581

     grp1  |   ,6555145   ,0261327                      ,6062452    ,7087879

 var(e.v8)|

      grp0  |   ,7222634   ,0105641                       ,701852    ,7432684

     grp1  |   ,7110449   ,0247073                      ,6642318    ,7611573

      var(L1)|

      grp0  |   ,5467377   ,0113418                       ,524954    ,5694254

     grp1  |   ,6115797   ,0240975                      ,5661271    ,6606814

     var(L2)|

      grp0  |   ,2110634   ,0093363                      ,1935354    ,2301788

     grp1  |   ,2207204   ,0147552                      ,1936153    ,2516201

---------------+----------------------------------------------------------------

 cov(L1,L2)|

      grp0  |   ,2274938   ,0070214    32,40   0,000      ,213732    ,2412555

     grp1  |   ,2568075   ,0139534    18,40   0,000     ,2294592    ,2841557

--------------------------------------------------------------------------------

Note:[*] identifies parameter estimates constrained to be equal across groups.

LR test of model vs. saturated: chi2(44)  =   1056,64, Prob > chi2 = 0,0000

 
-----------------------------------------------------------------------------------------------------

Method 2

-----------------------------------------------------------------------------------------------------



Structural equation model                       Number of obs     =     14.246

Grouping variable  = grp                    Number of groups  =          2

Estimation method  = mlmv

Log likelihood     = -143378,13



( 1)  [v1]0bn.grp#c.L1 - [v1]1.grp#c.L1 = 0

( 2)  [v2]0bn.grp#c.L1 - [v2]1.grp#c.L1 = 0

( 3)  [v3]0bn.grp#c.L1 - [v3]1.grp#c.L1 = 0

( 4)  [v4]0bn.grp#c.L1 - [v4]1.grp#c.L1 = 0

( 5)  [v5]0bn.grp#c.L2 - [v5]1.grp#c.L2 = 0

( 6)  [v6]0bn.grp#c.L2 - [v6]1.grp#c.L2 = 0

( 7)  [v7]0bn.grp#c.L2 - [v7]1.grp#c.L2 = 0

( 8)  [v8]0bn.grp#c.L2 - [v8]1.grp#c.L2 = 0

( 9)  [var(L1)]0bn.grp = 1

(10)  [var(L2)]0bn.grp = 1

(11)  [mean(L1)]0bn.grp = 0

(12)  [mean(L2)]0bn.grp = 0

(13)  [var(L1)]1.grp = 1

(14)  [var(L2)]1.grp = 1

(15)  [mean(L1)]1.grp = 0

(16)  [mean(L2)]1.grp = 0

--------------------------------------------------------------------------------

              |                 OIM

              |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]

---------------+----------------------------------------------------------------

Measurement    |

 v1 <-  |

          L1 |

[*] |   ,7456757   ,0074121   100,60   0,000     ,7311483    ,7602032

        _cons |

      grp0  |  -,0155544   ,0087969    -1,77   0,077    -,0327959    ,0016871

     grp1  |  -,0960055   ,0216347    -4,44   0,000    -,1384088   -,0536022

 -------------+----------------------------------------------------------------

 v2 <-  |

          L1 |

[*] |   ,7592743   ,0076589    99,14   0,000     ,7442632    ,7742855

        _cons |

      grp0  |  -,0121255   ,0090452    -1,34   0,180    -,0298538    ,0056029

     grp1  |  -,0754556   ,0221278    -3,41   0,001    -,1188252   -,0320859

 -------------+----------------------------------------------------------------

 v3 <-  |

          L1 |

[*] |   ,7757037   ,0072244   107,37   0,000      ,761544    ,7898633

        _cons |

      grp0  |   ,0653519   ,0087354     7,48   0,000     ,0482309     ,082473

     grp1  |  -,0241015   ,0212727    -1,13   0,257    -,0657953    ,0175923

 -------------+----------------------------------------------------------------

 v4 <-   |

          L1 |

[*] |   ,6133277   ,0078226    78,40   0,000     ,5979956    ,6286598

        _cons |

      grp0  |   ,0082939   ,0087345     0,95   0,342    -,0088254    ,0254132

     grp1  |  -,0643054   ,0216018    -2,98   0,003    -,1066441   -,0219668

 -------------+----------------------------------------------------------------

 v5 <-    |

         L2 |

[*] |   ,4608467   ,0100166    46,01   0,000     ,4412145     ,480479

        _cons |

      grp0  |  -,1985622   ,0091849   -21,62   0,000    -,2165644   -,1805601

     grp1  |  -,1057321   ,0223028    -4,74   0,000    -,1494448   -,0620194

 -------------+----------------------------------------------------------------

 v6 <-    |

         L2 |

[*] |   ,5275122   ,0098204    53,72   0,000     ,5082646    ,5467599

        _cons |

      grp0  |   ,0899936   ,0086992    10,35   0,000     ,0729435    ,1070438

     grp1  |    ,049921   ,0220721     2,26   0,024     ,0066604    ,0931815

 -------------+----------------------------------------------------------------

 v7 <-    |

         L2 |

[*] |   ,5802581   ,0098104    59,15   0,000       ,56103    ,5994862

        _cons |

      grp0  |   ,0614982   ,0088502     6,95   0,000     ,0441521    ,0788442

     grp1  |  -,0217359   ,0222082    -0,98   0,328    -,0652631    ,0217913

 -------------+----------------------------------------------------------------

 v8 <-    |

         L2 |

[*] |   ,4226212   ,0095175    44,40   0,000     ,4039672    ,4412753

        _cons |

      grp0  |  -,1119598   ,0086056   -13,01   0,000    -,1288264   -,0950932

     grp1  |  -,1231627   ,0210022    -5,86   0,000    -,1643262   -,0819991

---------------+----------------------------------------------------------------

     mean(L1)|

[*] |          0  (constrained)

    mean(L2)|

[*] |          0  (constrained)

---------------+----------------------------------------------------------------

var(e.v1)|

      grp0  |    ,387203   ,0066895                      ,3743113    ,4005387

     grp1  |   ,3920049   ,0158492                        ,36214    ,4243328

var(e.v2)|

      grp0  |    ,419419   ,0071823                      ,4055755    ,4337351

     grp1  |   ,4138567   ,0167153                      ,3823586    ,4479497

var(e.v3)|

      grp0  |   ,3286291   ,0062746                      ,3165584    ,3411601

     grp1  |   ,3148491   ,0141623                      ,2882798     ,343867

var(e.v4)|

      grp0  |   ,5540338   ,0080765                       ,538428    ,5700918

     grp1  |   ,5688743   ,0199523                      ,5310824    ,6093554

 var(e.v5)|

      grp0  |   ,8158667   ,0119443                       ,792789    ,8396161

     grp1  |   ,7932267   ,0275991                      ,7409367    ,8492069

 var(e.v6)|

      grp0  |   ,6392944   ,0108335                      ,6184099    ,6608842

     grp1  |   ,7021565   ,0262676                      ,6525152    ,7555745

 var(e.v7)|

      grp0  |    ,614374   ,0110028                       ,593183     ,636322

     grp1  |   ,6560921   ,0256568                      ,6076845    ,7083558

 var(e.v8)|

      grp0  |   ,7222555   ,0105635                      ,7018455    ,7432591

     grp1  |   ,7111734   ,0246796                      ,6644105    ,7612275

      var(L1)|

[*] |          1  (constrained)

     var(L2)|

[*] |          1  (constrained)

---------------+----------------------------------------------------------------

 cov(L1,L2)|

      grp0  |   ,6715884   ,0099561    67,46   0,000     ,6520748    ,6911019

     grp1  |   ,6881578   ,0222652    30,91   0,000     ,6445188    ,7317969

--------------------------------------------------------------------------------

Note:[*] identifies parameter estimates constrained to be equal across groups.

LR test of model vs. saturated: chi2(46)  =   1064,60, Prob > chi2 = 0,0000

[Weiwen Ng]

Normally, our results format quite nicely when we paste them in code delimiters, but I've never pasted SEM results in before. Maybe this would help:

sem (L1-> v1 v2 v3 v4) (L2 -> v5 v6 v7) (v1 <- L1 _cons@0) (v5 <- L2 _cons@0), ///
group(grp1) mean(L1 L2) /ginvariant(none)
estimates store configural_m1
sem (L1-> v1 v2 v3 v4) (L2 -> v5 v6 v7) (v1 <- L1 _cons@0) (v5 <- L2 _cons@0), ///
group(grp1) mean(L1 L2) /ginvariant(mcoef)
estimates store configural_m2
estimates table configural_m1 configural_m2, b(%9.3fc)

Then repeat for the other two sets of models. That should output only the betas to 3 decimal places.

That said, I can see one issue from your output. Your second set of models constrains the mean and variance of both groups to 0 and 1 respectively, so they have equal mean values of both latent traits. The first set of models freely estimates the means and variances of the latent traits.

Initially, I thought that constraining the means and variances for both groups to be equal in your second set of models didn't make sense. If you wanted to change that, this should work:

sem (L1-> v1 v2 v3 v4) (L2 -> v5 v6 v7) , ///
group(grp1) mean(0: L1@0) mean(0: L2@0) variance(0: L1@1) variance(0:L2@1) ginvariant(none)

Note: not tested, but this should invoke constraints only for grp1 == 0. The means and variances of each latent trait should be freely estimated for grp1 == 1.

However, I browsed the links you posted, and you do appear to have followed the initial steps in each link (each one discusses one method). I would just follow the procedures all the way through. I'm not familiar with traditional SEM, so I currently can't make a recommendation on what to do. I would be interested in hearing if other Statalisters have recommendations, because I intend to do a measurement invariance test like this within an item response theory framework.

[Renske Gaag]

Hi Weiwen,
Thanks a lot and sorry for my late reply! I also tried the same analysis in MPlus with the automated [model=configural metric] command and this generated a result that was similar to the first method. I also tried your method, but had problems with convergence, so after all I decided to go with the first method, but I would also be interested to hear what other statalisters have to comment on this.
Best,
Renkse

[Franziska Bellmann]

Hi Renske, I am actually not really sure whether the second approach is correct, despite being described on the Stata blog (https://blog.stata.com/2016/08/23/gr...nt-invariance/). Literature recommends to only fix the latent means to 0 and the latent variances to 1 in ONE group and estimate them freely in the second group, not fixing them in BOTH groups (http://www.agencylab.ku.edu/~agencyl...d,%202006).pdf). Do you have any other literature or source that says it's ok to fix them in both groups? Because when I fix them in only one group, my model does not converge. When I fix them in both groups, it does. It would be great to get some literature saying it's ok to fix them in both groups... Thanks a lot in advance for your reply! Kind regards, Franzi

[Renske Gaag]

Hi Franzi, I am also not sure whether the second approach from the stata blog is correct and had the same problems with convergence as you did when fixing the latent mean to 0 and variance to 1 in one group as Weiwen proposed. That is why I went with the first method I mentioned in my first post and I fixed loadings of the latent variable for item1 at 1 and intercept of item1 at 0 across all groups. Outcomes for this method were the same as outcomes in Mplus when using the automated [model=configural metric] command. In the end I think both approaches - fixing latent factor mean (in one group) or fixing factor loadings of item- should get you the same result in terms of testing measurement invariance (and then I don't mean the approach described in the stata blog, because that clearly produced different results), but just like you I didn't get this model with fixed factor means to run in stata. This reference also mentions both approaches: https://pdfs.semanticscholar.org/2ab...a3eec4357f.pdf (pp. 4 two ways of parameterization of the CFA model). Best wishes, Renske