Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Comparing differences between two xtmixed models

    Dear Statalist Users, I have run a series of xtmixed models but I want to compare the differences between men and women by stratifying the final model by sex. In the final model some coefficients are significant for men and some aren't for women (and vice-versa) but I want to know whether these differences in the two models are statistically different? How do I best go about this? I was thinking of a likelihood ratio test but the sample sizes for men and women are different. Your thoughts would be greatly appreciated

    Best
    Brendan

  • #2
    You also can't do this as a likelihood ratio test because the male and female models are not nested in either direction.

    I wouldn't do this as a stratified analysis at all. I would do a single analysis in which the variables are interacted with an indicator variable for sex. The interaction terms are then your estimators of the difference between the coefficients for males and the coefficients for female.

    If you want more specific advice, post back showing your actual commands and the output you got from Stata.

    By the way, if you are using version 13 or later, -xtmixed- is no longer documented; the newname is -mixed-. -xtmixed- is still recognized, but at some point in the future it may cease to be. So better to get in the habit of using the current terminology.

    Comment


    • #3
      Thank you for your response, Clyde.

      I've also thought about this option - using an indicator variable on my coefficients of interest, but i'm interested in the stratified approach because some coefficients for separate analyses for men are women are going in different directions and this is most interesting to me from an analysis point of view. Can you do Wald tests here?

      Best
      Brendan

      Comment


      • #4
        I've also thought about this option - using an indicator variable on my coefficients of interest, but i'm interested in the stratified approach because some coefficients for separate analyses for men are women are going in different directions and this is most interesting to me from an analysis point of view.
        It's not just a matter of adding an indicator variable. It's adding an indicator variable plus the interactions of that indicator variable with each of the other model variables. That will capture the phenomena you are interested in. The coefficients of the interaction terms will be your estimators of the male-female differences in the coefficients of the other constituent variables, and significance tests of them can be read straight off the regression output. If you want to jointly test several of those differences, yes, you can do Wald tests as well.

        Comment


        • #5
          Thanks again Clyde, I understand the points you are making in reference the indicator variables and I can do that. I just want to show you an example from my output to perhaps further illustrate what I am after....

          In Model 1 and Model 2 you will see that 'FT' is significant for both men and women but the coefficients are going in opposite directions. This indicates that men and women who are in full-time employment have different attitudes (that's the dependent measure). Men in full-time are more traditional and women are more non-traditional. It's interesting empirically but also has implications conceptually/theoretically too. This is the benefit of the stratified model, but I can't see this kind of information if I do it all in the one model. I won't see if the coefficients for men and women differ. Or is there a way I convey this in the model you propose.

          Model 1 - Men
          atwkwfr Coef. Std. Err. z P>z [95% Conf. Interval]
          cohort .0182807 .0096943 1.89 0.059 -.0007197 .0372811
          degree -.0018534 .0310107 -0.06 0.952 -.0626333 .0589264
          FT -.1111178 .0311435 -3.57 0.000 -.172158 -.0500776
          PT -.0494488 .0370376 -1.34 0.182 -.1220411 .0231435
          UNEMP .0275971 .0530636 0.52 0.603 -.0764056 .1315998
          married -.065268 .0408064 -1.60 0.110 -.1452471 .0147111
          cohabiting .020129 .038431 0.52 0.600 -.0551944 .0954523
          sepdivwid -.1137313 .053351 -2.13 0.033 -.2182973 -.0091653
          parenthood -.0737202 .0340951 -2.16 0.031 -.1405455 -.006895
          timec .1187525 .0230332 5.16 0.000 .0736083 .1638967
          cohortxtimec -.0223161 .0051335 -4.35 0.000 -.0323776 -.0122546
          degreextimec .0547336 .0160592 3.41 0.001 .0232581 .0862091
          FTxtimec .0104219 .0177453 0.59 0.557 -.0243582 .0452019
          PTxtimec .0292143 .0234538 1.25 0.213 -.0167543 .0751829
          UNEMPxtimec .0399228 .0338981 1.18 0.239 -.0265162 .1063618
          Model 1 - Men
          atwkwfr Coef. Std. Err. z P>z [95% Conf. Interval]
          cohort .0317941 .0087224 3.65 0.000 .0146986 .0488897
          degree .1537767 .0281751 5.46 0.000 .0985545 .2089988
          FT .2279848 .0274021 8.32 0.000 .1742776 .2816919
          PT .1160074 .0251275 4.62 0.000 .0667583 .1652564
          UNEMP .0651346 .0506628 1.29 0.199 -.0341626 .1644318
          married -.2480093 .0371757 -6.67 0.000 -.3208724 -.1751463
          cohabiting -.166229 .0356726 -4.66 0.000 -.236146 -.0963119
          sepdivwid -.1588666 .0451127 -3.52 0.000 -.2472859 -.0704472
          parenthood .1376886 .0316958 4.34 0.000 .075566 .1998111
          timec .1411526 .0198057 7.13 0.000 .1023342 .1799711
          cohortxtimec -.0266101 .0044779 -5.94 0.000 -.0353866 -.0178336
          degreextimec .0126407 .0146785 0.86 0.389 -.0161286 .0414099
          FTxtimec .0188864 .0161836 1.17 0.243 -.0128328 .0506057
          PTxtimec -.0029974 .0154599 -0.19 0.846 -.0332983 .0273034
          UNEMPxtimec .0395277 .0333831 1.18 0.236 -.025902 .1049575


          Comment


          • #6
            This is the benefit of the stratified model, but I can't see this kind of information if I do it all in the one model. I won't see if the coefficients for men and women differ.
            Yes you will. I don't understand why you think you won't.

            You don't show the commands you used to estimate these two models, so I can't show you how you would put it all together. It might also involve some re-coding of your data, because some of your variables look like 0/1 indicators for levels of the same categorical variable (e.g. FT, PT, and UNEMP look to my eye like they are three levels of a single categorical variable--this would be better handled by a single variable, call it employment_status, coded 1/2/3 and then entered into the model using factor variable notation as i.employment_status. Similarly, the marital status variables. And it looks like you have several interaction terms involving timec that you have implemented by calculating product variables: those too would be better handled with factor variable notation. Just how that looks depends on whether timec is a discrete or continuous variable.)

            If you use the -dataex- command to show a brief excerpt of your data, and show the commands you used to get the results you show above, I will be happy to show how you would implement the approach I'm advocating.

            Comment


            • #7
              Thanks Clyde:

              Dataex outout:

              Code:
              * Example generated by -dataex-. To install: ssc install dataex
              clear
              input byte atwkwfr float(cohort degree) byte(FT PT UNEMP married cohabiting sepdivwid) float(parenthood timec cohortxtimec degreextimec FTxtimec PTxtimec UNEMPxtimec)
              5 0 0 0 1 0 1 0 0 0 1 0 0 0 1 0
              3 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
              4 0 0 0 0 1 0 0 0 0 2 0 0 0 0 2
              4 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              5 0 0 0 0 1 0 1 0 0 2 0 0 0 0 2
              4 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
              7 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              6 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              3 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              5 0 0 1 0 0 0 1 0 0 2 0 0 2 0 0
              4 0 0 1 0 0 0 1 0 0 2 0 0 2 0 0
              7 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
              3 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              4 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              6 0 0 0 1 0 0 1 0 0 2 0 0 0 2 0
              3 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
              4 0 1 0 1 0 1 0 0 0 2 0 2 0 2 0
              5 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              6 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0
              7 0 0 1 0 0 0 1 0 0 2 0 0 2 0 0
              7 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
              6 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              7 0 1 1 0 0 0 1 0 0 2 0 2 2 0 0
              4 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              3 0 0 1 0 0 0 0 0 0 1 0 0 1 0 0
              7 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
              7 0 0 1 0 0 0 0 0 0 2 0 0 2 0 0
              7 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              7 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              7 0 1 0 1 0 0 1 0 0 2 0 2 0 2 0
              7 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              7 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0
              5 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0
              2 0 0 0 1 0 0 0 0 1 2 0 0 0 2 0
              7 0 0 0 0 1 0 0 0 0 2 0 0 0 0 2
              4 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
              4 0 0 1 0 0 0 0 0 0 2 0 0 2 0 0
              5 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
              5 0 0 0 0 1 0 0 0 0 2 0 0 0 0 2
              4 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
              3 0 1 0 1 0 0 0 0 0 2 0 2 0 2 0
              4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
              7 0 1 1 0 0 0 0 0 0 2 0 2 2 0 0
              5 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0
              5 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              5 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0
              1 0 0 1 0 0 0 0 0 0 2 0 0 2 0 0
              4 0 0 1 0 0 0 0 0 0 1 0 0 1 0 0
              7 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              4 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0
              5 0 0 0 0 1 0 0 0 0 2 0 0 0 0 2
              6 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
              5 0 0 1 0 0 0 1 0 0 2 0 0 2 0 0
              7 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              4 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
              6 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              5 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              5 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              4 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              4 0 0 0 0 1 0 0 0 0 2 0 0 0 0 2
              6 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              5 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              5 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
              6 0 0 0 1 0 0 1 0 0 2 0 0 0 2 0
              4 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
              7 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              4 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0
              3 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
              7 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              6 0 0 1 0 0 0 1 0 0 2 0 0 2 0 0
              6 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              7 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0
              4 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0
              5 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
              1 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              4 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
              7 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              7 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0
              5 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              7 0 1 1 0 0 0 1 0 0 2 0 2 2 0 0
              7 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              5 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              4 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              7 0 0 0 1 0 0 1 0 0 2 0 0 0 2 0
              7 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
              7 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
              6 0 0 0 1 0 0 0 0 0 2 0 0 0 2 0
              5 0 0 1 0 0 0 1 0 0 1 0 0 1 0 0
              7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
              4 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              4 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
              7 0 1 0 1 0 0 0 0 0 2 0 2 0 2 0
              6 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0
              6 0 1 1 0 0 0 1 0 0 2 0 2 2 0 0
              7 0 1 1 0 0 0 1 0 0 2 0 2 2 0 0
              5 0 1 1 0 0 0 0 0 0 2 0 2 2 0 0
              1 0 1 0 1 0 0 0 0 0 1 0 1 0 1 0
              3 0 0 0 1 0 0 1 0 0 2 0 0 0 2 0
              end
              label values atwkwfr OATWKWFR
              label def OATWKWFR 1 "[1] Strongly disagree", modify
              label def OATWKWFR 7 "[7] Strongly agree", modify

              Command

              by sex: xtmixed atwkwfr cohort degree FT PT UNEMP married cohabiting sepdivwid parenthood timec cohortxtimec degreextimec FTxtimec PTxtimec UNEMPxtimec || pid: timec, cov(un) variance reml

              Best
              Brendan

              Comment


              • #8
                Thanks Clyde for your assistance

                The code I am using:

                by sex: xtmixed atwkwfr cohort degree FT PT UNEMP married cohabiting sepdivwid parenthood timec cohortxtimec degreextimec FTxtimec PTxtimec UNEMPxtimec || pid: timec, cov(un) variance reml

                Time is continuous and you are right they are indicator variables of the same categorical variables. When I post the dataex command it refers it to the admin for checking cause it looks like spam.

                Best
                Brendan
                Last edited by Brendan Churchill; 21 Feb 2018, 15:41.

                Comment


                • #9
                  Odd that the -dataex- threw you into spam. So try this:
                  Code:
                  gen emp_status = 1 if FT
                  replace emp_status = 2 if PT
                  replace emp_status = 3 if UNEMP
                  label define emp_status    1    "FT"    ///
                                          2    "PT"    ///
                                          3    "UNEMP"
                  label values emp_status emp_status
                  
                  gen marital_status = 1 if married
                  replace marital_status = 2 if cohabiting
                  replace marital_status = 3 if sepdivwid
                  label define marital_status    1    "Married"    ///
                                              2    "Cohabiting"    ///
                                              3    "Sep/Wid/Div"
                  label values marital_status marital_status
                  
                  mixed atwkwfr i.sex##(i.cohort i.degree i.emp_status)##c.timec ///
                      i.sex##(i.parenthood i.marital_status) || pid:timec, cov(un) variance reml
                  
                  margins sex, dydx(*)
                  Because you could not post data, I could not test the code. Beware of typos, unbalanced quotes or parens, etc. I tried to be careful, but...

                  Comment


                  • #10
                    Dear Clyde

                    Many, many thanks! This is perfect! The code works - thank you for your patience.

                    The margins command works too and I can see now what you mean. Is there a way to get the margins for the sex * time coefficients as well? The margins sex, dydx(*) just gives me the margins for the sex differences at the intercept level.

                    And in terms of interpreting the output for the marginal effects for sex - can I say that these differences between men and women are significant?

                    Again thank you!
                    Last edited by Brendan Churchill; 21 Feb 2018, 17:37.

                    Comment


                    • #11
                      You cannot get the marginal effect of something that is itself an interaction term: it is undefined; it does not exist.

                      I would start by saying that the term "significant" should never be used in isolation. If you are referring to "statistically significant," then say both words. And in that case, the statistics in the regression output (not the margins output) in the rows that give coefficients for sex#whatever are the answer to that question. If you mean "significant in practical terms," then say those words. In that case, it is a judgment call you must make considering the marginal effects themselves in the context of your problem and what difference would be large enough to be significant in practical terms.

                      Comment

                      Working...
                      X