Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Interpreting marginal effects output where outcome is change in two logged values

    I’m running a regression in which the outcome is the change in log hourly wages between two time periods (l_occ_hwage_chg). To generate the outcome variable, I took the difference between log hourly wages in month t+1 with log hourly wages in month t. I want to understand a) how the outcome (change in log hourly wages) changes over two 6-month time periods (timeperiod==1 (months 7-12); timeperiod==0 (months 1-6)); and b) how the change over the two time periods differs between two sets of states: one where a policy of interest is in place (policy==1); and another where the policy is NOT in place (policy==0).

    I’m using the following code; and follow it with a margins command that--I think--tells me how much the outcome (l_occ_wage_chg) changes between timeperiod==1 and timeperiod==0, in states where policy==1; and in states where policy==0. My question: how do I interpret the output of the marginal effects command? Pasting some sample output below. My outcome is the change in two logged values. So, looking at the pasted output, is it accurate to exponentiate -.056406 and -.1061071? How do I interpret the difference in these values?

    Code:
    regress l_occ_wage_chg policy##timeperiod `controls’ …
    margins i.policy, dydx(i.timeperiod) pwcompare(cimargins effects)
    Code:
     margins i.policy, dydx(i.timeperiod) pwcompare(cimargins effects)
    
    Pairwise comparisons of average marginal effects
    
    Model VCE: Robust                                        Number of obs = 7,118
    
    Expression: Linear prediction, predict()
    dy/dx wrt:  1.prepostperiod_ma
    
    ---------------------------------------------------------------------
                        |            Delta-method         Unadjusted
                        |     Margin   std. err.     [95% conf. interval]
    --------------------+------------------------------------------------
    0.timeperiod  |  (base outcome)
    --------------------+------------------------------------------------
    1.timeperiod  |
                policy |
                     0  |   -.056406   .0573843     -.1716658    .0588539
                     1  |  -.1061071   .0592991     -.2252127    .0129986
    ---------------------------------------------------------------------
    Note: dy/dx for factor levels is the discrete change from the base
          level.
    
    -------------------------------------------------------------------------------------
                        |   Contrast Delta-method    Unadjusted           Unadjusted
                        |      dy/dx   std. err.      t    P>|t|     [95% conf. interval]
    --------------------+----------------------------------------------------------------
    0.timeperiod  |  (base outcome)
    --------------------+----------------------------------------------------------------
    1.timeperiod  |
                 policy |
                1 vs 0  |  -.0497011   .0233979    -2.12   0.039    -.0966971   -.0027051
    -------------------------------------------------------------------------------------
    Note: dy/dx for factor levels is the discrete change from the base level.

  • #2
    Clyde Schechter or Nick Cox do you have any thoughts on this? I posted it midday yesterday. Thank you in advance for any help you can provide.

    Comment


    • #3
      Well, yes, you can exponentiate those margins. But be careful about what that is. The log transformation is non-linear. So neither its its inverse, the exponential. The margins you have gotten are arithmetic mean values of modeled log wages (under the appropriate policy conditions). But if you exponentiate those you do not get the arithmetic mean value of wages. Rather what you get is the geometric mean of modeled wages. The geometric mean is always less than or equal to the arithmetic mean, with equality holding only when the numbers involved are all equal.

      As for the difference in those margins, it is the difference in the change in log wages between the policy and non-policy states. Now, the change in log wages is the same thing as the log of the ratio of the wages in the two time periods. So you could also call the difference between the margins the difference between policy and non-policy jurisdictions in the logarithm of the ratio of time 1 wages to time 0 wages. If you exponentiate it,you get a ratio of ratios: the ratio (policy:non-policy) of ratios(time 1:time 0) wages.



      Comment


      • #4
        Clyde, many thanks, as always. There's lots to unpack here. I'm trying to translate these values into dollar amounts, so I can understand if there are real/practical differences between the two groups of states. It sounds like you're not advocating NOT TO exponentiate the margins; just to be clear about what the exponentiated values are?

        Is there a way for me to ignore the margins values, and just report the difference in the margins? Is there a way to transform it into a number that can be described with plainer language?

        Comment


        • #5
          It sounds like you're not advocating NOT TO exponentiate the margins; just to be clear about what the exponentiated values are?
          My stance on whether to exponentiate the margins results is neutral. But if you do, you need to be clear in your presentation that the results are geometric means, not arithmetic means. If that is going to baffle your audience, then it is best not to do that. If they will take it in stride, then I would do it.

          Is there a way for me to ignore the margins values, and just report the difference in the margins?
          Well, it's your project and you can choose to present whatever you like. The question is whether it will be a fair and complete presentation of the results and whether your audience will understand it. When you exponentiate and go back to the wage (as opposed to log wage) metric, you change differences into ratios. So, yes, you can report the timepoint 1: timepoint 0 ratios in each policy group and then the policy-associated ratio of ratios, or you can just report the ratio of ratios. For my part, I am usually skeptical of ratios if I am not given the base value that the ratio applies to. In my field, we often hear claims about how some treatment reduces the incidence of some bad outcome by 70%, but nobody says that the outcome is so rare to start with that the absolute difference is too small to be of any importance. Or it can work the other way: a 2% reduction in incidence of a bad outcome sounds unimpressive, but if the bad outcome is really frequent, that can mean a lot of bad outcomes prevented. In general, I think that when you present ratios, it is best to also show the base values.

          The basic difficulty arises from having used a log-transformed outcome variable when what you really want is to report results in terms of the untransformed variable. That is always somewhat problematic. If the range of values of the wages in your data is small, then the log transformation might be approximately linear over that narrow range, and if that is the case, a linear model of wages will fit the data about as well as the model of log wages, but would lead to simpler interpretation. But if the range of wages is large, as is likely unless you specifically selected a sample with a narrow range of wages, then you are stuck with what you have.

          Comment


          • #6
            Clyde Schechter, I wanted to follow up with this, as I'm realizing that my initial interpretation of the above results was incorrect. As a reminder, my outcome is the change in log hourly wages between two time periods. The marginal effects help me to understand how the outcome, change in log hourly wages, *changes* once a cluster of federal policies activates/turns on (prepostperiod_ma==1). I'm also interested in understanding what this change looks like in certain states, distinguished by particular state policies of interest (lorech==1; lorech==0). I'm pasting some updated output below.

            Is it correct to say that the exponentiated versions of the below marginal effects results are *ratios*? (If so, apologies for misunderstanding your earlier responses, above.)

            So, below, -0.07947 == 0.92. This says to me that the change in log hourly wages between the two time periods declined; the average change in log hourly wages when prepostperiod_ma==1 is 92 percent of the change when prepostperiod_ma==0. Is this correct?

            Similarly, jumping to the next set of results, that incorporate my state policy of interest, in states where lorech==1, the change in log hourly wages in the post-period is 90 percent of the change in the pre-period (-0.1060 exponentiated); and in states where lorech==0, the change is 95 percent (-0.0559 exponentiated). Is this correct? Thank you for your help.

            Code:
            margins, dydx(prepostperiod_ma) 
            
            Average marginal effects                                 Number of obs = 7,118
            Model VCE: Robust
            
            Expression: Linear prediction, predict()
            dy/dx wrt:  1.prepostperiod_ma
            
            ------------------------------------------------------------------------------------
                               |            Delta-method
                               |      dy/dx   std. err.      t    P>|t|     [95% conf. interval]
            -------------------+----------------------------------------------------------------
            1.prepostperiod_ma |  -.0794732   .0569822    -1.39   0.169    -.1939252    .0349789
            ------------------------------------------------------------------------------------
            Note: dy/dx for factor levels is the discrete change from the base level.
            
            . margins i.lorech, dydx(prepostperiod_ma) pwcompare(cimargins effects)
            
            Pairwise comparisons of average marginal effects
            
            Model VCE: Robust                                        Number of obs = 7,118
            
            Expression: Linear prediction, predict()
            dy/dx wrt:  1.prepostperiod_ma
            
            ---------------------------------------------------------------------
                                |            Delta-method         Unadjusted
                                |     Margin   std. err.     [95% conf. interval]
            --------------------+------------------------------------------------
            0.prepostperiod_ma  |  (base outcome)
            --------------------+------------------------------------------------
            1.prepostperiod_ma  |
                         lorech |
                             0  |  -.0559067   .0571263     -.1706482    .0588348
                             1  |  -.1059966   .0592849     -.2250738    .0130807
            ---------------------------------------------------------------------
            Note: dy/dx for factor levels is the discrete change from the base
                  level.
            
            -------------------------------------------------------------------------------------
                                |   Contrast Delta-method    Unadjusted           Unadjusted
                                |      dy/dx   std. err.      t    P>|t|     [95% conf. interval]
            --------------------+----------------------------------------------------------------
            0.prepostperiod_ma  |  (base outcome)
            --------------------+----------------------------------------------------------------
            1.prepostperiod_ma  |
                         lorech |
                        1 vs 0  |  -.0500899   .0232503    -2.15   0.036    -.0967896   -.0033902
            -------------------------------------------------------------------------------------
            Note: dy/dx for factor levels is the discrete change from the base level.
            Code:
            regress l_occ_hwage_chg `mainivs1_pppre' `controls_none' if e(sample) & reemp==1 [pw=wtfinl], vce(cluster statefip) 
            (sum of wgt is 25,386,788.8075)
            
            Linear regression                               Number of obs     =      7,118
                                                            F(49, 50)         =          .
                                                            Prob > F          =          .
                                                            R-squared         =     0.1622
                                                            Root MSE          =     .34961
            
                                                             (Std. err. adjusted for 51 clusters in statefip)
            -------------------------------------------------------------------------------------------------
                                            |               Robust
                            l_occ_hwage_chg | Coefficient  std. err.      t    P>|t|     [95% conf. interval]
            --------------------------------+----------------------------------------------------------------
                                   1.lorech |   .0473856   .0204864     2.31   0.025     .0062375    .0885337
                         1.prepostperiod_ma |  -.0559067   .0571263    -0.98   0.332    -.1706482    .0588348
                                            |
                    lorech#prepostperiod_ma |
                                       1 1  |  -.0500899   .0232503    -2.15   0.036    -.0967896   -.0033902
                                            |
                             1.searchwaive1 |   .0551334   .0456437     1.21   0.233    -.0365446    .1468114
                                            |
                                  age_group |
                                     18-24  |  -.0435375   .0179908    -2.42   0.019    -.0796732   -.0074019
                                     25-34  |   -.005773   .0172215    -0.34   0.739    -.0403634    .0288175
                                     45-54  |   .0233387   .0141264     1.65   0.105     -.005035    .0517123
                                     55-64  |  -.0172704   .0156683    -1.10   0.276     -.048741    .0142003
                                            |
                                  race_wbho |
                                2 black nh  |  -.0083773   .0160294    -0.52   0.604    -.0405732    .0238187
                         3 hispanic/latino  |  -.0256024   .0199551    -1.28   0.205    -.0656833    .0144786
                                  other nh  |  -.0238075    .016227    -1.47   0.149    -.0564003    .0087853
                                            |
                                       edu4 |
                            1 Less than HS  |  -.1295791   .0207048    -6.26   0.000     -.171166   -.0879923
                               2 HS or GED  |  -.1099707   .0224062    -4.91   0.000    -.1549749   -.0649665
            3 Some college or Associate's'  |  -.0890481    .017445    -5.10   0.000    -.1240875   -.0540088
                                            |
                                    1.woman |  -.0292426     .01655    -1.77   0.083    -.0624842    .0039991
                                1.marstdum1 |    .035722   .0095032     3.76   0.000     .0166342    .0548098
                                            |
                                 ownkidd_18 |
               1: Own children, <18, in HH  |  -.0088366   .0162277    -0.54   0.588    -.0414308    .0237576
                                 1.womarkid |  -.0123339   .0198301    -0.62   0.537    -.0521639    .0274961
                                            |
                                   ind_nilf |
                                         2  |   .0153223   .1016407     0.15   0.881    -.1888291    .2194737
                                         3  |  -.0045394   .1028172    -0.04   0.965    -.2110537     .201975
                                         4  |  -.0397672    .106125    -0.37   0.709    -.2529255    .1733912
                                         5  |  -.0375082   .1048453    -0.36   0.722    -.2480963    .1730798
                                         6  |  -.0925678   .1010772    -0.92   0.364    -.2955874    .1104518
                                         7  |  -.0393424   .1257603    -0.31   0.756    -.2919394    .2132546
                                         8  |  -.0677047   .1091507    -0.62   0.538    -.2869403    .1515309
                                         9  |  -.0141583    .105671    -0.13   0.894    -.2264047    .1980882
                                        10  |   .0010643   .1051823     0.01   0.992    -.2102005    .2123291
                                        11  |  -.0072031   .1066915    -0.07   0.946    -.2214993    .2070932
                                        12  |  -.0654854   .1088095    -0.60   0.550    -.2840358     .153065
                                        13  |  -.0424712   .1126917    -0.38   0.708    -.2688191    .1838767
                                            |
                                   occ_nilf |
                                         2  |   .3168519   .0330694     9.58   0.000     .2504301    .3832736
                                         3  |   .5725422   .0236111    24.25   0.000     .5251178    .6199665
                                         4  |   .5406756   .0307955    17.56   0.000      .478821    .6025302
                                         5  |     .53857   .0278519    19.34   0.000     .4826278    .5945123
                                         6  |   .5851877   .1074292     5.45   0.000     .3694098    .8009655
                                         7  |   .3989838   .0362269    11.01   0.000     .3262199    .4717477
                                         8  |   .3781024     .03613    10.47   0.000     .3055331    .4506717
                                         9  |   .4872697   .0292269    16.67   0.000     .4285658    .5459736
                                        10  |   .5417359   .0336985    16.08   0.000     .4740505    .6094214
                                            |
                                    sampjl1 |   .0069019   .0166205     0.42   0.680    -.0264812    .0402851
                                      ur_sa |   .0209907   .0156384     1.34   0.186      -.01042    .0524013
                                     ur2_sa |  -.0010451   .0010695    -0.98   0.333    -.0031933    .0011031
                                     ur3_sa |   .0000182    .000023     0.79   0.431    -.0000279    .0000643
                                        iur |   .0046998   .0116543     0.40   0.688    -.0187084    .0281081
                                       iur2 |  -.0008163   .0009876    -0.83   0.412       -.0028    .0011673
                                       iur3 |   .0000291   .0000265     1.10   0.278    -.0000242    .0000823
                                 empgrowth1 |  -.0108142   .0034052    -3.18   0.003    -.0176537   -.0039747
                                       emp2 |  -.0000283   .0002341    -0.12   0.904    -.0004985     .000442
                                       emp3 |   9.87e-06   .0000178     0.55   0.582    -.0000259    .0000456
                                incrate_jhu |   .0000358   .0000255     1.40   0.166    -.0000154    .0000869
                                    stringd |   .0000282   .0006635     0.04   0.966    -.0013045    .0013609
                                      _cons |  -.4331511   .0972737    -4.45   0.000     -.628531   -.2377712
            -------------------------------------------------------------------------------------------------

            Comment


            • #7
              If the outcome variable itself is a change of log wages, then exponentiating the outcome produces a ratio of wages. The marginal effect of a regressor on the outcome variable is then a difference in change of log wages. Exponentiating that will give you a ratio of change in log wages.

              So, below, -0.07947 == 0.92. This says to me that the change in log hourly wages between the two time periods declined; the average change in log hourly wages when prepostperiod_ma==1 is 92 percent of the change when prepostperiod_ma==0. Is this correct?
              Yes, that is correct.

              Similarly, jumping to the next set of results, that incorporate my state policy of interest, in states where lorech==1, the change in log hourly wages in the post-period is 90 percent of the change in the pre-period (-0.1060 exponentiated); and in states where lorech==0, the change is 95 percent (-0.0559 exponentiated). Is this correct?
              Yes, this is also correct.

              Comment


              • #8
                Thank you, Clyde. I was interpreting the difference in the marginal effects as a ratio, but not the respective values themselves (for some reason).

                Comment


                • #9
                  Clyde Schechter, as a follow up, I was hoping you could confirm that I'm correctly interpreting descriptive stats of my outcome, change in log of occupational hourly wages. I'm pasting some sample output below. Each table shows average values of the outcome for a set of states without a policy (rq==0), and a set of states with a policy (rq==1). The first table is for months 1 to 6; the second table is for months 7 to 12. I was under the impression that I should exponentiate these values. So in the case of the first row of output, below, a value 0.01196 exponentiated equals 1.012 (a ratio). Subtracting 1 from the ratio gives me 0.012. Is it correct for me to say that, in months 1 to 6, in states where rq==0, the average value of my outcome, change in log of hourly wages, equaled 0.012, or +1.2 percent? Or do I take these at face value, and interpret them as mean percent change? Thank you as always.

                  Code:
                  . tabstat l_occ_hwage_chg if e(sample) & postperiod_c==0 [aw=wtfinl], by(rq) stat(mea
                  > n sd n) col(stat) long
                  
                  rq           Variable |      Mean        SD         N
                  ----------------------+------------------------------
                  0        l_occ_hwag~g |  .0119639  .3987895      1183
                  1        l_occ_hwag~g |  .0232074  .4511514       359
                  ----------------------+------------------------------
                  Total    l_occ_hwag~g |  .0145712   .411401      1542
                  -----------------------------------------------------
                  
                  . tabstat l_occ_hwage_chg if e(sample) & postperiod_c==1 [aw=wtfinl], by(rq) stat(mea
                  > n sd n) col(stat) long
                  
                  rq           Variable |      Mean        SD         N
                  ----------------------+------------------------------
                  0        l_occ_hwag~g | -.0281099    .45073       870
                  1        l_occ_hwag~g |  .0073266  .4111804       261
                  ----------------------+------------------------------
                  Total    l_occ_hwag~g | -.0197725    .44182      1131
                  -----------------------------------------------------

                  Comment


                  • #10
                    So in the case of the first row of output, below, a value 0.01196 exponentiated equals 1.012 (a ratio). Subtracting 1 from the ratio gives me 0.012. Is it correct for me to say that, in months 1 to 6, in states where rq==0, the average value of my outcome, change in log of hourly wages, equaled 0.012, or +1.2 percent?
                    You've got the gist of it, but not quite. The mean change in log hourly wage is just the 0.01196 that came out of -tabstat-. When you exponentiate ad subtract 1, the resulting 0.012 can be described as: the average hourly wage increase was 1.2%. (N.B. this is a change in the hourly wage increase itself, not the log wage increase.)

                    Comment


                    • #11
                      I see. Okay. This is very helpful clarification; I'm glad I checked. Thank you so much

                      Comment

                      Working...
                      X