Predicted probability for another probit regression

[Felix Stein]

Hi all,

I am performing the following task:

1. Perform a probit regression on a binary variable.
2. Predict the probability from 1.
3. Use the predicted probability in another probit regression on another binary variable as one of the explanatory variable.

The outcome of the marginal effects is however disturbing as the predicted variable has a marginal effect that is larger than 1 and statistical significant. Does anyone have an idea how to solve this?

Thanks in advance.
Felix

[Richard Williams]

Showing commands and output usually makes it much easier to offer help. My first guess is that the marginal effect is for xb, not pr, but I can't really say without seeing the commands and output. Or, x is measured in such a way that a one-unit increase is nonsensical. So say something about the ranges of the xariables.

[Felix Stein]

 probit event Wabnormal_return WBM Wl_MV WCASH WLEVERAGE WLIQUIDITY WGROWTH WPE WROA i.fyear, vce(cluster gvkey)

Iteration 0:   log pseudolikelihood = -6670.7601  
Iteration 1:   log pseudolikelihood = -6533.3363  
Iteration 2:   log pseudolikelihood = -6531.3059  
Iteration 3:   log pseudolikelihood = -6531.3037  
Iteration 4:   log pseudolikelihood = -6531.3037  

Probit regression                                 Number of obs   =      31658
                                                  Wald chi2(22)   =     298.11
                                                  Prob > chi2     =     0.0000
Log pseudolikelihood = -6531.3037                 Pseudo R2       =     0.0209

                                   (Std. Err. adjusted for 3946 clusters in gvkey)
----------------------------------------------------------------------------------
                 |               Robust
           event |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
Wabnormal_return |  -.0387346   .0146737    -2.64   0.008    -.0674946   -.0099746
             WBM |   .0827398   .0255166     3.24   0.001     .0327282    .1327515
           Wl_MV |  -.0749629   .0077432    -9.68   0.000    -.0901392   -.0597865
           WCASH |   .4258666   .1267448     3.36   0.001     .1774514    .6742819
       WLEVERAGE |   .0298889   .0368147     0.81   0.417    -.0422665    .1020444
      WLIQUIDITY |  -.1658398   .1175392    -1.41   0.158    -.3962124    .0645329
         WGROWTH |  -.0228236   .0274611    -0.83   0.406    -.0766464    .0309991
             WPE |  -.0005461   .0002466    -2.21   0.027    -.0010294   -.0000627
            WROA |   .2836581   .0674889     4.20   0.000     .1513823    .4159339
                 |
           fyear |
           2002  |   .1860382   .0677177     2.75   0.006      .053314    .3187624
           2003  |   .2772639   .0674374     4.11   0.000      .145089    .4094388
           2004  |    .376771    .066619     5.66   0.000     .2462001    .5073418
           2005  |    .541105   .0646411     8.37   0.000     .4144107    .6677993
           2006  |   .5321042   .0651992     8.16   0.000      .404316    .6598923
           2007  |    .379758   .0674916     5.63   0.000     .2474769     .512039
           2008  |   .2651151   .0690762     3.84   0.000     .1297283    .4005019
           2009  |   .4756964   .0660668     7.20   0.000     .3462078    .6051851
           2010  |   .3665336   .0682916     5.37   0.000     .2326845    .5003827
           2011  |   .3791613   .0690237     5.49   0.000     .2438772    .5144453
           2012  |   .3065319   .0713653     4.30   0.000     .1666585    .4464053
           2013  |   .3728689   .0708611     5.26   0.000     .2339836    .5117542
           2014  |   .3903023   .0735024     5.31   0.000     .2462403    .5343643
                 |
           _cons |  -1.320464    .109209   -12.09   0.000    -1.534509   -1.106418
----------------------------------------------------------------------------------

. predict event_new
(option pr assumed; Pr(event))

.
.
. probit poison_pill event_new  Wabnormal_return WBM Wl_MV WCASH WLEVERAGE WLIQUIDITY WGROWTH WPE WROA i.fyear, vce(cluster gvkey)

Iteration 0:   log pseudolikelihood =  -20826.63  
Iteration 1:   log pseudolikelihood = -19430.523  
Iteration 2:   log pseudolikelihood = -19421.063  
Iteration 3:   log pseudolikelihood = -19421.058  
Iteration 4:   log pseudolikelihood = -19421.058  

Probit regression                                 Number of obs   =      31658
                                                  Wald chi2(23)   =     896.64
                                                  Prob > chi2     =     0.0000
Log pseudolikelihood = -19421.058                 Pseudo R2       =     0.0675

                                   (Std. Err. adjusted for 3946 clusters in gvkey)
----------------------------------------------------------------------------------
                 |               Robust
     poison_pill |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
       event_new |   5.356169   2.648284     2.02   0.043     .1656285    10.54671
Wabnormal_return |     .02313   .0134241     1.72   0.085    -.0031807    .0494407
             WBM |   .0202504   .0414477     0.49   0.625    -.0609856    .1014863
           Wl_MV |   .0364977   .0226462     1.61   0.107    -.0078881    .0808834
           WCASH |   .0381984   .2174349     0.18   0.861    -.3879662     .464363
       WLEVERAGE |   .0975133   .0468744     2.08   0.037     .0056411    .1893854
      WLIQUIDITY |   .0930903   .1787079     0.52   0.602    -.2571707    .4433513
         WGROWTH |   .0020966   .0217377     0.10   0.923    -.0405085    .0447016
             WPE |   .0004134   .0002285     1.81   0.070    -.0000345    .0008612
            WROA |  -.4376833   .1184321    -3.70   0.000     -.669806   -.2055606
                 |
           fyear |
           2002  |  -.0615023   .0373987    -1.64   0.100    -.1348025    .0117978
           2003  |  -.0750833   .0600301    -1.25   0.211    -.1927401    .0425735
           2004  |  -.1517793   .0869717    -1.75   0.081    -.3222407    .0186821
           2005  |  -.2974706   .1407836    -2.11   0.035    -.5734015   -.0215397
           2006  |  -.4268777   .1372755    -3.11   0.002    -.6959327   -.1578227
           2007  |  -.4517787   .0912173    -4.95   0.000    -.6305614    -.272996
           2008  |  -.5685894   .0653396    -8.70   0.000    -.6966527   -.4405262
           2009  |  -.8121553     .12379    -6.56   0.000    -1.054779   -.5695314
           2010  |  -.8260459   .0906915    -9.11   0.000    -1.003798   -.6482938
           2011  |  -.9280481   .0950836    -9.76   0.000    -1.114409   -.7416877
           2012  |   -1.03727    .076577   -13.55   0.000    -1.187358   -.8871813
           2013  |  -1.179013   .0933201   -12.63   0.000    -1.361917   -.9961084
           2014  |  -1.258285   .0989688   -12.71   0.000     -1.45226    -1.06431
                 |
           _cons |  -.8141577   .2755597    -2.95   0.003    -1.354245   -.2740706
----------------------------------------------------------------------------------

 margins, dydx(*) post atmeans

Conditional marginal effects                      Number of obs   =      31658
Model VCE    : Robust

Expression   : Pr(poison_pill), predict()
dy/dx w.r.t. : event_new Wabnormal_return WBM Wl_MV WCASH WLEVERAGE WLIQUIDITY WGROWTH WPE WROA 2002.fyear 2003.fyear 2004.fyear
               2005.fyear 2006.fyear 2007.fyear 2008.fyear 2009.fyear 2010.fyear 2011.fyear 2012.fyear 2013.fyear 2014.fyear
at           : event_new       =    .0542094 (mean)
               Wabnormal_~n    =   -.0434894 (mean)
               WBM             =    .5626435 (mean)
               Wl_MV           =     6.26644 (mean)
               WCASH           =    .2267539 (mean)
               WLEVERAGE       =   -.0419197 (mean)
               WLIQUIDITY      =    .3702181 (mean)
               WGROWTH         =    .1229756 (mean)
               WPE             =    12.36587 (mean)
               WROA            =   -.9233084 (mean)
               2001.fyear      =    .0784636 (mean)
               2002.fyear      =    .0832965 (mean)
               2003.fyear      =    .0807695 (mean)
               2004.fyear      =     .078969 (mean)
               2005.fyear      =    .0758102 (mean)
               2006.fyear      =    .0748626 (mean)
               2007.fyear      =    .0723356 (mean)
               2008.fyear      =    .0712932 (mean)
               2009.fyear      =    .0711037 (mean)
               2010.fyear      =    .0678186 (mean)
               2011.fyear      =    .0651336 (mean)
               2012.fyear      =    .0639649 (mean)
               2013.fyear      =    .0623223 (mean)
               2014.fyear      =    .0538568 (mean)

----------------------------------------------------------------------------------
                 |            Delta-method
                 |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-----------------+----------------------------------------------------------------
       event_new |   1.996251   .9868536     2.02   0.043     .0620534    3.930449
Wabnormal_return |   .0086206   .0050036     1.72   0.085    -.0011862    .0184274
             WBM |   .0075473   .0154472     0.49   0.625    -.0227285    .0378232
           Wl_MV |   .0136027   .0084369     1.61   0.107    -.0029333    .0301388
           WCASH |   .0142366    .081039     0.18   0.861     -.144597    .1730702
       WLEVERAGE |   .0363433   .0174539     2.08   0.037     .0021342    .0705524
      WLIQUIDITY |   .0346949    .066594     0.52   0.602    -.0958269    .1652167
         WGROWTH |   .0007814   .0081016     0.10   0.923    -.0150975    .0166603
             WPE |   .0001541   .0000852     1.81   0.070    -.0000129     .000321
            WROA |  -.1631251   .0440838    -3.70   0.000    -.2495278   -.0767225
                 |
           fyear |
           2002  |  -.0243158   .0145992    -1.67   0.096    -.0529298    .0042982
           2003  |  -.0297092   .0235621    -1.26   0.207    -.0758901    .0164718
           2004  |  -.0602602   .0342768    -1.76   0.079    -.1274415    .0069212
           2005  |  -.1182241   .0555245    -2.13   0.033    -.2270501   -.0093982
           2006  |  -.1688105   .0535602    -3.15   0.002    -.2737865   -.0638345
           2007  |  -.1783744   .0357726    -4.99   0.000    -.2484874   -.1082614
           2008  |    -.22224   .0259889    -8.55   0.000    -.2731774   -.1713027
           2009  |  -.3066795   .0456548    -6.72   0.000    -.3961612   -.2171977
           2010  |  -.3111503    .034843    -8.93   0.000    -.3794412   -.2428593
           2011  |  -.3426978    .035907    -9.54   0.000    -.4130742   -.2723214
           2012  |  -.3738544   .0302473   -12.36   0.000    -.4331381   -.3145707
           2013  |  -.4100564   .0344836   -11.89   0.000    -.4776431   -.3424697
           2014  |  -.4281879   .0354128   -12.09   0.000    -.4975957     -.35878
----------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

[Richard Williams]

So, you are worried because the marginal effect of event_new = 1.996? But that doesn't mean much. Marginal effects are affected by the scaling of variables, and here you have a variable that only ranges between 0 and 1 (possibly much less). Change the scaling of variables and you change the marginal effect. For example,

webuse nhanes2f, clear
probit diabetes weight
margins, dydx(*) atmeans
gen xweight = weight/10000
probit diabetes xweight
margins, dydx(*) atmeans

In the original probit the marginal effect is .000877. After I rescale, the marginal effect is 8.770118. Nothing has really changed though; I have just rescaled weight.

There could be problems lurking in their somewhere but a marginal effect greater than one for a variable that varies at most between 0 and 1 does not strike me as being one of them.

If you need further proof try rescaling some of your other variables. You will see that their marginal effects change too.

[Felix Stein]

Thanks a lot for your comment. What I however do not understand then is the interpretation... How do I measure the effect of the an increase of the probability of event on the likelihood of poison_pill?

[Felix Stein]

Would it be permissible to scale the predicted probability as it maximum value is only 0.17. meaning that I have a variable that ranges from 0 to 1?

[Maarten Buis]

Your marginal effect tells you that if your predicted probability of event changes from 0 to 1, the predicted probability of poison pill increases by an impossible 2 assuming the effect remains constant over the range of the probability of event. The problem you have is that moving from 0 to 1 is a gross extrapolation, as your maximum is only 0.17. The easiest solution is to multiply the variable event_new by 100, i.e. measure the probability in percentage points. In that case the marginal effect tells you the effect of an percentage point increase in the probability of event rather than the unrealistic change from 0 to 100%.

[Richard Williams]

Even though economists love them, I personally find marginal effects for continuous variables much less interpretable than I do marginal effects for discrete variables. I briefly explain the problems in

http://www3.nd.edu/~rwilliam/stats3/Margins02.pdf

If I want to get a better feel for the effect of a continuous variable, I like to use things like the mcp command:

http://www3.nd.edu/~rwilliam/stats3/Margins03.pdf

http://www.stata-journal.com/article...article=gr0056

[Felix Stein]

Thank you both for the input. I have a follow-on question. I am trying to say whether the marginal effects are also economically significant. Other papers take the marginal effects in relation to the "unconditional" probability. I interpret this unconditional probability as the number of times the dependent variable equals one in relation to all observations used. Is that correct?

[Maarten Buis]

The unconditional probability is just the proportion of ones in your data, which is also the mean assuming you coded your variable as 0 or 1. So you can see that by just using sum poison_pill. You have to be careful, as the marginal effect also depends on the scaling of the explanatory variable, as we have seen earlier in this thread.

[Felix Stein]

Understood. But I have still two questions:

- So my unconditional probability, i.e. proportion poison_pill is 0.37, and my rescaled marginal effect for event_new is 0.0199. How do I interpret whether this is "economically" significant, i.e. how does the probability change when event_new changes by one standard-deviation?
- For another model I have a probit regression an also a binary explanatory as well. How do I calculate how the probability changes when the explanatory variable is 0 vs 1.?

Thanks a lot.

[Richard Williams]

I think adjusted predictions would better answer the questions you are asking than would marginal effects. Again, take a look at the mcp command I mentioned earlier. Also, I suggest you do

findit spost13_ado

and check out the listcoef and mchange commands.

[Felix Stein]

Hi Richard, thanks for your comment and this might be a better way to answer the question however I am supposed to replicate for my analysis something similar to the the following paper on p. 23 last paragraph: http://www.ruf.rice.edu/~jgsfss/Lone...tegemoller.pdf. So I understand how to calculate the unconditional probability and I have the marginal effects. However, in the last sentence of the paragraph on p. 23, they calculate how the probability changes relatively and I do not understand how that is done... I

[Maarten Buis]

That looks like something computed with mchange which is part of the spost package that Rich referred to in #12.

The "relative" that seems to bother you is not included in the computation, the unconditional probability is just mentioned as a value with which the reader can compare the discrete differences. Remember that "economically significant" is just a fancy way of saying "the effect looks big to me"; it is completely and utterly subjective. You can have reasons for thinking an effect is big, you can talk with others about those reasons, but in the end it is just your judgment call. So the authors in effect said "the effects seem small, but if you compare it to the unconditional probability it looks big". There are pros and cons to that argument, so I won't take that argument as is.

[Felix Stein]

Ok so I will look into for the relative change. I have one more question though for the marginal effect of the rescaled variable from the example above despite its downsides mentioned by Richard: How do i have to interpret this marginal effect? Now that the scale has been changed, could I still say what a one standard deviation chang implies? Sorry if this might be a stupid question but I had no education into that topic. Nonetheless, your help is highly appreciated!