conflicting results with -margins command in an ivprobit model

[Jake Fu]

Hello,I'm doing an ivprobit regression and the P value of the main variable is 0.001.However,when I used the -margins command(margins, dydx(*) predict(pr)),the P value of the same variable turned to 0.958.
How can I explain the conflicting results?
The command and the results are as follows.




. ivprobit dstock house tasset inc educ educaq age agesq agdp attitude rural (hrate=lsupply)

Fitting exogenous probit model

Iteration 0: log likelihood = -1996.7619
Iteration 1: log likelihood = -1631.0723
Iteration 2: log likelihood = -1595.3853
Iteration 3: log likelihood = -1594.677
Iteration 4: log likelihood = -1594.675
Iteration 5: log likelihood = -1594.675

Fitting full model

Iteration 0: log likelihood = 16816.674
Iteration 1: log likelihood = 16816.678
Iteration 2: log likelihood = 16816.678

Probit model with endogenous regressors Number of obs = 8,368
Wald chi2(11) = 741.05
Log likelihood = 16816.678 Prob > chi2 = 0.0000

----------------------------------------------------------------------------------------
| Coef. Std. Err. z P>|z| [95% Conf. Interval]
-----------------------+----------------------------------------------------------------
hrate | 12.96557 3.850553 3.37 0.001 5.418623 20.51251
house | .487335 .068053 7.16 0.000 .3539535 .6207165
tasset | .0015486 .0001805 8.58 0.000 .0011947 .0019024
inc | -.0016784 .0006526 -2.57 0.010 -.0029574 -.0003994
educ | .0527656 .0248787 2.12 0.034 .0040042 .1015269
educaq | -.002363 .0008222 -2.87 0.004 -.0039744 -.0007515
age | -.0037075 .0076583 -0.48 0.628 -.0187174 .0113025
agesq | 5.15e-06 .0000725 0.07 0.943 -.000137 .0001473
agdp | -.0324399 .0411792 -0.79 0.431 -.1131496 .0482698
attitude | -.1928362 .0193402 -9.97 0.000 -.2307423 -.1549301
rural | -.7744018 .0729297 -10.62 0.000 -.9173414 -.6314622
_cons | -2.913667 .3720613 -7.83 0.000 -3.642894 -2.18444
-----------------------+----------------------------------------------------------------
corr(e.hrate,e.dstock)| -.3421174 .1059054 -.5310273 -.1208094
sd(e.hrate)| .0268055 .0002072 .0264025 .0272147
----------------------------------------------------------------------------------------
Instrumented: hrate
Instruments: house tasset inc educ educaq age agesq agdp attitude rural lsupply
----------------------------------------------------------------------------------------
Wald test of exogeneity (corr = 0): chi2(1) = 8.83 Prob > chi2 = 0.0030

. margins, dydx(*) predict(pr)

Average marginal effects Number of obs = 8,368
Model VCE : OIM

Expression : Probability of positive outcome, predict(pr)
dy/dx w.r.t. : hrate house tasset inc educ educaq age agesq agdp attitude rural lsupply

------------------------------------------------------------------------------
| Delta-method
| dy/dx Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
hrate | .0217708 .4153892 0.05 0.958 -.7923771 .8359187
house | .045854 .0082969 5.53 0.000 .0295923 .0621157
tasset | .0001756 .0000155 11.33 0.000 .0001452 .000206
inc | -.0001821 .0000676 -2.70 0.007 -.0003145 -.0000497
educ | .0056089 .0026357 2.13 0.033 .000443 .0107748
educaq | -.0002561 .0000865 -2.96 0.003 -.0004257 -.0000866
age | -.0004035 .0008159 -0.49 0.621 -.0020026 .0011957
agesq | 1.46e-06 7.72e-06 0.19 0.850 -.0000137 .0000166
agdp | .0091676 .0037665 2.43 0.015 .0017854 .0165498
attitude | -.020195 .0021912 -9.22 0.000 -.0244898 -.0159003
rural | -.0775225 .0089348 -8.68 0.000 -.0950343 -.0600107
lsupply | -.0008359 .0000787 -10.62 0.000 -.0009902 -.0006816
------------------------------------------------------------------------------

[William Lisowski]

An essentially identical version of this topic was posted earlier at

http://www.statalist.org/forums/foru...ivprobit-model

[Richard Williams]

First off, this would be much easier to read if you used code tags. See pt. 12 of the FAQ.

The significance of a coefficient and of a marginal effect need not be consistent with each other. Also you said (hrate=lsupply) and lsupply is highly significant in the marginal effects so I am guessing that may be part of what is going on.

[Jake Fu]

First off, this would be much easier to read if you used code tags. See pt. 12 of the FAQ.

The significance of a coefficient and of a marginal effect need not be consistent with each other. Also you said (hrate=lsupply) and lsupply is highly significant in the marginal effects so I am guessing that may be part of what is going on.

Sorry，I don't quite understand what you mean.Since the marginal effect isn't significant ,is it still meaningful?
I have just found a question similar to mine,in which Mr.Schechter said

With -logistic- you are getting an odds ratio, and the significance test is on the null hypothesis that the odds ratio = 1. With -margins- you are getting predicted probabilities of the outcome in each category of x, and the significance tests are of the hypothesis that the predicted probability is zero. Since it is rarely if ever reasonable to even entertain a null hypothesis that a predicted probability is zero, the p-values in this output are not really useful and should just be ignored.

http://www.statalist.org/forums/foru...argins-command
Does what he said apply to my problem as well？

[Richard Williams]

I agree with Clyde about P values for probabilities. But, there are other situations where the P values for coefficients and marginal effects don't match up and need not match up. For a discussion see

http://www.statalist.org/forums/foru...s-significance

In your particular case marginal effects are being reported for both hrate and lsupply, and you have specified (hrate=lsupply). In Stata 14.2, if you run the code

webuse laborsup, clear
ivprobit fem_work fem_educ kids (other_inc = male_educ)
margins, dydx(*) predict(pr)

You get

. margins, dydx(*) predict(pr)

Average marginal effects                        Number of obs     =        500
Model VCE    : OIM

Expression   : Probability of positive outcome, predict(pr)
dy/dx w.r.t. : other_inc fem_educ kids male_educ

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
   other_inc |   -.014015   .0009836   -14.25   0.000    -.0159428   -.0120872
    fem_educ |   .0545129   .0066007     8.26   0.000     .0415758      .06745
        kids |  -.0470199   .0123397    -3.81   0.000    -.0712052   -.0228346
   male_educ |          0  (omitted)
------------------------------------------------------------------------------

So, I am kind of surprised that you get marginal effects for both hrate and lsupply, but I haven't run enough ivprobit models to say what usually happens. What version of Stata are you running? Note that in your example, the marginal effects of lsupply are highly significant.

[tolu jerumeh]

hello. please i need assistance on how i can run triple hurdle model on stata 12. i am working on health expenditures of rural households in Nigeria. my hurdle model includes four decision stages- 1Probability of reporting illness (Probit) 2. probability of seeking for treatment (probit) 3. probability of incurring positive medical expenditure (Probit)and 4. the actual medical expenditure of respondents (OLS). please how do i run this analysis using stata 12. please note i will be using the same set of explanatory variables for all the stages. thanks for your prompt response.

[Richard Williams]

Tolu, I suspect you will have a much better chance of getting an answer if you start a new thread with a meaningful title, e.g. "How do I estimate a triple hurdle model in Stata 12?" As it is people who might know the answer may never even look at this thread.

[Irene Selwaness]

I have the same issue as Jake.
Jake, If you bootstrap the standard error of the hrate calculated after the margins command, the hrate marginal effect would become significant again.

Irene

[Jake Fu]

I have the same issue as Jake.
Jake, If you bootstrap the standard error of the hrate calculated after the margins command, the hrate marginal effect would become significant again.

Irene

Hi,Irene. Can you tell me the command you used?I used

bootstrap,reps(1000) :ivprobit..........................
margins,dydx(*) predict(pr)

but the P value rarely changed.

[Irene Selwaness]

Hi Jake, So for your model, if you are interested in the effect of hrate, you would define a program, compute the marginal effect of hrate, extract it and then bootstrap. Here is an example. You can first try with 50 replications to check whether the standard errors is correctly calculated or not.

program define hrate ivprobit dstock house tasset inc educ educaq age agesq agdp attitude rural (hrate=lsupply)
margins, dydx(hrate) predict(p) post
mat beta=e(b)
scalar b= beta[1,1] end

bootstrap b, seed(39923) reps(400): hrate

However, why does this problem in the margins command happen with ivprobit ?

Irene

[Jake Fu]

Hi Jake, So for your model, if you are interested in the effect of hrate, you would define a program, compute the marginal effect of hrate, extract it and then bootstrap. Here is an example. You can first try with 50 replications to check whether the standard errors is correctly calculated or not.

program define hrate ivprobit dstock house tasset inc educ educaq age agesq agdp attitude rural (hrate=lsupply)
margins, dydx(hrate) predict(p) post
mat beta=e(b)
scalar b= beta[1,1] end

bootstrap b, seed(39923) reps(400): hrate

However, why does this problem in the margins command happen with ivprobit ?

Irene

Hi，Irene.Thanks to your code,I finally got my marginal effects significant.^_^ It's so nice of you.

[George Pham]

Hi Jake, So for your model, if you are interested in the effect of hrate, you would define a program, compute the marginal effect of hrate, extract it and then bootstrap. Here is an example. You can first try with 50 replications to check whether the standard errors is correctly calculated or not.

program define hrate ivprobit dstock house tasset inc educ educaq age agesq agdp attitude rural (hrate=lsupply)
margins, dydx(hrate) predict(p) post
mat beta=e(b)
scalar b= beta[1,1] end

bootstrap b, seed(39923) reps(400): hrate

However, why does this problem in the margins command happen with ivprobit ?

Irene

Hi Irene,

How would you write that program including a cluster variable?

Thank you,
George