What is the best test to compare regression coefficients when we use a probit model?

[Bilal rafhi]

Hello,

I run two regressions for the same sample when the only difference is that there I include one more variable in the second.
Regression 1 : y =beta0 + beta1 X1 + u
Regression 2 . y=beta0 + beta1 X1 + beta2 X2 + u

I want to know if there is a significant difference between the beta1 of the regression 1 and the beta1 of the regression 2.

I use the chi-square test providing by the "test" command : (the variable of interest is a binary variable "fa_ppr_man_2"

probit ppr_man_2 fa_ppr_man_2 `Socio_Eco' `Attitudes' `Parents_Socio_Eco'
est sto f0
probit ppr_man_2 fa_ppr_man_2 `Socio_Eco' `Attitudes' `Parents_Socio_Eco' mo_ppr_man_2
est sto f1
suest f0 f1, cluster(hhnrakt)
test [f0_ppr_man_2]fa_ppr_man_2=[f1_ppr_man_2]fa_ppr_man_2

It is the good thing to do? Or there is another way to test the hypothesis of equality of coefficients stemming from a probit regression?

Thank you very much

[Maarten Buis]

This is a surprisingly complex problem, see for instance (Kuha & Mills 2018). The bottom line is that the coefficients cannot be compared. So the test you are proposing is meaningless. The short version of the argument is that if you think of a probit model in terms of a latent propensity, then the scale of that dependent variable is defined in a probit model by the variance of the residual. So if you add variables to your model, you change the variance of the residual and thus the scale of the dependent variable. If you think of a probit model as a model for a probability, then it is a model for a conditional probability and the probability conditional on different variables are different and incomparable.You could consider using khb (type in Stata findit khb and install it, the help file contains references where you can read more about this method)


Kuha, J., & Mills, C. (2018). On Group Comparisons With Logistic Regression Models. Sociological Methods & Research. https://doi.org/10.1177/0049124117747306

[Joseph Coveney]

I want to know if there is a significant difference between the beta1 of the regression 1 and the beta1 of the regression 2.

The issue that Maarten mentions aside, what does the test of the difference of the regression coefficient between the two models actually test?

regress y c.a

versus

regress y c.(a b) // (same dataset)

It seems to me that the regression coefficient on a would only be affected if there is a correlation between a and b (regardless of whether b affects y directly).

Couldn't you get at this more economically by inspection of the interaction term in a single model?

regress y c.a##c.b

[Maarten Buis]

Joseph Coveney Your first two regression would be a model of mediation. The classic example in my field would be that y is own social status, a is parental social status, and b is own education. The argument is that children from high status parents attain a high status themselves, and they do so by getting a higher education. Testing whether the effect of a is equal across models is equivalent to testing whether there is no indirect effect of parental background through own education on own status. It test if this mechanism can explain part of the effect of parental background on own status. This would make sense, the problem is that this nice result does not generalize to a probit model as I mentioned in #2.

It seems to me that the regression coefficient on a would only be affected if there is a correlation between a and b (regardless of whether b affects y directly).

That is incorrect, if b doest not effect y, then the effect of a remains unchanged regardless of the correlation between a and b. In order for this test to make sense we need to believe that a effects b (and not the other way around, because then b would be a confounding variable rather than an mediating variable) and b must effect y and not the other way around.

Your second model would be a model of moderation, and tests something very different: it tests whether the effect of parental background is the same regardless of own education.

[Joseph Coveney]

Maarten, Thank you very much for taking the time to straighten me out on that. Your explanation is quite clear and I appreciate it.

[Bilal rafhi]

@Maarten Buis Thank you very much for your answers. I am reading the paper you sent, and I tried the khb stata command and can say that it is very interesting... I am looking now for some applications of this command on published papers. I will probably have some more questions about this, but I am waiting to master more or less the issue...
Thank you again.