Testing the equality of parameters from a two margins regressions

[Helder Costa]

Hello!
I'm estimating two probit models, the first doesn´t have an interaction term but the second does. I then take the marginal effect of the main variable of interest (this variable is interacted in the second model). My main goal is to check if the coefficients from both regressions are equal to each other, i.e., if they are the same, I´m considering that including the interaction term is not relevant.

After reading similar threads on Statalist and reading help files, I have attempted:

1. hasuman test: I have some concerns that the implementation of this command is not valid in my case. Also, the test applies to all variables, but only want it applied to the main coefficient of interest.
2. using the suest command: this yields a common error already discussed on Stalist "was estimated with a nonstandard vce (delta)"
3. saving the parameters as scalars and computing the pvalue manually: as far as I know, this works, but I would like the opinion of someone that has a better understanding of the problem.

Regarding the implementation of (1) and (3), the following example code shows what I´m doing.

   
est clear    
webuse margex, clear

probit outcome c.age c.distance
    margins, dydx(age distance) post
    est store m1
    scalar  me1 = _b[age]
    scalar se_me1 = _se[age]
    
probit outcome c.age##c.distance
    margins, dydx(age distance) post
    est store m2
    scalar  me2 = _b[age]
    scalar se_me2 = _se[age]

    hausman m1 m2, alleqs constant
    
    // "model fitted on these data fails to meet the asymptotic assumptions of the Hausman test; see suest for a generalized test."
    // on my data and regressions I don´t have this problem.
    
    test (me1 = me2)
    scalar se_diff_me = sqrt(se_me1^2 + se_me2^2)
    scalar test_stat = diff_me/se_diff_me
    scalar p_value = 2 * normal(-abs(test_stat))

    di "Test statistic: " test_stat
    di "P-value: " p_value

Would this manual implementation of the test be correct?

Thank you for your help.
Best,
Hélder

[Carlo Lazzaro]

Helder:
you can use -test- after -suest- to investigate the equality of coefficients:

. sysuse auto.dta
(1978 automobile data)

. probit foreign mpg trunk

Iteration 0:  Log likelihood =  -45.03321  
Iteration 1:  Log likelihood = -37.911178  
Iteration 2:  Log likelihood = -37.855949  
Iteration 3:  Log likelihood = -37.855923  
Iteration 4:  Log likelihood = -37.855923  

Probit regression                                       Number of obs =     74
                                                        LR chi2(2)    =  14.35
                                                        Prob > chi2   = 0.0008
Log likelihood = -37.855923                             Pseudo R2     = 0.1594

------------------------------------------------------------------------------
     foreign | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
         mpg |   .0642779   .0342888     1.87   0.061    -.0029269    .1314828
       trunk |  -.0827252   .0500744    -1.65   0.099    -.1808691    .0154188
       _cons |  -.8688439   1.225788    -0.71   0.478    -3.271345    1.533657
------------------------------------------------------------------------------

. estimates store A

. probit foreign c.mpg##c.trunk

Iteration 0:  Log likelihood =  -45.03321  
Iteration 1:  Log likelihood = -37.279225  
Iteration 2:  Log likelihood =   -36.8426  
Iteration 3:  Log likelihood = -36.838409  
Iteration 4:  Log likelihood = -36.838408  

Probit regression                                       Number of obs =     74
                                                        LR chi2(3)    =  16.39
                                                        Prob > chi2   = 0.0009
Log likelihood = -36.838408                             Pseudo R2     = 0.1820

-------------------------------------------------------------------------------
      foreign | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
--------------+----------------------------------------------------------------
          mpg |  -.1193748   .1369901    -0.87   0.384    -.3878705    .1491209
        trunk |  -.4069433    .247869    -1.64   0.101    -.8927575     .078871
              |
c.mpg#c.trunk |   .0148876   .0110341     1.35   0.177    -.0067387     .036514
              |
        _cons |   3.277949   3.264688     1.00   0.315    -3.120722    9.676619
-------------------------------------------------------------------------------

. estimates store B

. suest A B

Simultaneous results for A, B                               Number of obs = 74

-------------------------------------------------------------------------------
              |               Robust
              | Coefficient  std. err.      z    P>|z|     [95% conf. interval]
--------------+----------------------------------------------------------------
A_foreign     |
          mpg |   .0642779    .034186     1.88   0.060    -.0027255    .1312813
        trunk |  -.0827252   .0429758    -1.92   0.054    -.1669563    .0015059
        _cons |  -.8688439   1.198068    -0.73   0.468    -3.217015    1.479327
--------------+----------------------------------------------------------------
B_foreign     |
          mpg |  -.1193748   .1210122    -0.99   0.324    -.3565543    .1178047
        trunk |  -.4069433   .2006345    -2.03   0.043    -.8001797   -.0137068
              |
c.mpg#c.trunk |   .0148876   .0091054     1.64   0.102    -.0029586    .0327339
              |
        _cons |   3.277949   2.875403     1.14   0.254    -2.357738    8.913636
-------------------------------------------------------------------------------

. test [A_foreign]mpg=[B_foreign]mpg

 ( 1)  [A_foreign]mpg - [B_foreign]mpg = 0

           chi2(  1) =    2.58
         Prob > chi2 =    0.1083

. test [A_foreign]trunk=[B_foreign]trunk

 ( 1)  [A_foreign]trunk - [B_foreign]trunk = 0

           chi2(  1) =    2.61
         Prob > chi2 =    0.1063

.

[Helder Costa]

Hi Carlo, thanks a lot for your reply!

My goal would be applying something under the spirit of your implementation, but for the marginal effects, as I´m interested in testing if the overall effect of the main variable of interest changes with the interaction (evaluated at a certain point, such as the mean). When trying to use -suest- the error message

was estimated with a nonstandard vce (delta)

appears, thus, I tried to implement the test manually.

Therefore, my ultimate goal is to test if the coefficients of the marginal effects are equal to each other. I can´t use the probit coefficients because the interaction is not considered. I appreciate any comments you and other members might have in regards to this.

Thanks once again.
Best,
Hélder

[Carlo Lazzaro]

Helder:
I think your proposed solution sounds good.

[Helder Costa]

Thanks a lot for your comments Carlo!

But now I convinced myself my implementation won´t work, as I the denominator doesn't account for the covariance between the two parameters, and -suest- would be the only way to have a covariance matrix to address this.

[Carlo Lazzaro]

Helder:
yes, I had the same suspect about the missing covariance (I thought that you were inadvertently left it unreported in your code).
That said, I think that, for what you're after, -suest- is the best approach (admittedly, I would not consider -margins- as a viable tool for that).

[Helder Costa]

Thanks once again Carlo!
Indeed, I´m now considering the following, as probably the best solution possible:

1. estimate the probit coefficients, use -suest- to combine them, and then test the equality of the parameters.
2. after having the result of the test, compute the marginal effects, and use the test of the parameters as a reference to what the test of the marginal effects would be. this won't be 100% accurate, as in the simple case of testing if a parameter is different from zero, the hypothesis tests of the parameters don´t rule out the need to test the for the marginal effects, and vice-versa.

est clear    
webuse margex, clear

probit outcome c.age c.distance
    est store p1
    
probit outcome c.age##c.distance
    est store p2
    
suest p1 p2

test [p1_outcome]age = [p2_outcome]age + [p2_outcome]age#c.distance
    

est restore p1
margins,dydx(age) post
    est store m1
    
est restore p2
margins,dydx(age) post
    est store m2
    
esttab m1 m2

Best,
Hélder

[Carlo Lazzaro]

Hélder:
I still think that in your example, both -probit- and -suest- proves that interaction is not informative.
However, I find your approach interesting.

[Helder Costa]

Yes, I agree. Thanks a lot for all your input Carlo!