Calculations using b and SE from separate models

[Owen Gallupe]

Hi everyone,
I am trying to figure out a way to program the calculation of the z test comparing regression coefficients across separate subgroups outlined in:
Paternoster, R., Brame, R., Mazerolle, P., & Piquero, A. (1998). Using the correct statistical test for the equality of regression coefficients. Criminology, 36(4), 859-866.

The equation is attached. All it requires is the b and SE from each model.

z = (b from model 1) - (b from model 2)/(sqrt(SE of b from model 1 squared) + (SE of b from model 2 squared))

For example, if I wanted to compare whether the relationship between mpg and price (controlling for headroom) was the same for foreign and domestic cars*, I would run the following two models:

sysuse auto
regress price mpg headroom if foreign==0
regress price mpg headroom if foreign==1

In model 1, _b[mpg] = -374.1 and _se[mpg] = 91.6.
In model 2, _b[mpg] = -252.3 and _se[mpg] = 69.6.

Manually calculating the z test would be simple, but I'm curious about how to automate the process. I suspect that using -estimates store- is required, but I haven't managed to make it work (I don't have much programming experience).

Thanks!

Owen Gallupe

*I realise this is best done through a multiplicative interaction term. My goal is to compare the two approaches (the multiplicative approach and the subgroup correlations approach).

[Rich Goldstein]

what you are doing is not entirely clear to me, but the following should help

after estimating the first model, typing "ereturn li" will show you the coefficients and their SE's (as will "mat li r(table)"; from either of those sources save each item you want into a local (or a scalar); e.g., local coef1=_b[coef}" (you didn't give the names of your variables so I made things up); then do the same for the second model; you can then use the 4 locals in your equation

[Maarten Buis]

What is missing from your formula is the covariance between the two estimates, so that cannot be correct. You could try suest or use interaction terms.

[daniel klein]

I am sure you know this very well, but others might wonder. The two approaches are not, in general, equivalent.* The interaction model (i.e. the mode with a multiplicative term) allows the coefficient of mpg to vary over the groups of foreign. The coefficient for headroom, however, is constrained to be the same in both groups. With two models, one for each group of foreign, you allow all the coefficients to vary between these groups. Thus you would not expect the two approaches to give the same answer.

Also have a look at suest. (Maarten has already pointed that one out. Sometimes the forum is still remarkably slow).

Best
Daniel

* One notable exception is a fully saturated model. In the example given, if we do not control for headroom, then the two approaches give identical results.

[Owen Gallupe]

Thank you for your quick responses! The equation as posted was a direct screen shot from a well cited article (google scholar shows 988 citations at the moment). The equation is correct, but whether it is optimal is another story. I agree with you, Maarten, that interaction terms are probably the best way to go in general. But my goal is to compare the two approaches.

Richard, I had some success with your suggestion. The following syntax brought up the same result as a manual calculation (lines of stars intended to separate the approaches taken):
**********************
sysuse auto
quietly regress price mpg headroom if foreign==0
local mpgb1=_b[mpg]
local mpgse1=_se[mpg]

quietly regress price mpg headroom if foreign==1
local mpgb2=_b[mpg]
local mpgse2=_se[mpg]

display (`mpgb1'-`mpgb2')/(sqrt((`mpgse1'^2) + (`mpgse2'^2)))
-1.0585103
**********************

Manual calculation:
display ((-374.1402)-(-252.3255))/(sqrt((91.63609^2) + (69.617^2)))
-1.0585103

I suspect that it can be simplified but the following doesn't work. My guess is that the issue has something to do with my use of quotes but I'm not sure. The error message I get is "mpgb1 not found".
**********************
local testvar mpg
quietly regress price `testvar' headroom if foreign==0
local `testvar'b1=_b[`testvar']
(((This is where the error message is: mpgb1 not found. r(111)))
local `testvar'se1=_se[`testvar']

quietly regress price `testvar' headroom if foreign==1
local `testvar'b2=_b[`testvar']
local `testvar'se2=_se[`testvar']

display (`testvar'b1-`testvar'b2)/(sqrt((`testvar'se1^2) + (`testvar'se2^2)))
**********************
Any suggestions on how to clean this up would be greatly appreciated!

Thanks.

Owen

[Owen Gallupe]

Emoticon was unintentional...sorry.

[Owen Gallupe]

Thank you, Daniel. That's an excellent point.

Despite the arguments against the subgroup coefficients approach (which I am in agreement with!), I am really just doing this for teaching purposes. The overall message is that multiplicative terms are a better approach.

At this stage, my problem is just about programming. I haven't been able to get the last solution in a previous message to work (message #5 of this thread). If anyone could diagnose what I've done wrong, I would really appreciate it.

Thanks.

Owen

[daniel klein]

The error occurs in the last line of your code

display (`testvar'b1-`testvar'b2)/(sqrt((`testvar'se1^2) + (`testvar'se2^2)))

Note that `testvar'b1 does evaluate to mpgb1. This is a name, so Stata looks for a variable (or scalar) with this name, does not find any, and returns an error message. The point is that you need to evaluate mpgb1 to reference its contents.

Here is how this looks like

display ///
    (``testvar'b1'-``testvar'b2') ///
    /(sqrt((``testvar'se1'^2) + (``testvar'se2'^2)))

An arguably better approach would be setting up temporary scalars, as these might be more accurate - but I am not sure whether this is true.

Best
Daniel

[Owen Gallupe]

That works beautifully! Thank you, Daniel.