how to compare two logistic regression models

[Rens de Visser]

Hi all,

After searching on the web myself, I could not find a good answer to this question. I think my question is more like a general statistics question rather than a STATA question, but I hope you guys can me help out again.

I have conducted several logistic regression analyses with odds ratios as outcome. All analyses (four) have the same dichotomous outcome variable and the same independent variables. The thing that is not in common is the sample from which they are drawn (i.e. each of the four analyses is drawn from a different population).

Now, my question is, if and how do I compare the outcomes on the independent variables in logistic regression A with the outcomes on the independent variables in logistic regression B?

For example, in regression A the OR for men to have the exposed is 1.245 and in regression B (another patient population) the OR for men to have the exposed is 1.418. I cannot just say that men in B are more likely to have the exposed than men in A, right?

Hope you guys can provide some answers.

With regards,
Rens

[Jorrit Gosens]

You could look to see if confidence intervals for the coefficients from different regressions overlap or not.
If everything else is equal in these models though, I would rather consider pooling the data, and running the regression with dummies that indicate what sample the observations come from. Significant effects would indicate if one group differs form the average of the other groups.

[Rens de Visser]

Thanks for the reply Jorrit.

Do you mean that if confidence intervals overlap, that they are not different? And does overlapping in this case mean that even overlap even for a small part, that this is enough (e.g. A: 2.365 - 3.250 and B: 1.910 - 2.477) to say they overlap and thus are not different?

And, could you elaborate a bit more on your second option on pooling the data, and running the regression with dummies. Yes, both the dependent variable and independent variables in all these models are set up equal. The only thing that differs is actually is that it is panel data, so respondents have been interviewed on different moments in time.

[Stephen Jenkins]

I advise you to read the closely-related thread at http://www.statalist.org/forums/foru...s-n-and-m-size

[Rens de Visser]

Thanks, I'll read it closely.

I also found the command:

logistic ltcuse age_y male registeredormarried i.hospitalization2 i.education i.jobsituation borninothercountry income assets livingincity havingkids i.perceivedhealth adl iadl mobility chronic2ormore eurodcat if t == 2004
est store t2004

logistic ltcuse age_y male registeredormarried i.hospitalization2 i.education i.jobsituation borninothercountry income assets livingincity havingkids i.perceivedhealth adl iadl mobility chronic2ormore eurodcat if t == 2006
est store t2006

suest t2004 t2006
test [t2004_ltcuse]registeredormarried=[t2006_ltcuse]registeredormarried

which I believe tests whether the regression coefficients from the registeredormarried variable in 2004 differs from the one in 2006. Is that right?

And am I correct if I say p > 0.05 they do not differ and p < 0.05 they do differ? Or is it the other way around? Or am I completely wrong?

Cheers.

[Stephen Jenkins]

I would recommend that you read the literature cited before seeking definitive answers of the type that you are requesting.

[Rens de Visser]

I read some of them, but since my English and statistical knowledge isn't that developed I hoped you could give me an answer to my question

[Maarten Buis]

The answer is that there is no easy answer. It is still an area of active research and an easy well estabilished answer is yet to appear.

[Rens de Visser]

I read your publication on this as well Maarten, on your webpage. Still I'm trying to figure out how to compare odds ratios from two regression models that have equal IV's and DV's but different sample..

[Maarten Buis]

I would argue that you can, others disagree... Since there is no established conclusion yet, all you can do is understand the arguments out there and make up your own mind.

[Rens de Visser]

You mean you can by using the suest command?

[Maarten Buis]

Yes, though I would use interaction effects.

But beware, the other position that you cannot compare across groups is not without merrit. So I would first understand what this debate is about and only than look at ways of implementing your prefered choice.

[Rens de Visser]

Ok, thanks.

What interaction effects would you be looking after then? For instance with those variables that have been omitted in doing the fixed effects method?

To be clear, I have panel data and I would like to compare OR's from 2004 with those from 2006 (or 2010, or 2013 as I have data of 4 time lapses). I wanted to do this using the suest command. The IV's and DV's are set up the same in all years, only the sample differs (not all participants from 2004 are present in 2006 for instance, some new respondents are added).

[Rens de Visser]

Maarten:

I would like to create interaction effects between each variable and time variables of 2004 and 2013 to see if these effects significantly differ across both years. How do I handle this? I mean, sure, I can create interactions terms by simply multiplying them, but then, how do I compare them? Note: I read your article on http://www.stata-journal.com/sjpdf.h...iclenum=st0194

[Jorrit Gosens]

I actually see here that my post #2 wasn't entirely useful.
Although the current suggestion for interaction effects make sense mathematically, I would suggest you do come up with a hypothesis for such effects first. I've no idea what question it is exactly, but it's some medical data, so just as an example lets say nr of heart conditions is the DV and smoking habits the IV. Why would smoking in yr x have a different effect than it does in year y? Is it because you think the sample of people is different? That group A, surveyed in yr x, does more exercise than group B, interviewed in yr Y?
If you have (proxy) data on such habits, include them. In that case interaction effects also make sense: maybe doing exercise inst helpful for health when smoking, and an interaction term should show that.
Practically, if you do have a hypothesis and want to create interaction terms for years, and seen you have only two years, by creating a dummy for 'yearis2013'. Again, pool your data, and run one regression on all your data. If e.g., Var1 is significant and positive and Var1*Yearis2013 is too, than you have shown that the effect of Var1 on your DV has increased between 2006 and 2013.