Interpretation of odds ratios of interaction terms. Testing for effect modification

[Elana Olivier]

Hi
I am a new member, an older learner/student and busy with my Master's degree in Public Health. I am busy with my dissertation, "Willingness to initiate insulin in people living with type 2 diabetes Investigating the role of diabetes-related distress"

I, however, need advice in interpreting odds ratios of interaction terms, the ratio of odds ratios. I have read several answers from different post on the subject including Clyde Schechter https://www.statalist.org/forums/for...e-interactions
I struggle to keep to a standard way of reporting each logistic regression model and compare with the different models. I have done and understand marginal effects

For the purpose of this post, I choose the following a priori variables and 3 different types of models

Willingness2 = Binary variable - Outcome
Age = continuous - Predictor of interest 1
Sex = binary variable - Predictor of interest 2
Diabetes distress = DDSmeanitem (continuous) OR DDSmeanitem_cat (categorical) - possible effect modifier

1.Logistic regression with Interaction terms: Variable AGE (continuous) and DDS mean item as a CONTINUOUS variable

. logistic willingness2 c.age##c.DDSmeanitem,nolog

Logistic regression Number of obs = 117
LR chi2(3) = 4.81
Prob > chi2 = 0.1860
Log likelihood = -77.968229 Pseudo R2 = 0.0299


willingness2 Odds Ratio Std. Err. z P>z [95% Conf. Interval]

age .9091612 .0460478 -1.88 0.060 .8232442 1.004045
DDSmeanitem .030789 .0570083 -1.88 0.060 .0008172 1.160014

c.age#c.DDSmeanitem 1.066365 .03491 1.96 0.050 1.000092 1.13703

_cons 146.9265 432.4104 1.70 0.090 .4591778 47013.15

Note: _cons estimates baseline odds.


.Using the odds ratios of the interaction term and diabetes distress, the additional effect that diabetes distress has on the relationship with willingness is 1.066 + 0.030 = 1.096 > 1
Therefore, the interaction effect of diabetes distress is “small positive”. (p-value could probably be interpreted as statistically significant, p=0.05)
Therefore, diabetes distress as a continuous variable has a “small positive” effect of modifying age in relation to willingness.
1.Conclusion: Effect modification is present with the variable, diabetes distress

2.Logistic regression with variables SEX (categorical) and c.DDSmeanitem (continuous)

logistic willingness2 i.sex##c.DDSmeanitem,nolog

Logistic regression Number of obs = 117
LR chi2(3) = 1.81
Prob > chi2 = 0.6131
Log likelihood = -79.470387 Pseudo R2 = 0.0112


willingness2 Odds Ratio Std. Err. z P>z [95% Conf. Interval]

sex
Male .3031514 .2988301 -1.21 0.226 .0439128 2.0928
DDSmeanitem .7914705 .3192276 -0.58 0.562 .3590175 1.744833

sex#c.DDSmeanitem
Male 1.961714 1.090289 1.21 0.225 .6600115 5.830688

_cons 1.178385 .7846311 0.25 0.805 .3195301 4.345726

Note: _cons estimates baseline odds.

The researcher investigated if diabetes distress (continuous) acts as a possible effect modifier with the predictor of interest, sex (categorical) in relation with willingness to initiate insulin
Sex was therefore used as a binary variable and the following results are presented.
Using the odds ratios of the interaction term and diabetes distress, the additional effect that diabetes distress has on the relationship with willingness is 1.96 + 0.79= 2.75 > 1
Therefore, the interaction effect of diabetes distress have on the predictor of interest, sex, is extremely large positive????
2. Conclusion: Diabetes distress is an effect modifier?

3.Logistic regression with variable SEX (categorical) and c.DDSmeanitem_cat (categorical)

. logistic willingness2 i.sex##c.DDSmeanitem_cat ,nolog

Logistic regression Number of obs = 117
LR chi2(3) = 0.87
Prob > chi2 = 0.8322
Log likelihood = -79.938455 Pseudo R2 = 0.0054

willingness2 Odds Ratio Std. Err. z P>z [95% Conf. Interval]

sex
Male .4235411 .4035914 -0.90 0.367 .0654316 2.741597
DDSmeanitem_cat .7818847 .3587579 -0.54 0.592 .3181117 1.921789

sex#c.DDSmeanitem_cat
Male 1.854332 1.264619 0.91 0.365 .4871743 7.058147

_cons 1.107827 .6712464 0.17 0.866 .3378457 3.632668


.
The study determined the stratified odds ratios for the interaction term of diabetes distress as a categorical variable in association with sex (categorical variable) .
The adjusted odds ratio for diabetes distress in association with sex (males);
Category 2 (moderate distress) vs 1 (no distress) is 0.96
Category 3 (high distress) vs 1 (no distress) is 7.23
Category for 3 (high distress) vs 2 (moderate distress) is 7.23/0.96 = 7.53
Therefore, diabetes distress has a small (0.96) to very large (7.53) positive effect of modifying sex (males) in relation to willingness (but p-values non-significant?)
3.Conclusion: Diabetes distress acts as an effect modifier

Am I interpreting the odds ratios in these 3 models the correct way?

Thank you
Elana

[Clyde Schechter]

.Using the odds ratios of the interaction term and diabetes distress, the additional effect that diabetes distress has on the relationship with willingness is 1.066 + 0.030 = 1.096 > 1

This is incorrect on a couple of levels.

First, you ran -logistic-, not -logit-, so your outputs are not regression coefficients, they are odds ratios. When you combine odds ratios, you have to multiply them, not add them.

Next, your diabetes distress variable is continuous, as is age. And in the interaction model, the effect of diabetes distress will depend on age.. Consequently the incremental effect of a unit increase in diabetes distress is actually to multiply the odds of willingness2 by a factor of 0.030789 * (1.066365^age ). Whether this factor will be greater or less than 1 will depend on the value of age, crossing that 1 threshold at around 54.

I have not carefully read the rest of your post: after reconsidering in light of the above, if you have additional questions, do post back.

[Elana Olivier]

Thank you Clyde! I will do as you proposed

Keep safe!

[Elana Olivier]

Dear Clyde

I have corrected the odds ratios with the continuous*continuous interaction term. Thank you so much!

The following is an addition to the previous example. It is the odds ratios of a binary(categorical)*continuous interaction term...to determine if diabetes distress is an effect modifier in relationship with willingness to initiate insulin

Odds ratios and ratio of odds ratios
In this section, the researcher investigated if diabetes distress (continuous) acts as a possible effect modifier with the predictor of interest, sex (categorical) in relation with willingness to initiate insulin
Sex was therefore used as a binary variable and the following results are presented. (Table below…

	OR	p-value	95% CI
Sex(male)	0.3	0.23	0.04 to 2.09
DDS (diabetes distress)	0.7914705	0.56	1.36 to 1.74
Sex(male)*diabetes distress	1.961714	0.23	0.66 to 5.80

Using the odds ratios of the interaction term and diabetes distress, the additional effect that diabetes distress has on the relationship with willingness is 0.7914705*(1.961714²) = 3.045 > 1
Females and males are coded 1/2

Three questions:
1. The interaction effect of diabetes distress has on the predictor of interest, sex (males), is extremely large positive????
2. Can I interpreted the OR of diabetes distress of females as 0.7914705*1.961714 = 1.5526 >1
3. How do I quantify the Odds Ratio of 3.45?. Are there any articles you can refer me to?

I really am appreciating your efforts to guide me!

Regards
Elana

[Clyde Schechter]

The coding of sex as 1/2 instead of 0/1 makes it more complicated. But you do not square the 1.961714 coefficient to reflect a male, unless you want to compare males to non-existent people with a sex code of 0. It is the difference in value of sex that serves as the exponent. When you use the natural 0/1 coding, this is simpler.

When diabetes distress score = 0, the male:female odds ratio is 0.3. When diabetes distress score = X, then the male: female odds ratio is:

0.3*(0.79147050^X)*(1.961714^X)

So the 3.045 odds ratio you show is incorrect for any value of diabetes distress score because you do not incorporate it into the calculation in any consistent way..

For a diabetes distress score of 1, this male:female odds ratio calculates to 0.466. But as the diabetes distress goes up, the male female OR goes up rapidly. If you are using the same diabetes distress score I am familiar with, a score of 5 would not be uncommon, and for that the male:female odds ratio is 0.30*((0.79147050*1.961714)^5), which is about 2.71.

[Elana Olivier]

Dear Clyde

Thank you for your patience and clear answer. I've changed the coding of sex (just to understand the principle!) and got basically the same answers

Finally, just to answer my dissertation's research question; if sex is an effect modifier? How do I understand these results in terms of effect modification?

I have read the article "How Big is a Big Odds Ratio? Interpreting the Magnitudes of Odds Ratios in Epidemiological Studies" by Chen et al, 2010,
If I postulate that disease rate in the nonexposed group are 1%, the above ORs indicate a "weak association"?

Thank you for the amazing work you are doing!

Kind regards
Elana

[Clyde Schechter]

Well, I think that's an ill-posed question. In the real world, it is a rare situation that any variable fails to be an effect modifier of any other. So, absent unusual conditions, the answer is yes in a trivial sense. A better way to think about it is: is the effect modification by sex large enough to matter for practical purposes. Some people would answer that question by looking at the p-value of the interaction term in the logistic regression output and comparing it to a magic number like 0.05. I'm not a fan of that approach, because depending on sample size and data noisiness, a trivial effect of no real world consequence can be statistically significant, or a large effect that one should not want to neglect might fail to be.

So I like a different approach. Since one of the variables involved is continuous, it's a bit harder. But what I do is pick a range of important values of the continuous variable, diabetes distress score in your case. This would typically not be the full range of possible values, but only the range of values that occur with appreciable frequency--leave out the extreme values that show up only rarely. Evaluate the sex OR at both the upper and lower bounds of that range. If the sex OR's are materially different with those values of diabetes distress, then I would say you have a materially important interaction. If not, you don't.

That said, if your dissertation advisor tells you to do it by p-values, while I would say that he or she is misguiding you, you need to assess how accepting he or she is to pushback from a student. If you think you can persuade your advisor, you might refer him/her to the American Statistical Association's recommendation that statistical significance as a concept be abandoned. See https://www.tandfonline.com/doi/full...5.2019.1583913 for the "executive summary" and
https://www.tandfonline.com/toc/utas20/73/sup1 for all 43 supporting articles. Or https://www.nature.com/articles/d41586-019-00857-9 for the tl;dr.. In the end, though, if you are told to do it, don't make this a hill to die on. Just tuck it away in your mind as something that you can do better when you're in a more senior stage in your career.

[Elana Olivier]

Dear Clyde

Thank you again for your thorough explanation. I determined the differences of values between the ORs of sex and the ORs of diabetes distress if I predetermine values of diabetes distress at a range of high frequencies. I've then found sex as an effect modifier.

Luckily I've got an excellent advisor and I have frequently quoted (Wasserstein et al., 2019) pertaining p-values!

Have a lovely day and be safe!
Kind regards
Elana

[Elana Olivier]

Dear Clyde

I do not seem to get any information on the web whether I can use marginal effects to determine effect modification: I have 3 scenarios to determine if the variables age and sex are responsible for effect modification using marginal effects

First of all. Can I use marginal effects for effect modification? (I suppose I'm too late with this question seeing that I've completed it!)
I've read NORTON, E. C., DOWD, B. E. & MACIEJEWSKI, M. L. 2019. Marginal Effects—Quantifying the Effect of Changes in Risk Factors in Logistic Regression Models. JAMA, 321, 1304-1305. but I think I've got all mixed up!

Scenario 1
.logistic willingness2 c.age##c.DDSmeanitem,nolog
.margins, dydx(age) at ( DDSmeanitem=(1(1)5)) vsquish
.marginsplot,ylin(0)




). If the diabetes distress has a constant value of 2, the predicted probability of willingness is minimal positive (0.007) If the diabetes distress has a constant value of 3,4 or 5, the predicted probability of willingness is small positive (0.018 to 0.03) Predicted probabilities are measured on a curve from 0 to 1, where 0.5 is relatively large positive.

Conclusion: Effect modification is present when the variable age is used with the predictor of interest, diabetes distress


Scenario 2
.logistic willingness2 i.sex##c.DDSmeanitem,nolog
.margins, dydx(sex) at (DDSmeanitem=(1(1)5)) vsquish
.marginsplot,ylin(0)





Conclusion: Effect modification is present when the variable sex is used with the predictor of interest, diabetes distress

Scenario 3
.logistic willingness2 i.sex##i.DDSmeanitem_cat ,nolog
.lincom 1.sex#1.DDSmeanitem_cat - 1.sex#2.DDSmeanitem_cat
.lincom 2.sex#1.DDSmeanitem_cat - 2.sex#2.DDSmeanitem_cat
.incom 2.sex#1.DDSmeanitem_cat - 2.sex#2.DDSmeanitem_cat
.lincom (1.sex#1.DDSmeanitem_cat - 1.sex#2.DDSmeanitem_cat)- (2.sex#1.DDSmeanitem_cat-2.sex#2.DDSmeanitem_cat)
.margins sex,at (DDSmeanitem_cat =(1(1)3)) vsquish
.marginsplot,ylin(0)



Conclusion: Effect modification is present when the variable sex is used with the predictor of interest, diabetes distress

Are these scenarios or ad hoc analysis in any way possible?
Always appreciating your comments

Kind regards
Elana