Mediation analysis and standard errors/CIs with -khb-

[David Quinn]

Hello,

I would like to run a mediation analysis with a logit model using a binary independent variable, a binary mediator, and a binary dependent variable.

I have been searching for ways to do it in Stata and have come across several possible options. The most promising option is the user-written program -khb-, created by Ulrich Kohler, Kristian Bernt Karlson, and Anders Holm, and detailed in the following article:

Kohler, U., K.B. Karlson, and A. Holm. 2011. "Comparing Coefficients of Nested Nonlinear Probability Models." Stata Journal, 11(3): 420-38.

My biggest question at this time relates to the way in which the standard errors and confidence intervals are calculated using -khb-. Kohler, Karlson, and Holm indicate that their program can be used with various types of IVs, mediators, and DVs, including dichotomous ones. But they indicate that the standard errors are calculated using Sobel's (1982) method, which assumes a normal distribution. Since dichotomous variables cannot be assumed to be normally distributed, Hayes (2009) and Preacher and Hayes (2004) argue that Sobel's method cannot be used and that instead you need to bootstrap the standard errors when working with dichotomous variables.

But I notice that the -vce- option in -khb- does not allow use of the command -bootstrap-, but rather only -cluster- and -robust-.

Have there been any further changes made to -khb- of which I am unaware that allow the use of bootstrapping standard errors? Or am I misunderstanding the capabilities of the -khb- command with regard to use of the bootstrap? If not, my guess is that I may have to look to other options for running my desired mediation analysis.

Thanks in advance for any insight,
--David

[Phil Schumm]

For mediation analyses with nonlinear models, look at the approach of Valeri and VanderWeele (2013) based on a counterfactual framework (and building on the work of others). These analyses may be performed in Stata using the paramed package (type search paramed), including obtaining bootstrap bias-corrected confidence intervals.

[David Quinn]

For mediation analyses with nonlinear models, look at the approach of Valeri and VanderWeele (2013) based on a counterfactual framework (and building on the work of others). These analyses may be performed in Stata using the paramed package (type search paramed), including obtaining bootstrap bias-corrected confidence intervals.

Thanks, Phil. I will have a look at the -paramed- package in more depth. Do you know if there is an article detailing its rationale, construction, process, and examples in more depth? I can't seem to find one. Something that goes in more depth than the associated help file.

[Rich Goldstein]

see the "Further Reading" section of the -paramed- help file

[Phil Schumm]

Rich is right; those references (one of which I cited above) are a good place to start, together with the other references cited therein.

[David Quinn]

Hi everyone,

So, I've done some additional reading on -paramed,- some of the sources cited in the reference section of the -paramed- help file, and some other sources (some of which were referenced therein).

I have determined that I need to do sensitivity analysis along with the mediation analysis because there are some potential unobserved confounding variables. I am less clear about whether or not I also need to run a log-link (aka log-linear) model, which Valeri and VanderWeele (2013) argue needs to be done if the outcome is "not relatively rare."

This leads me to ask several follow-up questions:

1.) Are there any guidelines as to how "relatively rare" Y=1 must be in order to not necessitate the use of a log-link (aka log-linear) model? For the two outcomes that I will be analyzing, Y=1 happens in 34% and 26%, respectively, of the cases in my sample. Thus, Y=1 is always less common than Y=0 for both of my outcome variables, but I'm not sure how "rare" they must be in order to not necessitate a log-link model. Those percentages don't strike me as particularly "rare."

2.) Regarding sensitivity analysis, I can't seem to find any Stata commands that execute such an analysis for the -paramed- package. Has someone written a set of sensitivity analysis commands to use with -paramed- that I have missed, or is there another set of substitutable sensitivity analysis commands that will work in conjunction with -paramed-?

3.) I've looked into another Stata package, -medeff- (Imai, Keele, and Tingley 2010), as an alternative to -paramed-. -medeff- allows dichotomous outcomes and mediators, and it contains a sensitivity analysis command, -medsens-. However, it appears that -medsens- is only applicable in cases where there are unobserved variables that confound the relationship between the mediator (M) and the outcome (Y). In my analysis, the unobserved confounding variables confound the relationship between the treatment (X) and the outcome, not the relationship between the mediator and the outcome. Am I correct in my conclusion that -medsens- does not allow for confounders of the treatment-outcome relationship? If not, can anyone point me to a source or set of commands within -medsens-/-medeff- that will allow for this type of confounding? If I am correct, are there a set of substitutable commands that allow for confounding of the treatment-outcome relationship and that will work in conjunction with -medsens-/-medeff-?

Thanks in advance for any insight that you have!
--David

[Maarten Buis]

As a matter of terminology: log-linear has (at least) two different meanings depending on the discipline you are from. In my discipline (sociology) and in more medical/biological oriented disciplines it means you have only categorical explanatory/independent/right-hand-side/x-variables, you have collapsed the data such that each row in your dataset represents a distinct combination of explanatory and explained variables, plus you have an additional variable representing the number of observations who have that combination. You model that count with a log-link (and Poisson family), hence the term log-linear. You get exactly the same results as in logistic regression. A useful analogy is to think of log-linear modeling as the categorical equivalent of ANOVA and logistic regression as the categorical equivalent to linear regression. Some work has been done on modeling intervening variables in log-linear models, so I suspect that that is what is meant. However, this type of modeling is not natively supported by Stata. You can "trick" Stata into estimating these models (and I gave some hints on how you could do it) but you really need to know what you are doing in order to make sure that you used the "tricks" correctly. So my recommendation would be: when in doubt, don't do it.

You can also use the log link for binary dependent variables. Based on what you have written, I suspect that this is what you mean with log-linear. However this is a very unusual use of the term, so I recommend against using the term that way. This model is used to estimate risk ratios (instead of odds ratios), which can work as long as the dependent variable is pretty rare. The log link function does not force the predicted probability to remain less than 1, so if in your data the risk is fairly high, you can easily end up with predicted probabilities larger than 1.

In economics the term log-linear is often used for a continuous dependent variable that has been log transformed (so no link function) and than used in a linear regression. This is not appropriate for your problem (if ever, see: www.blog.stata.com/2011/08/22/).

[David Quinn]

As a matter of terminology: log-linear has (at least) two different meanings depending on the discipline you are from. In my discipline (sociology) and in more medical/biological oriented disciplines it means you have only categorical explanatory/independent/right-hand-side/x-variables, you have collapsed the data such that each row in your dataset represents a distinct combination of explanatory and explained variables, plus you have an additional variable representing the number of observations who have that combination. You model that count with a log-link (and Poisson family), hence the term log-linear. You get exactly the same results as in logistic regression. A useful analogy is to think of log-linear modeling as the categorical equivalent of ANOVA and logistic regression as the categorical equivalent to linear regression. Some work has been done on modeling intervening variables in log-linear models, so I suspect that that is what is meant. However, this type of modeling is not natively supported by Stata. You can "trick" Stata into estimating these models (and I gave some hints on how you could do it) but you really need to know what you are doing in order to make sure that you used the "tricks" correctly. So my recommendation would be: when in doubt, don't do it.

You can also use the log link for binary dependent variables. Based on what you have written, I suspect that this is what you mean with log-linear. However this is a very unusual use of the term, so I recommend against using the term that way. This model is used to estimate risk ratios (instead of odds ratios), which can work as long as the dependent variable is pretty rare. The log link function does not force the predicted probability to remain less than 1, so if in your data the risk is fairly high, you can easily end up with predicted probabilities larger than 1.

In economics the term log-linear is often used for a continuous dependent variable that has been log transformed (so no link function) and than used in a linear regression. This is not appropriate for your problem (if ever, see: www.blog.stata.com/2011/08/22/).

Hi Maarten,

Sorry for the confusion on my use of the terms log-linear and log-link. And thank you much for the analysis.

As you suspected, I was using them in the second sense that you described. I know what you are referring to with regard to the first sense that you described, particularly regarding count models and Poisson.

But in an article on mediation analysis written by the authors of the -paramed- command (the article is actually about the mediation analysis macros that they created for SAS and SPSS, and they have a forthcoming one on -paramed- in Stata specifically), they indicate that when you have a binary mediator (M) and a binary outcome (Y), or when you have interaction between an explanatory variable (X) and a mediator (M) in a model with a binary outcome (Y), then it is the case that:

"When the outcome is not rare, the odds ratio does not approximate the risk ratio anymore. Therefore, the causal effects previously defined will be biased if logistic regression is used to model the outcome. In this case the investigator can estimate the causal effect by running a generalized linear model regression with a binomial distribution and a log link and the causal effects will have a risk ratio interpretation and the formulas hold exactly" (Valeri and VanderWeele 2013, 140-41). They indicate that when the outcome is common, "the odds ratios with the mediator in the model versus without the mediator in the model are thus not directly comparable, and so the difference method essentially breaks down. The risk ratio does not suffer this problem, and it is for this reason that we propose using a log-linear model...when the outcome is common" (142).

In essence, they are stating the opposite of what you are, that logistic regression can be used when the outcome is rare, but that log-link/log-linear should be used when the outcome is common. Perhaps this is only relevant to their particular formulas and macros for mediation analysis.

Hence, that is why I raised the question about how common (or not rare) an outcome must be in order to necessitate using a log-link/log-linear model for mediation analysis when you use the -paramed- macro.

Thanks,
--David

Reference: Valeri, L., and T.J. VanderWeele. 2013. "Mediation Analysis Allowing for Exposure–Mediator Interactions and Causal Interpretation: Theoretical Assumptions and Implementation With SAS and SPSS Macros." Psychological Methods, 18(2): 137-50.

[Maarten Buis]

I have a big problem with the that way of using logistic regression. If you want risk ratios, you should use a model that gives you risk ratios. It is as simple as that. So, if you want risk ratios you should never use logistic regression, because it won't give you what you want. The fact that under special circumstatnce an odds ratio can be treated as an approximation of a risk ratio is no excuse, as espcially in those cases models that output risk ratios are well behaved.

So your question can be easily answered: you should either always use the log-link (if you want risk ratios) or never (if you want odds ratios). The authors you cite are just wrong when they state that it depends on how common successes are, instead it depends on what you exactly want to get out of your model.

[Emmanuel W]

hello, I would like to run a bootstrap with KHB too.

I found a solution for calculate the IC of reduction percent (Conf_Pct) with a little program but I don't know if my results are correct (IC and biais seem to be very large...) But if you want run a bootstrap with KHB, don't try with -vce- option but with:

bootstrap XXX, reps(XX): khb regress XXX XXX || XXX

[David Quinn]

Thanks to Maarten and Emmanuel for their recent responses.

Maarten: I definitely see your point on this issue of when to use log-link. I was frankly a bit surprised to see Valeri and VanderWheele discuss the issue as they did, as my prior familiarity with the subject led me to the understanding that the choice between using log-link and logit is one driven by preference for a particular type of measure of association (risk or odds ratios). Upon reading their article, I thought that perhaps there was a new wrinkle that I had not yet considered.

Emmanuel: Thanks for the suggestion on leading with the bootstrap command when using -khb-. So, have you figured out whether or not the results you are getting are correct? You seemed skeptical.

I'm not sure that I can use -khb- when I have unobserved confounders, though. I could control for such confounders by running a biprobit model, but it doesn't appear that -khb- allows for the use of biprobit.

For what I require, -paramed- and -medeff- still look like they may provide the most potential utility, but as I indicated above there are issues or questions related to conducting sensitivity analysis with each of them that I have not yet been able to resolve or answer.

Thanks,
--David