Using -KHB- in mediation analysis & results don't match

[Emily Frank]

Hi everyone,

I am using the KHB command to conduct mediation analysis for a probit model. Previously, I used the more simple mediation analysis method (i.e. running the regression with and without the mediators).

As I run the KHB analysis now, I am finding that none of the differences between the reduced and full models are significant, even those which from my analysis I'm fairly sure should be significant (i.e. that length of unemployment spell affects welfare receipt, which is mediated by employment level and income). I'm finding the lack of significance of "Diff" in the results surprising.

Does anyone have suggestions for how I can troubleshoot this? Might there be something I'm missing here?

For context:
khb probit alg2h (independent variables) (mediators related to employment and income), concomitant(control variables) disentangle zstandard

alg2h is welfare receipt.

I'm also having trouble figuring out whether KHB allows the very simple command of conducting the analysis for one group within the sample (i.e. probit... if group==1).

Thank you in advance!

Best,
Emily

[Felix Bittmann]

The KHB method was invented to account for the rescaling issues that arise when outcomes are binary. Given these rather recent developments I would assume that KHB will produce the better estimates compared to a linear model, otherwise the entire process would be rather useless. What other mediation tests did you use in linear models, for example, bootstrapping the share mediated or the Sobel test? These can also serve as indicators.

Regarding the second question, you can restrict your sample before running the command like

preserve
keep if group == 1
khb...
restore

[Emily Frank]

Hi Felix,

Thank you for your response! I have used bootstrapping as well. If anything, it indicated that even more variables are significant than when I ran simple linear models with robust standard errors. The Sobel seems to be unavailable in Stata now, but if you've found it I'd be happy to know how to install it.

Thanks again!

Best,
Emily

[Felix Bittmann]

I am not aware of a command but there is a simple online tool. You need to run the regressions and then copy paste the relevant coefficients. For smaller models this works fine (for me). http://quantpsy.org/sobel/sobel.htm

[Wanying Ling]

I would like to share an example of how I used the KHB command today.

I started out based on stata16 and ran the code and it looked like this,

The mediating effect at this point is not significant. After that, I reinstalled the KHB command and ran the same code and data using stata17, and it looked like this,