Help KHB Interpretation: Large Percentage of Total Effect Explained

[Jason Smith]

Hello, I am using the khb command to perform a decompostion. It is a user-written command that is discussed in detail here:

Kohler, U., Karlson, K. B., & Holm, A. (2011). Comparing coefficients of nested nonlinear probability models. The Stata Journal, 11(3), 420-438.

I ran the following code:

khb reg y x || var1 var2 var3, vce(cluster id)  c($controls) disentangle notable

The results look as follows:

Z-variable	Coef	Std_Err	P_Diff	P_Reduced
X
var1	0.0199	0.0067	35.95	923.42
var2	0.0256	0.0051	46.45	1193.11
var3	0.0097	0.0042	17.60	452.19

My question is about the P_Reduced column, which represents the proportion of the total effect of X explained by each mediator (i.e., var1, var2, and var3). What does it mean when the proportion of the total effect explained exceeds 100? How does one explain these numbers to readers?

Ulrich Kohler, would you be able to provide some insight? I would greatly appreciate it!

[George Ford]

The results will present "Conf_Pct R" under "Summary of Confounding".

The sum of P_reduced should equal that. (pp. 431-2 in Karlson et al).

[Jason Smith]

Hi George Ford, thanks for the reply! I am still wondering what it would mean to have an overall confounding percentage of 2568.72? How can the percentage of the total effect explained exceed 100?

[Ulrich Kohler]

A confounding percentage of 2568.72 means that the coefficient of the full model has changed by 2569 percent in comparison to the coefficient of the reduced model. The confounding percentage is calculated as "coefficient of the reduced" minus "coefficient of the full model" divided by the "coefficient of the full model" times 100. If the coefficient of the full model is more than double the size of the coefficient of the reduced model, we get percentages below -100. If the coefficient also changes in sign, we get percentages above 100. It is possible and even not unusual that coefficients get larger or change in sign, once controlling for other covariates, but it depends on the selection of the covariates, whether the results deserve to be interpreted. As I wrote somewhere else: "The selection of control variables used in a statistical model is the single most important research design decision in an observational study." (https://doi.org/10.1093/esr/jcac078) If 2568% is the result of a real world analysis, I would distrust the results. Things like that often happen if you have multicollinearity in a small dataset. You may want to do something on the multicollinearity or collect more data before drawing far-reaching conclusions.