Dominance analysis (domin) after xthybrid

[Felicia Hollingsworth]

Dear all,

I'm using the user-written command xthybrid (see Schunck & Perales, 2017, full reference below) from SSC in my research (using Stata 15.1) and would like to apply dominance analyses (preferably) using the user-written command domin (see Luchmann, 2021) from SSC to analyse the relative importance of factors. Does anyone happen to know how to conduct dominance analyses after xthybrid?

xthybrid does not return an R², so I tried using domin with the log-likelihood (ll) and the chi2 as fit metrics, typing in:

domin dep_var indep_var1 indep_var2 indep_var3, reg(xthybrid) fitstat(e(ll))
domin dep_var indep_var1 indep_var2 indep_var3, reg(xthybrid) fitstat(e(chi2))

Both return the following:

xthybrid resulted in an error.
r(198);

- to me that sounds like domin cannot be used with xthybrid as the underlying estimation method (although xthybrid follows the depvar indepvars format). Can anyone confirm that or am I missing something?

What can I do instead? Would the mixdom-wrapper (which produces a within-ID model fit metric (r2_w) and a between-ID model fit metric (r2_b)) be a good choice, since hybrid models essentially are mixed models?

domin dep_var indep_var1 indep_var2 indep_var3, reg(mixdom, id(ID_var)) fitstat(e(r2_w))
domin dep_var indep_var1 indep_var2 indep_var3, reg(mixdom, id(ID_var)) fitstat(e(r2_b))

Those commands give me back results - I'm just not sure, whether using the mixdom wrapper is the right choice. Does anyone have any thoughts on that?

If dominance analysis cannot be conducted using xthybrid, another idea would be to additionally estimate a fixed-effects model (using xtreg, fe) and a between-effects model (using xtreg, be) and run dominancye analyses (using domin) on those models.The coefficients of the fixed-effects model are identical to those of the within-component of the hybrid model whereas the coefficients of the between-effects model are nearly identical to the between-component of the hybrid model (taking into account the bias due to unobserved heterogeneity), so maybe that procedure could serve as an approximation of the relative importance. Again, my question is: Does anyone have any thoughts on that? Or is this irrelevant because there's another way of conducting dominance analysis after xthybrid?

Thanks a lot in advance!
Felicia



References mentioned above:
Schunck, R. & Perales, F. (2017). Within- and Between-cluster Effects in Generalized Linear Mixed Models: A Discussion of Approaches and the Xthybrid command. The Stata Journal: Promoting communications on statistics and Stata, 17(1), 89–115.
Luchman, J. N. (2021). Determining relative importance in Stata using dominance analysis: domin and domme. The Stata Journal: Promoting communications on statistics and Stata, 21(2), 510–538.

[daniel klein]

From the help file of xthybrid (SSC), it appears as if the clusterid() option is required. This option appears to be absent in the code reported in #1.

I have never dealt with dominance analyses, and rarely looked into xthybrid, so I cannot comment much substantively. However, I wonder, both substantively and technically, how dominance analyses works in a situation in which some of the variables are decomposed into their within and between units variations.

[Felicia Hollingsworth]

You're right - clusterid() was missing from the first two lines of code in my initial question, thanks for pointing that out!

Domin (using xthybrid as the underlying estimation command, with, for example, the log-likelihood) gives back an overall result of the dominance analysis for the hybrid model (no differentiation between within-variation and between-variation) - so here I'm wondering, too, what this overall result means or how it's calculated given that the variables are decomposed in the process, just like you said.

What I was hoping for, is that dominance analysis allows me to identify the relative importance of factors explaining variation in my dependent variable within-units and between-units. So, that I could say something along the lines of "Indep_var1 is the most important driver of variation in my dependent variable between units" and "Indep_var2 is the most important driver of variation in my dependent variable within units" - I would expect the relative importance to be different within-units and between-units. To my understanding, that is what I get when running dominance analyses on a separately estimated FE model (using xtreg, fe) and a separately estimated BE model (using xtreg, be), as those coefficients (approximately) correspond to the coefficients of the within-/between-components of the hybrid model - here, I do find differences in the relative importance, but I'm not sure whether that (or the use of mixdom, giving back results for R² within and R² between separately) is an approach I can use or whether breaking down the relative importance of factors into a between- and a within-part even makes sense - do you have any thoughts on that?

Thanks a lot!

[daniel klein]

I am struggling with the notion of "importance", as I am not aware of its statistical/mathematical definition (if such a definition exists). Depending on your specific goals, separate analyses of within and between variation might be the better way to go.

To comment further, I would need to know more about the background (and have the time to process the information); some questions include: Which substantive question(s) are you trying to answer? How do you define "importance" with respect to your question(s) and does this definition correspond to what you are getting from dominance analyses? How does the difference between with- and between-unit variance relate to the question(s)?

[Joseph Luchman]

Felicia Hollingsworth, -mixdom- assumes a two-level model with random effects for the intercept only (it is quite literally the model discussed by Luo & Azen (2013) cited in the program's helpfile). This is because the Snijders & Bosker (1994) fit metric produced by -mixdom- assumes the underlying model is a two-level random-intercept model. That is, it is unlikely to mirror the results from -xthybrid- given that it is a model that intends to build in fixed effects based on my quick read of the Stata Journal article.

If you would like to dominance analyze the -xthybrid- command's results by within- and between-cluster I recommend you find or generate a program that can produce a fit metric that does so. Thus, it would be possible to dominance analyze the command you would like to in the way you would like if you can find fit metrics that do what you intend. I can't say that I am familiar with metrics that can off hand.

The -e(ll)- and -e(chi2)- returned values _can_ be used for dominance analysis but they do not separate within- and between-cluster variability and will mix both cluster levels based on their contribution to the log-likelihood. Most likely, using these as fit statistics will emphasize the largest source of variance in the results - if the data have a lot more within-cluster variance than between, the within-cluster results will be emphasized and vice versa. This can be useful but it sounds like you would like the levels separated which these metrics cannot do.

daniel klein, also makes a good point that there really is no mathematical/statistical definition of importance. -domin- produces a Shapley value decomposition (with a few other useful statistics for evaluating the model) of a fit statistic that is oftentimes used as a metric to reflect importance of independent variables/IVs. From this perspective, "importance" has historically focused on improvement to predictive quality/reduction of prediction error that can be ascribed to an IV using this method.

- joe