Dominance Analysis - comparing between (not within) models and relative importance determination for systems GMM

[Sam Murgatroyd]

Hello,

My post concerns the application of dominance analysis using Stata's -domin- command. I learned about it here.

Issue #1

I am using panel data on 56 countries over five years. I had initially intended to apply dominance analysis to a model that compares the relative importance of two independent variables - I'll call them Most Interesting Variable 1 (MIV1) and Most Interesting Variable 2 (MIV2). However, due to the highly correlated nature of MIV1 and MIV2, including them in the same regression is problematic.

So I have decided to split the model in two. In other words, I run a model as shown in eq(1) and a model as shown in eq(2), where X_jt is a vector of country-level macroeconomic factors. The regression is run using OLS. Country and year fixed effects are included and standard errors are clustered at the country-level. The only difference between these two models is that eq1 includes MIV1, while eq2 includes MIV2. The panel is balanced in both cases.

Y_jt= β₀ + β₁MIV1_jt + β₂X_jt + δ_t + α_j + u_it (Eq1)

Y_jt= β₀ + β₁MIV2_jt + β₂X_jt + δ_t + α_j + u_it (Eq2)

I will then run -domin- for each model (I plan on renting online space to run the models because of my inclusion of fixed effects will probably make this take very long). Because I am not interested in determining relative importance within each model for its own sake, but rather, interested in understanding the relative importance of MIV1 and MIV2, which are in separate models, I was planning to compare the standardised general dominance statistic for MIV1 from eq1 and the standardised dominance statistic for MIV2 from eq2 and to use their respective contributions to the within R-squared of each model to inform an assessment of their relative importance.

Question: Is it appropriate to compare the standardised general dominance statistics on two variables from different regressions, where these different regressions are identical in every respect except that one regression includes MIV1 and one includes MIV2? If not, and one just obtains coefficients from MIV2 and MIV2 as outputs for estimating two separate models, are there commonly accepted methods for comparing the relative importance of the coefficients across models? Simply comparing the size and statistical significance of MIV1 and MIV2, or the within R-squares of each model seems a bit naive.

Issue #2:

My dependent variable Y_jt is serially correlated, which means the fixed effects OLS estimates are biased.

Question: If it is okay to use standardised general dominance statistics to compare the independent variables of interest between two models, and if one's research question is about determining relative importance between two independent variables using generalised dominance statistics, does the bias matter that much if this bias is 'consistent' between the two models?

Issue #3

Because Y_jt is serially correlated, in addition to the static specification, I'd also like to run a specification that includes the lag of the dependent variable. For this, I have determined that systems GMM using -xtabond2- is most appropriate given my short T and N>T. However, to my understanding, this throws out the use of Dominance Analysis as an option to assess relative importance of MIV1 and MIV2 because -xtabond2- does not produce an appropriate fit-statistic.

Question: Is there a way that an -xtabond2- regression can be dominance analysed? If not, is there a procedure for comparing the relative importance of independent variables across two (almost) identical models in a dynamic panel setting? The short T and moderate N of my data constrain my options of dynamic panel data methods.

Thank you for taking the time to read this.

Sam

[Joseph Luchman]

Hi Sam,

Interesting question here. Overall, the value in comparing dominance statistics between these two models is low as they are really intended to compare predictors within a model.

Given the models are identical save for MIV1 and MIV2, you could get a pretty good sense of which is likely the better of the MIV-s by comparing some sort of R² value of the models without the full dominance analysis/DA as well as the bivariate correlation (or a similar suitable alternative given the structure of the data) for the two MIV-s. If one of the MIV-s ends up with a higher value in the full model as well as the bivariate case, that one is likely to be better.

To answer issue #1, the short answer is that that there's not a lot of value in using DA to compare a single predictor in two different models. Again, DA's value is in getting comparisons within a model and using it to compare across is a lot of extra computational time spent divvying up predictors that are of less interest. Recommend comparing the fit of the full model and that of the MIV-s alone with the Y.

For #2, given that they are intended to be 'relative importance' statistics - if you can make an argument that the bias is relatively the same or consistent across models, it should not unduly affect the importance comparisons. It's not hard to put together some simulations to support that idea either.

For #3, I know _of_ -xtabond2- as it is a rather well-known command but I have not in my research practice had the need to use it and am not in a position to comment on the ease with which it could be DA-ed. In principle, any model can be DA-ed so long as there's a valid fit statistic and way to include and exclude a predictor - which I suspect is true of the model. The details are important though and that's where I cannot be of much assistance.

- joe

[Sebastian Kripfganz]

For system GMM, the Andrews-Lu model and moment selection criteria (MMSC) could potentially be of use (although I do not have any experience with dominance analysis). xtabond2 does not report them, but you can get them with estat mmsc after my xtdpdgmm command, which is an alternative to xtabond2.

I have some slides on those MMSC in my 2019 London Stata Conference presentation:

• Kripfganz, S. (2019). Generalized method of moments estimation of linear dynamic panel data models. Proceedings of the 2019 London Stata Conference.