Problem with margins and marginsplots for categorical variables in a fixed-effects regression model

[Charlotte Marx]

Dear all,

currently I'm analyzing Linked-Employer-Employee Daten via fixed effects regression models. To illustrate the effects nicely, I want to display my results with marginsplots. The regression model looks like this: I have a metric variable as a depent variable in a fixed effects model, thats why I calculate with xtreg var1 var2 ..., fe vce (cluster groupvar). My groupvariable is organization/employer. I want to analyze the impact of special organizational policies on the dependent variable. I know, since I do a fixed effects regression I can only include the organizational variables using an interactional term with an individual variable. My interaction terms are built like this: categorical organizational level var (dummy; called cfam) x categorical individual level var (4 categories; called par). Since this can be very complex, I want to use marginsplots to illustrate this interaction more properly.
I already studied a book about margins and marginsplots, but my plots aren't very decent and either don't display the effects from the regression model (esp. regarding significance), or they don't display any effect, since they are "not estimable". I think maybe the problems originate from the lacking main effect of the organizational variable (fixed effects), but I actually don't know how to fix this.

Does anybody know, how I can solve this problem und get decent marginsplots for my analyses?

I already tried:
contrast cfam#par
contrast cfam@par
margins cfam#par
marginsplot

and

margins par, at (cfam = (0 (1) /*
*/ 1)) level (90) contrast noestimcheck
marginsplot

I would really appreciate your time and effort for reading and answering this post!

Best regards,
Charlotte

[Scott Merryman]

It is not clear what you mean by plots that are not decent. Using a sample data set would be helpful in understanding your difficulty. For example:

webuse nlswork,clear
qui {
    egen agecat = cut(age), at(10, 20, 30, 40, 50) icode
    xtreg ln_wage tenure i.not_smsa##i.agecat, fe vce(cluster idcode)
    margins r.agecat, at(not = (0 1)) contrast
}
marginsplot, xdim(agecat)

[Charlotte Marx]

Thank you for your quick answer! I'm sorry, that I didn't verbalize it clearly. I'm unfortunately not that into marginsplots already. With decent I mean that I think the marginsplot looks right, but it in my opinion it doesn't fit with the data in the regression model. I hope I can make it a bit more clear by this. The coefficients in my regression model are the following:

Main effects

par (Ref. 0)
1	.1212005	.0640901	1.89	0.063	-.0065607	.2489617
2	-.1033515	.042346	-2.44	0.017	-.1877667	-.0189363
3	-.2316742	.0547791	-4.23	0.000	-.3408744	-.1224741

cfam
Ja	0	(omitted)

Interaction

cfam#par
Ja#1	.0213216	.0343198	0.62	0.536	-.0470937	.089737
Ja#2	-.0597684	.0354222	-1.69	0.096	-.1303812	.0108444
Ja#3	-.1151813	.0439448	-2.62	0.011	-.2027836	-.0275789

The marginsplot looks like this:

Perhaps I don't understand it, but shouldn't the values pictured in the plot be the same as the model? For example

Ja#2

-.0597684

at -0.05 and not at about -1.5? Even if I add the interactioneffect to the main effect this doesn't work. For this plot I used the following command:
margins par, at (cfam = (0 (1) /*
*/ 1)) level (90) contrast noestimcheck
marginsplot

I actually don't get if the values in the plot are the coefficients or the y-value? And where is the significance pictured in the plot? Because I have plots where the confidence intervals doesn't cut y=0 but in my regression model the coefficient isn't significant.


Kind regards,
Charlotte

[Scott Merryman]

Keep in mind what marginsplot after margin, contrast is plotting. Referring to the "par: 2 vs base" line:

The first point is (par = 2 , cfam=0) - (par = 0, cfam =0) the second point is (par =2 , cfam =1) - (par= 0 , cfam= 1)

Which given your coefficients equals
First Point = (-.1033515 + constant ) - (constant) = -.1033515
Second Point = (-.1033515 + -.0597684 + constant) - (0 + constant) = - .1631199


Take at look at this example:

clear
set seed 45342
set obs 1000
gen id = _n
generate u_i = rnormal(0,.1)
expand 20
bysort id: gen t = _n
generate e_ij = rnormal(0,.1)
gen e = runiform()
egen par = cut(e), at(0 .3, .5, .7, 1) icode

gen x = rnormal()
gen cfam = rbinomial(1, normal(x))
tab par ,gen(_p)
tab cfam, gen(_c)

gen y =-1.5 +  .1*_p2 - .1*_p3 - .23*_p4 /// 
    + .02*_c2*_p2  - .06*_c2*_p3 - .12*_c2*_p4  /// 
    + u_i + e_ij 
xtset id t
xtreg y i.p##i.c, r
margins , at(p = (0 1 2 3) c = (0 1))  vsquish 
margins r.p, at(c = (0 1)) contrast  
marginsplot

disp "PAR=2 & CFAM = 0 vs Par = 0 , CFAM = 0"
lincom (2.p  + _cons) - (_cons)

disp "PAR=2 & CFAM = 1 vs Par = 0 , CFAM = 1"
lincom (2.p + 1.c+  2.p#1.c + _cons) - (1.c + _cons)

[Charlotte Marx]

Oh, ok! I think, I get the values in the plot. Actually that was the biggest problem. I wasn't sure, what to see in the plot.
I ran the plot and I understand it. Just one thing I don't get. Can I also see in the plot if the groups differ significantly? Since par = 0 is my reference the confidence intervals in cfam = 1 shouldn't cut y=0, should they? Because in some analyses the interaction effect is not significant, but in the margins plot the confidence intervals of par = 2 and par = 3 dont't cut the y=0 line. Or are these two different things, since the coefficient in the regression model actually doesn't equal the value on the par = 2 & cfam = 1 point in the plot.

Kind regards,
Charlotte

[Scott Merryman]

The significance is tied to the points on the graph not the lines connecting them.

Take a look at the graphs below. The top two show the adjusted prediction and the contrast prediction when the x-axis is either PAR or CFAM. The bottom two show the contrasts. Looking at graph 3, one could say that the difference between CFAM =0 and CFAM =1 is insignificant when PAR = 0, otherwise the contrasts are significant. In graph 4, the six points are showing the difference between the points in graph 2 relative to when PAR = 0. In this case, the all the contrasts are significantly different from zero.

clear
set seed 45342
set obs 1000
gen id = _n
generate u_i = rnormal(0,.1)
expand 20
bysort id: gen t = _n
generate e_ij = rnormal(0,.1)
gen e = runiform()
egen par = cut(e), at(0 .3, .5, .7, 1) icode

gen x = rnormal()
gen cfam = rbinomial(1, normal(x))
tab par ,gen(_p)
tab cfam, gen(_c)

gen y =-1.5 +  .1*_p2 - .1*_p3 - .23*_p4 /// 
    + .02*_c2*_p2  - .06*_c2*_p3 - .12*_c2*_p4  /// 
    + u_i + e_ij 
xtset id t
xtreg y i.p##i.c, r
margins , at(p = (0 1 2 3) c = (0 1))  vsquish 
marginsplot, name(gr1,replace) legend(ring(0))
marginsplot, xdim(cfam) name(gr2,replace)  legend(row(1) pos(6) ring(0) size(*.8))

margins r.p, at(c = (0 1)) contrast  
marginsplot, name(gr3,replace) legend(row(1) pos(6) ring(0) size(*.8))

margins r.c, at(p  = (0 1 2 3)) contrast
marginsplot, name(gr4,replace) 

graph combine gr1 gr2 gr4  gr3

[Igor Paploski]

Great post Scott Merryman, thanks for it.

Looking at graph 1 on your prior post, I would be tempted to say that there is no difference in the predicted value of y for CFAM == 0 and CFAM == 1 when PAR == 1, based on the confidence intervals displayed on the graphic. However, looking at graph 3, as you interpreted before, one could say that the difference in y between CFAM =0 and CFAM =1 is significant when PAR == 1. Am I missing something obvious here? How do you deal with those (apparent) contradictions?

[Scott Merryman]

Overlapping confidence intervals for the predictions do not necessary imply that the differences in predictions will be insignificant.


Note, in the code above the u_i error term should be:

generate u_i = rnormal(0,1)

[Igor Paploski]

Ahh, I see, great! Thank you!

[Charlotte Marx]

Dear Scott Merryman,

first thank you for your effort and patience with me as I'm a beginner with plotting and interpreting margins and also for this great example. After that I understand the graphs and why some points are significant or not significant (esp. that it's the points, which count). Just one further question: If the confidence interval of par = 1 in graph 3 or graph 4 would cut the y= 0 line, would par = 1 still be significant?

[Scott Merryman]

No, if the interval includes zero then there is not a statistically significant difference in the contrasts. This can also be determined from the margins output.

[Charlotte Marx]

Ok, that‘ s what I thought, but I wasn‘t so sure about it. Many thanks to you for explaining this in a really understandable way!

I really appreciate your time and effort very much!

Kind regards,
Charlotte