Margins- cross-level interactions with fixed effects

[Osman Sabri]

Hello,
I have a question with regard to margins. In my model, I am trying to estimate cross-level interactions between useonline, an individual level variable, and broadband, a region-level variable. I also use fixed effects at the regional level. Now, when I run the model like this below, and run margins for various levels of broadband observable in the data, Stata fails to estimate the marginal effects.

logit trust i.regions c.useonline##c.broadband  , vce(robust) nocons
margins, dydx(useonline ) at (broadbandnuts=(86(1)98) )

. margins, dydx(useonline ) at (broadbandnuts=(86(1)98) )

Average marginal effects                                 Number of obs = 1,127
Model VCE: Robust

Expression: Pr(trust), predict()
dy/dx wrt:  useonline
1._at:  broadbandnuts = 86
2._at:  broadbandnuts = 87
3._at:  broadbandnuts = 88
4._at:  broadbandnuts = 89
5._at:  broadbandnuts = 90
6._at:  broadbandnuts = 91
7._at:  broadbandnuts = 92
8._at:  broadbandnuts = 93
9._at:  broadbandnuts = 94
10._at: broadbandnuts = 95
11._at: broadbandnuts = 96
12._at: broadbandnuts = 97
13._at: broadbandnuts = 98

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
useonline    |
         _at |
          1  |          .  (not estimable)
          2  |          .  (not estimable)
          3  |          .  (not estimable)
          4  |          .  (not estimable)
          5  |          .  (not estimable)
          6  |          .  (not estimable)
          7  |          .  (not estimable)
          8  |          .  (not estimable)
          9  |          .  (not estimable)
         10  |          .  (not estimable)
         11  |          .  (not estimable)
         12  |          .  (not estimable)
         13  |          .  (not estimable)
------------------------------------------------------------------------------

I first thought that this was because of region fixed effects. However, when I run the following command, it can estimate them for each value of broadband.

margins, dydx(useonline ) over(broadband)

Average marginal effects                                 Number of obs = 1,127
Model VCE: Robust

Expression: Pr(trust), predict()
dy/dx wrt:  useonline
Over:       broadbandnuts

-------------------------------------------------------------------------------
              |            Delta-method
              |      dy/dx   std. err.      z    P>|z|     [95% conf. interval]
--------------+----------------------------------------------------------------
useonline     |
broadbandnuts |
          86  |   .2246624   .1033492     2.17   0.030     .0221018    .4272231
          89  |   .2701895   .0731823     3.69   0.000     .1267548    .4136242
          90  |   .2474129   .0626573     3.95   0.000     .1246068     .370219
          93  |   .2558975   .0417731     6.13   0.000     .1740236    .3377714
          94  |   .2583967   .0419266     6.16   0.000      .176222    .3405714
          95  |   .2524966   .0475349     5.31   0.000     .1593298    .3456633
          96  |   .2580284   .0627717     4.11   0.000      .134998    .3810587
          98  |   .2501801   .0947657     2.64   0.008     .0644428    .4359174
-------------------------------------------------------------------------------

My question is are these commands not the same? Why does the first one fail, but the second one succeeds?

Relatedly, when I run the model without region fixed effects, and again run the same marginal effects codes above, both estimates run fine expectedly, but their results slightly differ from each other. Why is that?

Thank you very much

[Clyde Schechter]

I cannot say for certain why Stata is not able to estimate the effects in your first situation. Most likely it is because there are combinations of bandwidth and region that do not co-occur in the estimation sample of the logistic regression.

What I can tell you with certainty is that -at()- and -over()- are not at all equivalent--that is a common misunderstanding. Indeed, it is really just a coincidence in your data that they gave somewhat similar results. In general, they are almost unrelated to each other. When you use -over()- the calculations are done separately in subsets of the estimation sample that are defined by the distinct values of the variable specified in -over()-. In particular, these calculations do not, and cannot, adjust for differences in the distributions of other model variables for differing values of the -over()- variable.. These are conditional margins/marginal effects, not adjusted for variables whose distributions differ when values of the -over()- variable differ.

By contrast, when you use -at()- (or, for factor variables, specify the variable in the varlist of the -margins- command), Stata uses the entire estimation sample of the regression, in turn substituting each value specified in -at()- for the -at()- variable in every observation of the estimation sample. Consequently, in these results, the margins/marginal effects estimated are fully adjusted to the joint distribution of all other model variables.

[Osman Sabri]

Clyde, thank you very much -as always. In that case, I need to find a way to make margins -at()- command work. It is odd why it cannot estimate it though, as even when I try to run it for a single broadband level, for which there is at least one region exists - (like the code below), it still gives me non-estimable output. I hope I can figure it out.

margins, dydx(useonline ) at (broadbandnuts=(94))

[Osman Sabri]

I reduced my dataset to a mere two regions with two different broadband values (86 and 94), no missing data. Margins -at() still does not work.

I think, there should be a theoretical reason for why it does not work. I have no idea why.

I attach 100 observations using dataex. Could you please take a look?

* Example generated by -dataex-. For more info, type help dataex
clear
input byte trust long regions byte(useonline broadbandnuts)
0 86 0 94
0 86 0 94
1 86 0 94
0 86 0 94
0 86 0 94
1 86 0 94
0 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
0 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
0 86 0 94
0 86 0 94
1 86 0 94
0 86 0 94
1 86 0 94
1 86 0 94
0 86 0 94
1 86 1 94
1 86 1 94
0 86 0 94
1 86 0 94
1 86 0 94
0 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
0 86 0 94
0 86 0 94
1 86 0 94
0 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
0 86 0 94
0 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
0 86 0 94
0 86 0 94
0 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
0 86 1 94
0 86 0 94
0 86 0 94
1 86 0 94
1 86 0 94
0 86 0 94
1 86 0 94
0 86 0 94
0 86 0 94
1 86 1 94
1 86 0 94
1 86 0 94
0 86 0 94
0 86 0 94
0 86 0 94
0 86 1 94
1 86 0 94
1 86 0 94
1 86 0 94
0 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
0 86 0 94
0 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
1 86 0 94
0 86 0 94
1 86 0 94
1 86 0 94
0 86 0 94
0 86 0 94
0 86 1 94
1 86 0 94
0 86 0 94
1 86 0 94
0 86 0 94
0 86 0 94
end
label values trust QA6A_10
label def QA6A_10 1 "Tend to trust", modify
label values regions nuts222
label def nuts222 86 "DEA", modify
label values useonline QA6A_5
label def QA6A_5 1 "Tend to trust", modify

logit trust i.regions c.useonline##c.broadband , vce(robust) nocons
margins, dydx(useonline ) at (broadbandnuts=(86) )

thanks, and best
OSK

[Clyde Schechter]

Though this problem is quite restricted, and what's going on here may not apply to your general problem, if you change the value in the -at()- option from 86, which does not occur as an actual value in the data, to 94, which does, you will get results:

. margins, dydx(useonline ) at (broadbandnuts=(94) )

Average marginal effects                                   Number of obs = 100
Model VCE: Robust

Expression: Pr(trust), predict()
dy/dx wrt:  useonline
At: broadbandnuts = 94

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
   useonline |  -.1034362   .2018312    -0.51   0.608    -.4990181    .2921457
------------------------------------------------------------------------------

Unfortunately, I do not, myself, quite understand why this is happening. While I can argue that, in principle, one should not extrapolate outside the range of the data, in fact, in other situations -margins- quite freely does just that. I also note that, if you put the 86 back, the trick of removing i.region, which worked for you earlier, no longer does. Sorry I can't offer more helpful advice.

[Osman Sabri]

No problem. Thank you very much for all your help Clyde. I guess in that case I have to decide whether to give up region FEs or stick with margins, over(), despite the obvious drawbacks you mentioned before. I hope this thread will be useful for people testing cross-level interactions and experience a similar problem.