interpretation of three-way interaction and margins

[Patrick Lang]

Dear Clyde Schechter,

as I appreciate your insights regarding interaction terms, I was hoping you might help me interpret my results.
I am working on graphing the interaction results of my fixed-effect regression. (panel data: UltimateParentID Year).

I have a sample of organizations from the U.S. and Europe. I analyze the effect of DSGR_L1 on ENV.

xtreg ENV c.DSGR_L1##i.US Totalassets_L1 ROA_L1 SustCommitt_L1 TMTESG_L1 GDP_L1 FDI_L1 i.Year, fe vce(robust)
​​​​​​​

The results indicate that the effect of DSGR_L1 is contingent upon whether the organization is based in the U.S. (positive effect, US=1) or Europe (negative effect, US=0).

I have two other variables that interact with my IV DSGR_L1. One of these variables is esg_diff_l1. Running the model with both interaction terms shows that both are significant.

xtreg ENV c.DSGR_L1##i.US c.DSGR_L1##c.esg_diff_l1 Totalassets_L1 ROA_L1 SustCommitt_L1 TMTESG_L1 GDP_L1 FDI_L1 i.Year, fe vce(robust).
​​​​​​​

So far so good, my problem, however, arises when I use the margins command to plot my interactions. Specifically, I wanted to show the moderation effect of esg_diff_l1 for both regions (US and Europe). So I started with:

xtreg ENV c.DSGR_L1##i.US c.DSGR_L1##c.esg_diff_l1 Totalassets_L1 ROA_L1 SustCommitt_L1 TMTESG_L1 GDP_L1 FDI_L1 i.Year, fe vce(robust)

margins US, dydx(DSGR_L1) at(esg_diff_l1 = (0 50 100 150 200 250 300)) noestimcheck

marginsplot
​​​​​​​

Maginsplot shows that the moderation effect of esg_diff_l1 is negative for organizations from the U.S. and Europe. From a subsample analysis, I know that the effect of esg_diff_l1 is negative for the U.S. and positive for Europe. My initial thought was that my command did not tell Stata correctly to differentiate between the U.S. and Europe when creating the graph as both interaction terms are separately included in my model. Thus, I conducted a three-way interaction.

xtreg ENV c.DSGR_L1##i.US##c.esg_diff_l1 Totalassets_L1 ROA_L1 SustCommitt_L1 TMTESG_L1 GDP_L1 FDI_L1 i.Year, fe vce(robust)

margins US, dydx(DSGR_L1) at(esg_diff_l1 = (0 50 100 150 200 250 300)) noestimcheck

marginsplot
​​​​​​​

This plot indeed shows the correct relationship of esg_diff_l1 being a negative moderator for the organizations from the U.S. and a positive moderator for organizations from Europe. However, I am still a little bit confused. The, the three-way interaction term c.DSGR_L1##i.US##c.esg_diff_l1 is not statistically significant even though the term DSGR_L1##c.esg_diff_l1 is significant if I test it in the US-only sample. Can you help me with interpreting these results? In my discipline, it is common to introduce the moderation effects in isolation and then test them in combination.

xtreg ENV c.DSGR_L1##i.US Totalassets_L1 ROA_L1 SustCommitt_L1 TMTESG_L1 GDP_L1 FDI_L1 i.Year, fe vce(robust)
 
​​​​​​​xtreg ENV c.DSGR_L1##c.esg_diff_l1 Totalassets_L1 ROA_L1 SustCommitt_L1 TMTESG_L1 GDP_L1 FDI_L1 i.Year, fe vce(robust)

​​​​​​​xtreg ENV c.DSGR_L1##i.US c.DSGR_L1##c.esg_diff_l1 Totalassets_L1 ROA_L1 SustCommitt_L1 TMTESG_L1 GDP_L1 FDI_L1 i.Year, fe vce(robust)
​​​​​​​

Thanks in advance for your help
Patrick

* Example generated by -dataex-. To install: ssc install dataex
clear
input double UltimateParentID int Year double ENV float DSGR_L1 int esg_diff_l1 byte US float(Totalassets_L1 ROA_L1 SustCommitt_L1 TMTESG_L1) double(GDP_L1 FDI_L1)
4295641240 2017 5.44087665647298  0 305 1   510564992   4.485109 . .     18695110842000  2.537497659196815
4295858988 2014 34.5515083440308 18 154 0   653032000   9.450018 . .   430190979705.962  .1048621859852468
4295858988 2015 35.3520671834625 13 159 0   954963968   12.13187 0 0 442584815286.03375   .386858115054066
4295858988 2016 30.9385167110322  7 236 0  1223380992  13.649537 0 0  381971148530.5428  -2.08805653898309
4295858988 2017 29.2615245214633  7 305 0  1422988032   7.776542 0 0   395837353031.499 -7.310916958037089
4295859306 2011 17.1794871794871  0 143 0   7.547e+08   6.573964 0 0  481420882905.0009  26.06698199624618
4295859306 2012 16.4672841094245  0 146 0   814566976   9.716128 0 0  523330354138.1333  31.26598334890303
4295859306 2013 21.3055865229778  0 149 0   921878976  10.791467 0 0  496152879924.7267  2.380369601496919
4295859306 2014 25.7891737891737  0 154 0  1047822016   5.718838 0 0  521791015247.0603 -5.680612374306783
4295859306 2015 45.8000949667616  0 159 0  1075384064  2.0372021 0 0  535390200131.0177 -2.840147495087677
4295859306 2016 60.0688538188538  0 236 0  1140327040 -1.7584423 1 1 462335574841.48413  -4.22079695282357
4295859306 2017 60.5692918192918  0 305 0  1159230976  2.2100768 1 0 476062757356.92725  12.08629547636072
4295860976 2016  14.033264033264  0 236 1   416044992   8.771708 . .     18206020741000  2.809147629104088
4295860976 2017 12.7060439560439  1 305 1   425636992   5.228341 0 0     18695110842000  2.537497659196815
4295865864 2011 38.0459459459459  0 143 0  2.5617e+09  -1.259957 1 0  321995279401.5016 -3.654669083017719
4295865864 2012 35.4941094941094  0 146 0  2.5082e+09  1.1045582 1 0  344003137611.2712  3.941503187586579
4295865864 2013 29.7948717948717  0 149 0  2.8922e+09   2.714614 1 0  327148943812.1366 -4.997670325554282
4295865864 2014 32.4207504207504  0 154 0  2.7568e+09  -5.625774 1 0 343584391647.92706  .1980121805520809
4295865864 2015 31.5469975752389  0 159 0  2.8921e+09  -8.061747 1 0   352993631617.708  1.863305283018756
4295865864 2016 29.9643033858768  0 236 0   3.449e+09 -19.154406 1 0 302673070846.85724  .6114751833331019
4295865864 2017 31.8978723404255  0 305 0  2.8318e+09  -6.298561 1 0  313115929314.3386  2.492374260307158
4295866378 2011 70.4494767541602  0 143 0  1.2406e+09   4.064206 0 0 249424310816.66714  4.899686358623893
4295866378 2012 68.7433129280081  0 146 0  1.2799e+09   5.070423 1 0 275604356167.31573 -2.179936953365001
4295866378 2013 68.2020149399234  0 149 0  1.1796e+09    2.39073 1 0 258290060227.73444  1.909129174632278
4295866378 2014 71.9788673877528  0 154 0  1.0946e+09   5.470056 1 0   271362405890.589 -1.813913578914756
4295866378 2015 73.1561302681992  0 159 0  1.0315e+09   3.292413 1 0 274862826772.15588  6.377217393475116
4295866378 2016 71.8427660400657  0 236 0  1.0863e+09   8.546605 1 0  234534382384.7655  7.455255505912498
4295866378 2017 69.3128493603397  0 305 0  1.0745e+09   9.922251 1 0 240771351298.83328  2.128191378780703
4295866480 2011 68.4313725490196 68 143 0  3.9123e+10   3.587983 1 1 249424310816.66714  4.899686358623893
4295866480 2012  89.576124567474 56 146 0  3.6205e+10  -3.950722 1 1 275604356167.31573 -2.179936953365001
4295866480 2013 83.9965397923875 72 149 0  2.9984e+10  -4.481107 1 1 258290060227.73444  1.909129174632278
4295866480 2014 82.1940953884133 57 154 0  2.5191e+10   .4639783 1 1   271362405890.589 -1.813913578914756
4295866480 2015 83.5994397759103 41 159 0  2.1063e+10  11.752497 1 0 274862826772.15588  6.377217393475116
4295866480 2016 92.8566176470588 15 236 0  2.0926e+10   5.687204 1 0  234534382384.7655  7.455255505912498
4295866480 2017 93.1421256897035  5 305 0  4.4901e+10 -2.7708995 1 0 240771351298.83328  2.128191378780703
4295866790 2016 76.4606102262552  0 236 0  3.8213e+09   2.247896 . . 2439188643162.4985  1.755723741416579
4295866790 2017 72.4421928318466  0 305 0  3.8849e+09  3.7632036 1 0 2472964344587.1655  1.326491698127181
4295866983 2011                0  0 143 0  4717032960   6.058258 1 0  2645187882116.672  1.470195726973971
4295866983 2012 16.1906625445153  0 146 0  4751194112   7.493335 1 0  2865157541994.169  1.542875408373239
4295866983 2013 18.4599619382228  0 149 0  4.9704e+09   7.017368 1 0  2683671716967.188  1.227596621061495
4295866983 2014 23.5836627140975  0 154 0   5.605e+09   6.993589 1 0 2811876903329.0273  1.123423424160419
4295866983 2015 24.8447204968944  0 159 0  7.0059e+09   5.014709 1 0  2855964488590.186  .2032545627180351
4295866983 2016 36.3739183040059  0 236 0  7.6589e+09   5.048824 1 0 2439188643162.4985  1.755723741416579
4295866983 2017 39.0338675189111  0 305 0  8.8821e+09  4.3866754 1 0 2472964344587.1655  1.326491698127181
4295867015 2011 47.0530489315872  1 143 0  4.4791e+09   2.643231 1 0  2645187882116.672  1.470195726973971
4295867015 2012 47.3304924806802  1 146 0  7.3778e+09    3.08681 1 0  2865157541994.169  1.542875408373239
4295867015 2013  45.185271502035  3 149 0  7.4473e+09   3.058327 1 0  2683671716967.188  1.227596621061495
4295867015 2014 44.4700102967594  0 154 0  7.2022e+09    3.52367 0 0 2811876903329.0273  1.123423424160419
4295867015 2015 44.0469348659003  0 159 0  9.0388e+09    3.50471 0 1  2855964488590.186  .2032545627180351
4295867015 2016 42.1656520937096  0 236 0 1.06808e+10   4.405769 0 1 2439188643162.4985  1.755723741416579
4295867015 2017 71.9128003046615  1 305 0 1.34003e+10   5.235641 0 1 2472964344587.1655  1.326491698127181
4295867357 2011 74.2808943786813  0 143 0  3071834112   8.217268 0 0  2645187882116.672  1.470195726973971
4295867357 2012 72.8828410796942  1 146 0  3516784128   8.818268 1 0  2865157541994.169  1.542875408373239
4295867357 2013 77.0099893982598  2 149 0  3612425984   9.505373 1 0  2683671716967.188  1.227596621061495
4295867357 2014 72.9262709769089  0 154 0  4187892992    9.10655 1 0 2811876903329.0273  1.123423424160419
4295867357 2015 77.0162835249042  0 159 0  4962383872   6.392156 1 0  2855964488590.186  .2032545627180351
4295867357 2016 78.4504030074149  1 236 0  6311430144   7.201662 1 0 2439188643162.4985  1.755723741416579
4295867357 2017 76.0641370718923  2 305 0  6943066112   6.823451 1 0 2472964344587.1655  1.326491698127181
4295867377 2011  33.397190293742  0 143 0  1184118016  -3.768238 1 0  2645187882116.672  1.470195726973971
4295867377 2012 63.3349084314576  0 146 0  1057902016  -4.649379 1 0  2865157541994.169  1.542875408373239
4295867377 2013 64.2050370451609  0 149 0  1098616064   3.461228 1 0  2683671716967.188  1.227596621061495
4295867377 2014 63.0012041575122  0 154 0  1257437952   5.503439 1 0 2811876903329.0273  1.123423424160419
4295867377 2015 64.1810344827586  0 159 0  1339314944  -5.046687 1 0  2855964488590.186  .2032545627180351
4295867377 2016 40.8958986656828  0 236 0  1752472064   5.618045 1 0 2439188643162.4985  1.755723741416579
4295867377 2017 41.5608903605592  0 305 0  2020829056   4.950996 1 0 2472964344587.1655  1.326491698127181
4295868093 2017 36.8995789387621  0 305 0  1328967040    8.62386 . . 2472964344587.1655  1.326491698127181
4295868112 2011 14.1975308641975  0 143 0  1.9704e+09   7.744469 1 0  2645187882116.672  1.470195726973971
4295868112 2012 19.3672839506172  1 146 0  2.0556e+09   7.794337 1 0  2865157541994.169  1.542875408373239
4295868112 2013  18.287037037037  1 149 0  2.2429e+09   7.193207 1 0  2683671716967.188  1.227596621061495
4295868112 2014 23.5339506172839  1 154 0  2.4859e+09   6.822027 1 0 2811876903329.0273  1.123423424160419
4295868112 2015 21.2962962962962  1 159 0  2.5793e+09   6.475559 1 0  2855964488590.186  .2032545627180351
4295868112 2016 18.8271604938271  0 236 0  3.0422e+09  4.7247176 1 0 2439188643162.4985  1.755723741416579
4295868112 2017 64.8119908859713  0 305 0  2.8433e+09   4.421035 1 0 2472964344587.1655  1.326491698127181
4295868416 2011 81.9483150252156  3 143 0  9.4275e+10   4.111564 1 0  2645187882116.672  1.470195726973971
4295868416 2012 83.4472713575705  1 146 0  9.6083e+10  4.0218954 1 0  2865157541994.169  1.542875408373239
4295868416 2013 85.4223393353828  5 149 0   8.998e+10  1.3124586 1 0  2683671716967.188  1.227596621061495
4295868416 2014 85.6642958392693  1 154 0  8.5833e+10  2.4776325 1 0 2811876903329.0273  1.123423424160419
4295868416 2015 85.0033675072962  3 159 0  8.8404e+10   1.561092 1 0  2855964488590.186  .2032545627180351
4295868416 2016 82.9752886714537  2 236 0   9.143e+10   2.791463 1 1 2439188643162.4985  1.755723741416579
4295868416 2017 87.0392298122048  0 305 0  9.4668e+10  1.0854496 1 1 2472964344587.1655  1.326491698127181
4295869055 2017 37.0058896817306  0 305 0  1.0568e+09   .8295195 . . 3469853463945.5337  1.865906916904581
4295869278 2011 2.33079336620349  0 143 0  1271328000   9.843366 0 0 3399667820000.0063  2.530762023160379
4295869278 2012 1.58102766798419  0 146 0  1187010944  13.206722 0 0  3749314991050.591  2.601419304202527
4295869278 2013 1.58102766798419  0 149 0  1107708032   9.437844 0 0  3527143188785.157  1.855413407665223
4295869278 2014 1.75376957985653  0 154 0  1270294016  17.408396 . . 3733804649549.0303  1.799764603835211
4295869278 2015 2.32919254658385  0 159 0  3673424896  18.100706 0 0 3889093051023.5156  .5022271926341265
4295869278 2016 14.1481890633631  0 236 0  3885430016   9.699327 0 0 3357585719351.5605  1.860729181617582
4295869278 2017 35.8103858347575  0 305 0  4073733888   4.416519 0 0 3469853463945.5337  1.865906916904581
4295869822 2011                0  0 143 0  1599609984   10.79304 0 0 3399667820000.0063  2.530762023160379
4295869822 2012                0  0 146 0  1680695040  10.804422 0 0  3749314991050.591  2.601419304202527
4295869822 2013                0  0 149 0  1771858944   9.539431 0 0  3527143188785.157  1.855413407665223
4295869822 2014                0  0 154 0  1996852992   7.111766 0 0 3733804649549.0303  1.799764603835211
4295869822 2015                0  1 159 0  1848935936   5.749197 0 0 3889093051023.5156  .5022271926341265
4295869822 2016 4.67625899280575  0 236 0  1814769024   7.620974 0 0 3357585719351.5605  1.860729181617582
4295869822 2017 10.9271523178807  0 305 0  1957217024   7.441862 0 0 3469853463945.5337  1.865906916904581
4295870063 2011 61.1204430332945  9 143 0   4.993e+09   6.667379 0 0 3399667820000.0063  2.530762023160379
4295870063 2012 51.2465277777777 10 146 0   5.873e+09  13.694092 0 0  3749314991050.591  2.601419304202527
4295870063 2013 58.1161748220823 16 149 0   5.898e+09   7.340073 1 0  3527143188785.157  1.855413407665223
4295870063 2014 62.8256345241261 21 154 0   5.905e+09   4.795391 1 0 3733804649549.0303  1.799764603835211
4295870063 2015 72.3091498671667 18 159 0   6.438e+09   7.907316 1 0 3889093051023.5156  .5022271926341265
end
format %ty Year

[Clyde Schechter]

However, I am still a little bit confused. The, the three-way interaction term c.DSGR_L1##i.US##c.esg_diff_l1 is not statistically significant even though the term DSGR_L1##c.esg_diff_l1 is significant if I test it in the US-only sample.

The three way interaction term c.DSGR_L1##i.US##c.esg_diff_l1 estimates the difference between the two way interaction c.DSGR_L1##c.esg_diff_l1 in the US and the EU. The fact that one of these is statistically significant has no particular implication for whether the three way interaction will be statistically significant. The US and EU two-way interactions may be not all that different from each other. Even if one is negative and the other positive and both are statistically significant, it is important to remember that the difference between "statistically significant" and "not statistically significant" is, itself, not "statistically significant." The statistical significance of a difference between two things is more or less independent of the statistical significance of the two things themselves. This is one of the many paradoxes that people encounter with the concept of statistical significance, and one of the reasons why I discourage people from even thinking in those terms, and to focus instead on estimates of effect sizes and confidence intervals.

Finally, it is important also to remember that interaction terms are underpowered relative to their "main" effects. And a three way interaction is, similarly, substantially underpowered relative to its constituent two-way interactions. As a rule of thumb, whatever sample size you need to adequately power a test of a simple effect of a given size at a given statistical significance level, you need a sample 4 to 16 times as large to power the interaction of two such effects. For a three way interaction an even larger sample is needed.

[Erik Ruzek]

I agree with all of Clyde's points. In addition, it is worth mentioning that the data you shared only has three rows where US==1. So all the models you show in #1 do not give estimates for any predictors involving US, for example:

. xtreg ENV c.DSGR_L1##i.US##c.esg_diff_l1 Totalassets_L1 ROA_L1 SustCommitt_L1 TMTESG_L1 GDP_L1 FDI_L1 i.Year, fe vce(robust)
note: 1.US omitted because of collinearity
note: 1.US#c.DSGR_L1 omitted because of collinearity
note: 1.US#c.esg_diff_l1 omitted because of collinearity
note: 1.US#c.DSGR_L1#c.esg_diff_l1 omitted because of collinearity
note: 2017.Year omitted because of collinearity

[Patrick Lang]

Clyde Schechter thanks for the helpful comment.
Erik Ruzek thanks for the comment. Yes that's true - dataex did not provide a good sample of the data. I should have checked that before posting.

[Patrick Lang]

If I want to graph the different effects of the moderator of esg_diff_l1 for the U.S. and Europe is the three-way interaction nonetheless the right way to go? As mentioned before, the first margin command provides negative graphs for both the U.S. and Europe, which is not true.

[Clyde Schechter]

is the three-way interaction nonetheless the right way to go?

Probably. But I am a little concerned that you used the -noestimcheck- option in the -margins- command. Did you get a "(not estimable)" result without that? If so, there may be a problem. So without seeing both the regression output and the -margins- output without -noestimcheck-, I can't give a full-throated endorsement.

[Patrick Lang]

Indeed, I get a "(not estimable)" result without it. As I run a fixed effects regression, I followed your advice in another thread and used the noestimcheck option.
The first results refer to the command with the interactions tested in singulation and the second to the three-way interaction.

. xtreg ENV c.DSGR_L1##i.US c.DSGR_L1##c.esg_diff_l1 Totalassets_L1 ROA_L1 SustCommitt_L1 TMTESG_L1 GDP_L1 FDI_L1 i.Year
> , fe vce(robust)
note: 1.US omitted because of collinearity
note: DSGR_L1 omitted because of collinearity
note: 2017.Year omitted because of collinearity

Fixed-effects (within) regression               Number of obs     =      1,282
Group variable: UltimatePa~D                    Number of groups  =        303

R-sq:                                           Obs per group:
     within  = 0.1842                                         min =          1
     between = 0.0607                                         avg =        4.2
     overall = 0.0070                                         max =          7

                                                F(13,302)         =          .
corr(u_i, Xb)  = -0.0959                        Prob > F          =          .

                                (Std. Err. adjusted for 303 clusters in UltimateParentID)
-----------------------------------------------------------------------------------------
                        |               Robust
                    ENV |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
------------------------+----------------------------------------------------------------
                DSGR_L1 |  -.0643386   .0491014    -1.31   0.191    -.1609629    .0322857
                   1.US |          0  (omitted)
                        |
           US#c.DSGR_L1 |
                     1  |   .2347374   .0623407     3.77   0.000     .1120602    .3574147
                        |
                DSGR_L1 |          0  (omitted)
            esg_diff_l1 |   .0578514    .013551     4.27   0.000     .0311851    .0845177
                        |
c.DSGR_L1#c.esg_diff_l1 |  -.0005446   .0002011    -2.71   0.007    -.0009403   -.0001488
                        |
         Totalassets_L1 |  -2.36e-11   1.98e-11    -1.19   0.236    -6.26e-11    1.55e-11
                 ROA_L1 |  -.0259213   .0553429    -0.47   0.640    -.1348278    .0829852
         SustCommitt_L1 |   4.797617   2.304236     2.08   0.038     .2632264    9.332008
              TMTESG_L1 |   1.454091   1.859567     0.78   0.435    -2.205259     5.11344
                 GDP_L1 |   2.43e-13   8.06e-13     0.30   0.763    -1.34e-12    1.83e-12
                 FDI_L1 |  -.1297842   .0411172    -3.16   0.002    -.2106967   -.0488717
                        |
                   Year |
                  2012  |   .9139088    .685504     1.33   0.183    -.4350604    2.262878
                  2013  |   .2371034   .9091111     0.26   0.794    -1.551891    2.026098
                  2014  |    .930497   1.231904     0.76   0.451    -1.493705    3.354699
                  2015  |   4.035635   1.445189     2.79   0.006      1.19172    6.879551
                  2016  |   2.479129   .7742889     3.20   0.002     .9554445    4.002814
                  2017  |          0  (omitted)
                        |
                  _cons |   18.83465   8.207087     2.29   0.022     2.684336    34.98497
------------------------+----------------------------------------------------------------
                sigma_u |   28.01354
                sigma_e |  8.3715438
                    rho |  .91801665   (fraction of variance due to u_i)
-----------------------------------------------------------------------------------------

. margins US, dydx(DSGR_L1) at(esg_diff_l1 = (0 50 100 150 200 250 300))

Average marginal effects                        Number of obs     =      1,282
Model VCE    : Robust

Expression   : Linear prediction, predict()
dy/dx w.r.t. : DSGR_L1

1._at        : esg_diff_l1     =           0

2._at        : esg_diff_l1     =          50

3._at        : esg_diff_l1     =         100

4._at        : esg_diff_l1     =         150

5._at        : esg_diff_l1     =         200

6._at        : esg_diff_l1     =         250

7._at        : esg_diff_l1     =         300

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
DSGR_L1      |
      _at#US |
        1 0  |          .  (not estimable)
        1 1  |          .  (not estimable)
        2 0  |          .  (not estimable)
        2 1  |          .  (not estimable)
        3 0  |          .  (not estimable)
        3 1  |          .  (not estimable)
        4 0  |          .  (not estimable)
        4 1  |          .  (not estimable)
        5 0  |          .  (not estimable)
        5 1  |          .  (not estimable)
        6 0  |          .  (not estimable)
        6 1  |          .  (not estimable)
        7 0  |          .  (not estimable)
        7 1  |          .  (not estimable)
------------------------------------------------------------------------------

With noestimceck.

. margins US, dydx(DSGR_L1) at(esg_diff_l1 = (0 50 100 150 200 250 300)) noestimcheck

Average marginal effects                        Number of obs     =      1,282
Model VCE    : Robust

Expression   : Linear prediction, predict()
dy/dx w.r.t. : DSGR_L1

1._at        : esg_diff_l1     =           0

2._at        : esg_diff_l1     =          50

3._at        : esg_diff_l1     =         100

4._at        : esg_diff_l1     =         150

5._at        : esg_diff_l1     =         200

6._at        : esg_diff_l1     =         250

7._at        : esg_diff_l1     =         300

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
DSGR_L1      |
      _at#US |
        1 0  |  -.0643386   .0491014    -1.31   0.190    -.1605756    .0318985
        1 1  |   .1703989   .0486989     3.50   0.000     .0749508    .2658469
        2 0  |  -.0915668   .0435856    -2.10   0.036    -.1769931   -.0061406
        2 1  |   .1431706   .0473803     3.02   0.003      .050307    .2360342
        3 0  |  -.1187951   .0398837    -2.98   0.003    -.1969657   -.0406245
        3 1  |   .1159423    .048171     2.41   0.016     .0215289    .2103558
        4 0  |  -.1460234   .0385222    -3.79   0.000    -.2215254   -.0705213
        4 1  |   .0887141   .0509731     1.74   0.082    -.0111913    .1886194
        5 0  |  -.1732516   .0397424    -4.36   0.000    -.2511452    -.095358
        5 1  |   .0614858   .0554825     1.11   0.268    -.0472578    .1702295
        6 0  |  -.2004799   .0433267    -4.63   0.000    -.2853986   -.1155611
        6 1  |   .0342576   .0613238     0.56   0.576    -.0859349      .15445
        7 0  |  -.2277081   .0487565    -4.67   0.000    -.3232691   -.1321471
        7 1  |   .0070293   .0681554     0.10   0.918    -.1265528    .1406114
------------------------------------------------------------------------------

. xtreg ENV c.DSGR_L1##i.US##c.esg_diff_l1 Totalassets_L1 ROA_L1 SustCommitt_L1 TMTESG_L1 GDP_L1 FDI_L1 i.Year, fe vce(r
> obust)
note: 1.US omitted because of collinearity
note: 2017.Year omitted because of collinearity

Fixed-effects (within) regression               Number of obs     =      1,282
Group variable: UltimatePa~D                    Number of groups  =        303

R-sq:                                           Obs per group:
     within  = 0.1844                                         min =          1
     between = 0.0693                                         avg =        4.2
     overall = 0.0041                                         max =          7

                                                F(15,302)         =          .
corr(u_i, Xb)  = -0.1207                        Prob > F          =          .

                                   (Std. Err. adjusted for 303 clusters in UltimateParentID)
--------------------------------------------------------------------------------------------
                           |               Robust
                       ENV |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
---------------------------+----------------------------------------------------------------
                   DSGR_L1 |  -.1471329   .1384604    -1.06   0.289    -.4196022    .1253364
                      1.US |          0  (omitted)
                           |
              US#c.DSGR_L1 |
                        1  |   .3208385   .1474718     2.18   0.030      .030636    .6110409
                           |
               esg_diff_l1 |   .0570669   .0147148     3.88   0.000     .0281103    .0860235
                           |
   c.DSGR_L1#c.esg_diff_l1 |   .0001504   .0009695     0.16   0.877    -.0017574    .0020581
                           |
          US#c.esg_diff_l1 |
                        1  |  -.0006892   .0196213    -0.04   0.972     -.039301    .0379226
                           |
US#c.DSGR_L1#c.esg_diff_l1 |
                        1  |  -.0007278   .0010015    -0.73   0.468    -.0026985     .001243
                           |
            Totalassets_L1 |  -2.33e-11   1.99e-11    -1.17   0.243    -6.26e-11    1.59e-11
                    ROA_L1 |  -.0259096   .0558493    -0.46   0.643    -.1358127    .0839935
            SustCommitt_L1 |   4.760346   2.318514     2.05   0.041     .1978573    9.322834
                 TMTESG_L1 |   1.452299    1.86284     0.78   0.436    -2.213491    5.118088
                    GDP_L1 |   3.12e-13   1.05e-12     0.30   0.767    -1.76e-12    2.39e-12
                    FDI_L1 |  -.1305047   .0419851    -3.11   0.002    -.2131251   -.0478843
                           |
                      Year |
                     2012  |   .8910507    .699785     1.27   0.204    -.4860214    2.268123
                     2013  |   .1817947   .9896076     0.18   0.854    -1.765605    2.129194
                     2014  |   .8570884   1.372285     0.62   0.533    -1.843362    3.557539
                     2015  |   3.938322    1.70787     2.31   0.022     .5774907    7.299154
                     2016  |   2.434112   .9195943     2.65   0.009     .6244879    4.243735
                     2017  |          0  (omitted)
                           |
                     _cons |   18.23865   10.33998     1.76   0.079    -2.108884    38.58618
---------------------------+----------------------------------------------------------------
                   sigma_u |  28.148644
                   sigma_e |  8.3793555
                       rho |  .91859857   (fraction of variance due to u_i)
--------------------------------------------------------------------------------------------


. margins US, dydx(DSGR_L1) at(esg_diff_l1 = (0 50 100 150 200 250 300))

Average marginal effects                        Number of obs     =      1,282
Model VCE    : Robust

Expression   : Linear prediction, predict()
dy/dx w.r.t. : DSGR_L1

1._at        : esg_diff_l1     =           0

2._at        : esg_diff_l1     =          50

3._at        : esg_diff_l1     =         100

4._at        : esg_diff_l1     =         150

5._at        : esg_diff_l1     =         200

6._at        : esg_diff_l1     =         250

7._at        : esg_diff_l1     =         300

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
DSGR_L1      |
      _at#US |
        1 0  |          .  (not estimable)
        1 1  |          .  (not estimable)
        2 0  |          .  (not estimable)
        2 1  |          .  (not estimable)
        3 0  |          .  (not estimable)
        3 1  |          .  (not estimable)
        4 0  |          .  (not estimable)
        4 1  |          .  (not estimable)
        5 0  |          .  (not estimable)
        5 1  |          .  (not estimable)
        6 0  |          .  (not estimable)
        6 1  |          .  (not estimable)
        7 0  |          .  (not estimable)
        7 1  |          .  (not estimable)
------------------------------------------------------------------------------

With noestimceck.

. margins US, dydx(DSGR_L1) at(esg_diff_l1 = (0 50 100 150 200 250 300)) noestimcheck

Average marginal effects                        Number of obs     =      1,282
Model VCE    : Robust

Expression   : Linear prediction, predict()
dy/dx w.r.t. : DSGR_L1

1._at        : esg_diff_l1     =           0

2._at        : esg_diff_l1     =          50

3._at        : esg_diff_l1     =         100

4._at        : esg_diff_l1     =         150

5._at        : esg_diff_l1     =         200

6._at        : esg_diff_l1     =         250

7._at        : esg_diff_l1     =         300

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
DSGR_L1      |
      _at#US |
        1 0  |  -.1471329   .1384604    -1.06   0.288    -.4185103    .1242445
        1 1  |   .1737056    .049296     3.52   0.000     .0770871    .2703241
        2 0  |  -.1396144   .0926229    -1.51   0.132    -.3211521    .0419232
        2 1  |   .1448362   .0479101     3.02   0.003     .0509341    .2387382
        3 0  |   -.132096   .0518279    -2.55   0.011    -.2336768   -.0305151
        3 1  |   .1159667   .0486361     2.38   0.017     .0206418    .2112916
        4 0  |  -.1245775   .0386349    -3.22   0.001    -.2003006   -.0488545
        4 1  |   .0870973   .0513845     1.70   0.090    -.0136144     .187809
        5 0  |  -.1170591   .0707006    -1.66   0.098    -.2556297    .0215116
        5 1  |   .0582279   .0558576     1.04   0.297    -.0512511    .1677069
        6 0  |  -.1095406   .1149082    -0.95   0.340    -.3347566    .1156753
        6 1  |   .0293584   .0616814     0.48   0.634     -.091535    .1502518
        7 0  |  -.1020222   .1615816    -0.63   0.528    -.4187163    .2146719
        7 1  |    .000489   .0685123     0.01   0.994    -.1337926    .1347706
------------------------------------------------------------------------------

[Clyde Schechter]

OK, this may be a problem. I am aware that in previous posts I have been fairly liberal in advising the use of the -noestimcheck- option with -margins, dydx()-. More recently, I have come to realize that in many circumstances this advice is incorrect. And yours is one of those circumstances. The key problem here is that you have received a message

note: 2017.Year omitted because of collinearity

What this tells us is that there is some variable in your model that, in effect, identifies some subset of the years. It might be a variable that indicates recession years, or the financial crisis, or something like that. I can't discern from the names of your variables what they are, so I can't advise you which one it might be. But the key thing is that it is constant within years. This variable is necessarily colinear with the year fixed effects. As a result, the effects of that variable, as well as the year fixed effects themselves, are all unidentifiable--it is to identify the model that Stata chooses to eliminate one of them. Now, if that group of variables is independent of everything else, then you could just ignore the problem because everything else would be unaffected. The use of -noestimcheck- would be appropriate in that case. But that is rarely the case. It is likely that there is at least some correlation between these variables and some or all of the other variables in your model. In that case, the estimates of those variables' effects may also be unidentifiable. That is why you are getting the "not estimable" response.

So you need to figure out which variable is colinear with the year fixed effects, and remove it from the model. (Or, if that variable is important to your research goals, drop the year indicators instead. You can have one or the other, but not both.) Re-run the analyses, and then -margins- will estimate these marginal effects without complaint, and give you correct results. The results you have now cannot be trusted.

[Patrick Lang]

Indeed, the variable esg_diff_l1 indicates the diffusion of a novel practice among peers. Thus, the variable is the same for every organization in a given year x (e.g. 100 in year 2010 121 in 2011 etc.). That is the reason why the effect of year 2017 is dropped. I assumed I was fine, as esg_diff_l1 is part of an interaction term with my main variable DSGR_L1 and the interaction term is not collinear with the year-fixed effects. Moreover, a test for year-fixed effects indicates that controlling for these effects is necessary. Hence, I assume that I will run into endogeneity issues if I do not control for them.

[Clyde Schechter]

Well, if the diffusion actually changes every year, then the diffusion variable itself adjusts for time effects. Of course, that still leaves the concern that you are unable to distinguish whether the diffusion itself is acting causally or is just a proxy for a secular trend. You're working in a domain where I have no expertise, so I really can't advise you what's best to do here.

[Patrick Lang]

Ok. Let me ask two further questions. First, in #8 you stated that all the year-fixed effects are unidentifiable. So it is not just that the year 2017 is dropped from the calculation but all year-fixed effects? For ensuring robustness, I might run the model with and without the year-fixed effects. Second, besides the issue of collinearity, the margins command with the three-way interaction would provide the results for highlighting the differences between the U.S. and Europe as mentioned in #6?

[Clyde Schechter]

First, in #8 you stated that all the year-fixed effects are unidentifiable. So it is not just that the year 2017 is dropped from the calculation but all year-fixed effects?

Right. The colinear relationships are diffusion = diffusion(2011) * 2011.year + diffusion(2012) * 2012.year + ... + diffusion(2017) * 2017.year, and the usual dummy-variable colinearity equation _cons = 2011.year + 2012.year + ... + 2017.year. So the rank of the matrix involving these terms is two less than the number of variables involved. This model can be identified then by imposing two linear constraints on these variables. The omission of the base 2011.year, as always with dummy variables, takes care of one. And the omission of another year is the second. Now the model is identifiable. BUT the identified coefficients of anything involving these variables is now an artifact of the way the identification was accomplished. Had Stata omitted, say, _cons and 2013 instead, you would get different coefficients. Importantly, the model's predicted values would be the same (except possibly for different tiny rounding errors during calculations) regardless of which 2 constraints are used to identify the model. But the coefficients would differ--and would differ precisely in ways that are guaranteed to produce the same predicted values in the end. So the point is that the marginal effects you want are unidentifiable, because they are functions of the coefficients that, in turn, are sensitive to the particular way in which the colinearity is dealt with. (By contrast, something like _b[2013.year] - _b[2015.year] would give you the same result regardless of which way the colinearity is dealt with.)

besides the issue of collinearity, the margins command with the three-way interaction would provide the results for highlighting the differences between the U.S. and Europe as mentioned in #6?

It's hard to say. You can test this by running the model again and forcing Stata to break the colinearity differently. For example, you could use the noconstant option, plus an omission of some year other than 2017 (e.g. maybe io2015.year). If you get the same results for your US related marginal effects, then you can rely on them. But if they come out different, then they will have been affected by the colinearity.

[Patrick Lang]

Thanks Clyde. Really appreciate your help.