Testing Difference between Moderation Effects across two Ebalance-Weighted Regressions

[Samuel Jalalian Asl]

Hello everyone,

I'm estimating the wage penalty from unemployment (wage scarring) in Stata 18, examining whether it is moderated by household wealth (quartiles). Specifically, I want to test whether this moderation differs between short-term and long-term unemployment.

To this end, I estimate two separate entropy-balanced regressions using the "ebalance" command (https://doi.org/10.18637/jss.v054.i07):

// short_unemp = 1 if individual experienced short-term unemployment, 0 = no unemployment
// long_unemp = 1 if individual experienced long-term unemployment, 0 = no unemployment
// l4wealth_q_i = quartiles of household wealth (1 = lowest, 4 = highest)

// Generate entropy balancing weight (short unemployment)
ebalance short_unemp ///
    i.l4wealth_q_i##i.l4indlabinc_h_q ///
    i.l4wealth_q_i##c.l4incshare ///
    i.l4wealth_q_i##c.l4expue ///
    i.l4wealth_q_i##i.l4mpairs ///
    i.l4wealth_q_i##c.l4whours ///
    i.l4wealth_q_i##c.l4empdur ///
    i.l4wealth_q_i##c.l4age ///
    i.l4wealth_q_i##i.l4men ///
    i.l4wealth_q_i##i.l4race ///
    i.l4wealth_q_i##i.l4edu ///
    i.l4wealth_q_i##i.l4kids ///
    i.l4wealth_q_i##i.l4sector3 ///
    i.l4wealth_q_i##i.l4d_east ///
    , targets(2) generate(entropy_balancing_weight_short)

// Generate entropy balancing weight (long unemployment)
ebalance long_unemp ///
    i.l4wealth_q_i##i.l4indlabinc_h_q ///
    i.l4wealth_q_i##c.l4incshare ///
    i.l4wealth_q_i##c.l4expue ///
    i.l4wealth_q_i##i.l4mpairs ///
    i.l4wealth_q_i##c.l4whours ///
    i.l4wealth_q_i##c.l4empdur ///
    i.l4wealth_q_i##c.l4age ///
    i.l4wealth_q_i##i.l4men ///
    i.l4wealth_q_i##i.l4race ///
    i.l4wealth_q_i##i.l4edu ///
    i.l4wealth_q_i##i.l4kids ///
    i.l4wealth_q_i##i.l4sector3 ///
    i.l4wealth_q_i##i.l4d_east ///
    , targets(2) generate(entropy_balancing_weight_long)

Then I run two weighted regressions with treatment × wealth interactions:

// Regression: Short Unemployment
reg s4lnindlab_hourly ///
    i.short_unemp##i.l4wealth_q_i /// // <- interaction effect of interest
    i.short_unemp##c.l4incshare ///
    i.short_unemp##i.l4indlabinc_h_q ///
    i.short_unemp##c.l4whours  ///
    i.short_unemp##i.l4mpairs ///
    i.short_unemp##c.l4ueexp ///
    i.short_unemp##c.l4empdur ///
    i.short_unemp##c.l4age ///
    i.short_unemp##i.l4men  ///
    i.short_unemp##i.l4race ///
    i.short_unemp##i.l4edu ///
    i.short_unemp##i.l4sector3 ///
    i.short_unemp##i.l4d_east ///
    i.wave_cat ///
    s4hhsize ///
    [pweight=entropy_balancing_weight_short], vce(cl persnr)
estimates store Short_Unemployment

// Regression: Long Unemployment
reg s4lnindlab_hourly ///
    i.long_unemp##i.l4wealth_q_i /// // <- interaction effect of interest
    i.long_unemp##c.l4incshare ///
    i.long_unemp##i.l4indlabinc_h_q ///
    i.long_unemp##c.l4whours ///
    i.long_unemp##i.l4mpairs ///
    i.long_unemp##c.l4ueexp ///
    i.long_unemp##c.l4empdur ///
    i.long_unemp##c.l4age ///
    i.long_unemp##i.l4men ///
    i.long_unemp##i.l4race ///
    i.long_unemp##i.l4edu ///
    i.long_unemp##i.l4sector3 ///
    i.long_unemp##i.l4d_east ///
    i.wave_cat ///
    s4hhsize ///
    [pweight=entropy_balancing_weight_long], vce(cl persnr)
estimates store Long_Unemployment

My Goal
I would like to test the following cross-model difference in differences:
(Effect of short_unemp in wealth Q1 − Effect of short_unemp in wealth Q4) − (Effect of long_unemp in wealth Q1 − Effect of long_unemp in wealth Q4)

In words: does the wealth gap in wage scarring differ between short and long unemployment spells?

Ideally, I would like to test it like this:

(1.short_unemp#1.l4wealth_q_i - 1.short_unemp#4.l4wealth_q_i) - ///  
(1.long_unemp#1.l4wealth_q_i - 1.long_unemp#4.l4wealth_q_i)

My Problem/Question
I'm unsure how to do this in Stata. Is there a way in Stata 18 to test whether these interaction effects differ significantly across the two separately estimated and differently weighted regressions?


Below I describe what I’ve tried so far.

1. lincom
Did not work here, since the estimates come from two separate models.
In other settings (e.g. gender differences), I have successfully used lincom within one model (see below):

ebalance ///
    all_unemp ///
    i.l4men##i.l4wealth_q_i##i.l4indlabinc_h_q ///
    i.l4men##i.l4wealth_q_i##c.l4incshare ///
    i.l4men##i.l4wealth_q_i##c.l4expue ///
    i.l4men##i.l4wealth_q_i##i.l4mpairs ///
    i.l4men##i.l4wealth_q_i##c.l4whours ///
    i.l4men##i.l4wealth_q_i##c.l4empdur ///
    i.l4men##i.l4wealth_q_i##c.l4age ///
    i.l4men##i.l4wealth_q_i##i.l4race ///
    i.l4men##i.l4wealth_q_i##i.l4edu ///
    i.l4men##i.l4wealth_q_i##i.l4kids ///
    i.l4men##i.l4wealth_q_i##i.l4sector3 ///
    i.l4men##i.l4wealth_q_i##i.l4d_east ///
    , targets(2) generate(entropy_balancing_weight_gender)

reg s4lnindlab_hourly ///
    i.l4men##i.all_unemp##i.l4wealth_q_i ///
    i.l4men##i.all_unemp##c.l4incshare ///
    i.l4men##i.all_unemp##i.l4indlabinc_h_q ///
    i.l4men##i.all_unemp##c.l4whours ///
    i.l4men##i.all_unemp##i.l4mpairs ///
    i.l4men##i.all_unemp##c.l4ueexp ///
    i.l4men##i.all_unemp##c.l4empdur ///
    i.l4men##i.all_unemp##c.l4age ///
    i.l4men##i.all_unemp##i.l4race ///
    i.l4men##i.all_unemp##i.l4edu ///
    i.l4men##i.all_unemp##i.l4sector3 ///
    i.l4men##i.all_unemp##i.l4d_east ///
    i.l4men##i.wave_cat ///
    i.l4men##c.s4hhsize ///
    [pweight=entropy_balancing_weight_gender] ///
    , vce(cl persnr)
    
// Hypothesis Test (WealthQ1men - WealthQ4men) - (WealthQ1women - WealthQ4women)
lincom ///
(1.all_unemp#1.l4wealth_q_i#1.l4men - 1.all_unemp#4.l4wealth_q_i#1.l4men) - ///   // (Q1men - Q4men)
(1.all_unemp#1.l4wealth_q_i#0.l4men - 1.all_unemp#4.l4wealth_q_i#0.l4men)         // (Q1women - Q4women)

2. Suest
Combining the models postestimation using suest failed due to different pweights (from ebalance).

3. A combined model to test my hypothesis was not feasible either, as short_unemp and long_unemp are mutually exclusive by design.


I’d greatly appreciate any suggestions on how to implement this test in Stata.


Thanks in advance!

Best,
Samuel

[George Ford]

clear all

sysuse auto, clear

eststo e1: reg mpg weight [aw=length]
eststo e2: reg price foreign [aw=turn]
suest e1 e2 // will fail

g cmpg = 1
foreach var in cmpg mpg weight {
    g `var'_w = `var'*sqrt(length)
}

g cprice = 1
foreach var in cprice price foreign {
    g `var'_w = `var'*sqrt(turn)
}

eststo w1: reg mpg_w weight_w cmpg_w, nocons
eststo w2: reg price_w foreign_w cprice_w , nocons
suest w1 w2

** check for identical
esttab e1 w1 e2 w2

[Samuel Jalalian Asl]

clear all

sysuse auto, clear

eststo e1: reg mpg weight [aw=length]
eststo e2: reg price foreign [aw=turn]
suest e1 e2 // will fail

g cmpg = 1
foreach var in cmpg mpg weight {
g `var'_w = `var'*sqrt(length)
}

g cprice = 1
foreach var in cprice price foreign {
g `var'_w = `var'*sqrt(turn)
}

eststo w1: reg mpg_w weight_w cmpg_w, nocons
eststo w2: reg price_w foreign_w cprice_w , nocons
suest w1 w2

** check for identical
esttab e1 w1 e2 w2

Hi George,

thanks for your suggestion! The sqrt(weight) approach worked well and allowed me to use suest for the comparison I had in mind. That was very helpful. I'm now thinking through how to handle categorical variables like wealth quintiles in this transformed setup. After the sqrt(weight) transformation, the wealth variable no longer reflects quartile membership but instead takes on values between 0 and 1, which means I can’t directly test differences between Q1 and Q4. Unfortunately, leaving the wealth variable unweighted isn’t really an option in our setup. If you have any ideas on how to handle this, I’d appreciate any thoughts.

Best,
Samuel

[Felix Bittmann]

As long as you can produce a point estimate, bootstrapping can always get you a confidence interval. However, given the setup, one should think about clustering and strata selection before conducting this.

[George Ford]

convert categoricals to dummies and treat as such.

clear all

sysuse auto, clear
drop if mi(rep78)

eststo e1: reg mpg weight i.rep78 [aw=length]
eststo e2: reg price foreign i.rep78 [aw=turn]

g cmpg = 1
foreach var in cmpg mpg weight {
g `var'_w = `var'*sqrt(length)
}

g cprice = 1
foreach var in cprice price foreign {
g `var'_w = `var'*sqrt(turn)
}

tab rep78 , generate(rep_)
foreach var in rep_1 rep_2 rep_3 rep_4 rep_5 {
g `var'_wl = `var'*sqrt(length)
g `var'_wt = `var'*sqrt(turn)
}

eststo w1: reg mpg_w weight_w rep_2_wl rep_3_wl rep_4_wl rep_5_wl cmpg_w, nocons
eststo w2: reg price_w foreign_w rep_2_wt rep_3_wt rep_4_wt rep_5_wt cprice_w , nocons
suest w1 w2

** check for identical
esttab e1 w1 e2 w2

[Samuel Jalalian Asl]

convert categoricals to dummies and treat as such.

clear all

sysuse auto, clear
drop if mi(rep78)

eststo e1: reg mpg weight i.rep78 [aw=length]
eststo e2: reg price foreign i.rep78 [aw=turn]

g cmpg = 1
foreach var in cmpg mpg weight {
g `var'_w = `var'*sqrt(length)
}

g cprice = 1
foreach var in cprice price foreign {
g `var'_w = `var'*sqrt(turn)
}

tab rep78 , generate(rep_)
foreach var in rep_1 rep_2 rep_3 rep_4 rep_5 {
g `var'_wl = `var'*sqrt(length)
g `var'_wt = `var'*sqrt(turn)
}

eststo w1: reg mpg_w weight_w rep_2_wl rep_3_wl rep_4_wl rep_5_wl cmpg_w, nocons
eststo w2: reg price_w foreign_w rep_2_wt rep_3_wt rep_4_wt rep_5_wt cprice_w , nocons
suest w1 w2

** check for identical
esttab e1 w1 e2 w2

Thank you, that was very helpful.