Wald test comparing coefficients across two regressions with different N (different dependent variables)

[hussein bataineh]

Hi all,
I have panel data (firm id = id) and I’m running two OLS regressions with the same RHS variables but different dependent variables, so the estimation samples differ because each dependent variable has different missing values.

reg z_NET  HARD_FREEZE CSO_PRESENCE HARD_FREEZEXCSO_PRESENCE ///
    SIZE ROA LEV MB OCF SD_OCF ///
    BOARD_SIZE GENDER_RATIO BOARD_IND SUS_COMM FUND_STATUS FUND_RATIO ///
    PLAN_SIZE i.year i.ff_12, vce(cluster id)

reg z_DISC HARD_FREEZE CSO_PRESENCE HARD_FREEZEXCSO_PRESENCE ///
    SIZE ROA LEV MB OCF SD_OCF ///
    BOARD_SIZE GENDER_RATIO BOARD_IND SUS_SCORE SUS_COMM FUND_STATUS FUND_RATIO ///
    PLAN_SIZE i.year i.ff_12, vce(cluster id)

I want to run a Wald test to compare coefficients across the two models (e.g., test whether the coefficient on HARD_FREEZE and/or HARD_FREEZEXCSO_PRESENCE is the same in both regressions), without forcing both regressions to use the same number of observations (i.e., without restricting to the intersection sample).

What is the correct way to do this in Stata with clustered SEs?

Thanks!

[Clyde Schechter]

-help suest-

[hussein bataineh]

Hi Clyde, thanks. I’m using suest to test equality of coefficients across two OLS models estimated on different samples (different dependent variables). It works once I estimate both models without vce(cluster) and then do suest ..., vce(cluster id).

However, because the samples differ, Stata sometimes chooses different omitted categories for i.year (and i.ff_12), which causes suest to fail with “factor variable base category conflict” unless I force a common base (e.g., ib2010.year ib3.ff_12 or fvset base 2010 year and fvset base 3 ff_12).

Is there any way to use suest without explicitly forcing common base categories, or is setting a base (via fvset or ib#.) the correct/necessary approach when samples differ?

Also, when suest runs it reports “Number of obs = 3,792” (the union of samples). Is it fine to report my two regression Ns separately (3,087 and 3,210) and only report the suest Wald test p-values?

[Clyde Schechter]

Yes, you will need to force a common base level for factor variables. Even apart from issues of different samples, it is meaningless to do a comparison of coefficients of factor variables if they do not have the same base level.

It is true that -suest- reports an N equal to the size of the intersection (not union) of the samples. But the documentation of -suest- is explicit that there is no requirement for a common sample. Indeed, -suest- will work even if the samples don't overlap at all.

[hussein bataineh]

Hi Clyde, thanks again.

I implemented suest with different dependent variables (different samples) and clustered SEs. Could you please review whether the workflow below is correct and whether there is a cleaner/better way?

. 
. *------------------------------------------------------------
. * 1) Standardise NET_CSR and CSR_DISC by industry (ff_12)
. *------------------------------------------------------------
. bysort ff_12: egen z_NET  = std(NET_CSR)
(2,919 missing values generated)

. bysort ff_12: egen z_DISC = std(CSR_DISC)
(2,027 missing values generated)

. 
. label var z_NET  "Std CSR engagement (NET_CSR), by industry"

. label var z_DISC "Std CSR disclosure (CSR_DISC), by industry"

. 
. *------------------------------------------------------------
. * 2) Force common base levels for factor variables (must exist in BOTH samples)
. *    (these worked for you before; change if needed)
. *------------------------------------------------------------
. fvset base 2010 year

. fvset base 3 ff_12

. 
. *------------------------------------------------------------
. * 3) MAIN regressions (clustered) – these are the ones you report
. *------------------------------------------------------------
. reg z_NET  HARD_FREEZE CSO_PRESENCE HARD_FREEZEXCSO_PRESENCE ///
>     SIZE ROA LEV MB OCF SD_OCF ///
>     BOARD_SIZE GENDER_RATIO BOARD_IND SUS_COMM FUND_STATUS FUND_RATIO ///
>     PLAN_SIZE i.year i.ff_12, vce(cluster id)

Linear regression                               Number of obs     =      3,087
F(42, 266)        =      20.38
Prob > F          =     0.0000
R-squared         =     0.2915
Root MSE          =     .93566

(Std. err. adjusted for 267 clusters in id)

Robust
z_NET  Coefficient  std. err.      t    P>t     [95% conf. interval]

HARD_FREEZE    .5426068   .1457661     3.72   0.000     .2556046     .829609
CSO_PRESENCE    .2299951    .088679     2.59   0.010     .0553931    .4045972
HARD_FREEZEXCSO_PRESENCE    .7703552   .3167733     2.43   0.016     .1466531    1.394057
SIZE    .0251656   .0587456     0.43   0.669    -.0904999     .140831
ROA   -3.334543   .7853347    -4.25   0.000    -4.880806    -1.78828
LEV    .2746622   .2981996     0.92   0.358    -.3124697    .8617941
MB    .0036703   .0019434     1.89   0.060    -.0001561    .0074966
OCF    4.883632   .9234886     5.29   0.000     3.065355     6.70191
SD_OCF    2.155443   2.055104     1.05   0.295    -1.890897    6.201782
BOARD_SIZE    .0554057   .0173643     3.19   0.002     .0212168    .0895946
GENDER_RATIO    3.199932   .3537734     9.05   0.000     2.503379    3.896484
BOARD_IND    .1059477   .3670269     0.29   0.773    -.6166997    .8285951
SUS_COMM    .0349891   .0894132     0.39   0.696    -.1410586    .2110368
FUND_STATUS   -.4132233   1.347302    -0.31   0.759    -3.065956    2.239509
FUND_RATIO    .2239457   .2646523     0.85   0.398     -.297134    .7450255
PLAN_SIZE    .0519574     .04598     1.13   0.259    -.0385737    .1424885

year 
2004     -.880395   .1007765    -8.74   0.000    -1.078816   -.6819739
2005    -.7889862   .0942176    -8.37   0.000    -.9744933    -.603479
2006    -.7925856   .0954634    -8.30   0.000    -.9805456   -.6046256
2007    -.7697657   .0994902    -7.74   0.000    -.9656542   -.5738771
2008     -.786764    .086503    -9.10   0.000    -.9570817   -.6164463
2009    -.9026355   .0826172   -10.93   0.000    -1.065302   -.7399686
2011     .2577559   .0604707     4.26   0.000     .1386937    .3768181
2012    -.3897956    .089273    -4.37   0.000    -.5655671   -.2140241
2013    -.3830442   .0982973    -3.90   0.000     -.576584   -.1895045
2014    -.8998882   .1043255    -8.63   0.000    -1.105297   -.6944793
2015    -.6556112   .1023801    -6.40   0.000    -.8571896   -.4540328
2016    -.5371076    .105369    -5.10   0.000    -.7445709   -.3296442
2017    -.5714094   .1101457    -5.19   0.000    -.7882778    -.354541
2018    -.5765738   .1171856    -4.92   0.000    -.8073031   -.3458444
2019    -.6751663    .120548    -5.60   0.000    -.9125159   -.4378168

ff_12 
1    -.2134401   .1682669    -1.27   0.206    -.5447446    .1178644
2    -.0423566   .2524626    -0.17   0.867    -.5394359    .4547227
4    -.3000848   .2022633    -1.48   0.139    -.6983255    .0981559
5    -.1481941   .1715746    -0.86   0.389    -.4860111    .1896229
6     .2371027   .2078807     1.14   0.255    -.1721983    .6464037
7    -.1918903   .3449501    -0.56   0.578    -.8710702    .4872895
8    -.1880669   .1492469    -1.26   0.209    -.4819225    .1057887
9     .1656019   .2607637     0.64   0.526    -.3478215    .6790254
10     .1189463   .1892235     0.63   0.530    -.2536201    .4915126
11     .1663465   .2155413     0.77   0.441    -.2580376    .5907306
12    -.0407918   .1570621    -0.26   0.795    -.3500348    .2684513

_cons   -1.837741   .5078447    -3.62   0.000    -2.837648   -.8378347


. est store net_cl

. 
. reg z_DISC HARD_FREEZE CSO_PRESENCE HARD_FREEZEXCSO_PRESENCE ///
>     SIZE ROA LEV MB OCF SD_OCF ///
>     BOARD_SIZE GENDER_RATIO BOARD_IND SUS_SCORE SUS_COMM FUND_STATUS FUND_RATIO ///
>     PLAN_SIZE i.year i.ff_12, vce(cluster id)

Linear regression                               Number of obs     =      3,210
F(45, 265)        =      82.25
Prob > F          =     0.0000
R-squared         =     0.7192
Root MSE          =     .53193

(Std. err. adjusted for 266 clusters in id)

Robust
z_DISC  Coefficient  std. err.      t    P>t     [95% conf. interval]

HARD_FREEZE    .3856676   .1321721     2.92   0.004     .1254265    .6459086
CSO_PRESENCE    .2992607   .0576615     5.19   0.000     .1857277    .4127937
HARD_FREEZEXCSO_PRESENCE    .7515366   .2439528     3.08   0.002     .2712041    1.231869
SIZE   -.0099417   .0335382    -0.30   0.767    -.0759769    .0560934
ROA   -.9737734   .5112759    -1.90   0.058    -1.980453    .0329066
LEV    .1786056   .1882451     0.95   0.344    -.1920408     .549252
MB   -.0038137   .0015945    -2.39   0.017    -.0069532   -.0006742
OCF     2.05759   .6190596     3.32   0.001     .8386887    3.276491
SD_OCF    .4459352   1.098708     0.41   0.685    -1.717373    2.609243
BOARD_SIZE    .0134128   .0105903     1.27   0.206     -.007439    .0342646
GENDER_RATIO    .0343031   .2314186     0.15   0.882    -.4213499    .4899562
BOARD_IND   -.0842603   .2533657    -0.33   0.740    -.5831262    .4146056
SUS_SCORE    .0243122   .0015287    15.90   0.000     .0213023    .0273222
SUS_COMM    .1131449    .052742     2.15   0.033     .0092981    .2169917
FUND_STATUS   -.1376151   .7326321    -0.19   0.851    -1.580136    1.304906
FUND_RATIO   -.0753848   .1442979    -0.52   0.602    -.3595009    .2087314
PLAN_SIZE    .0219252    .024508     0.89   0.372    -.0263301    .0701804

year 
2005    -.1565781    .089674    -1.75   0.082    -.3331424    .0199862
2006    -.0978593     .06753    -1.45   0.148    -.2308228    .0351043
2007    -.0484807   .0468494    -1.03   0.302    -.1407252    .0437638
2008    -.0863536   .0362316    -2.38   0.018     -.157692   -.0150152
2009    -.0669788    .028809    -2.32   0.021    -.1237025   -.0102552
2011     .0475926   .0299413     1.59   0.113    -.0113605    .1065457
2012       .13099    .042821     3.06   0.002     .0466774    .2153026
2013     .2117517   .0384343     5.51   0.000     .1360761    .2874272
2014     .3179515   .0442658     7.18   0.000     .2307941    .4051089
2015     .5045084   .0476626    10.58   0.000     .4106628     .598354
2016     .5658173   .0545267    10.38   0.000     .4584567    .6731779
2017     .6899284   .0569147    12.12   0.000     .5778658    .8019911
2018     .6806051   .0628033    10.84   0.000     .5569481     .804262
2019     .7726055   .0674565    11.45   0.000     .6397867    .9054244
2020     .8372989   .0714437    11.72   0.000     .6966293    .9779685
2021     .8788475   .0769747    11.42   0.000     .7272876    1.030407
2022     .8982151   .0787441    11.41   0.000     .7431714    1.053259

ff_12 
1    -.2210299   .1222036    -1.81   0.072    -.4616436    .0195837
2    -.2448478   .1203666    -2.03   0.043    -.4818444   -.0078512
4     .0707387   .1457247     0.49   0.628    -.2161869    .3576644
5    -.2175241    .129159    -1.68   0.093    -.4718324    .0367843
6     .2667938   .1045028     2.55   0.011     .0610323    .4725553
7     .4980767   .2187991     2.28   0.024     .0672709    .9288825
8     .0367421    .093557     0.39   0.695    -.1474676    .2209519
9     .0109226   .1216638     0.09   0.929    -.2286282    .2504733
10      .264161   .1194483     2.21   0.028     .0289725    .4993495
11     .1493139   .1149137     1.30   0.195    -.0769461    .3755739
12      .372683   .1101983     3.38   0.001     .1557075    .5896586

_cons   -2.163606    .340833    -6.35   0.000    -2.834691   -1.492521


. est store disc_cl

. 
. 
. *------------------------------------------------------------
. * 4) suest step (for cross-model Wald tests)
. *    IMPORTANT: re-estimate WITHOUT vce() and then cluster in suest
. *------------------------------------------------------------
. quietly reg z_NET  HARD_FREEZE CSO_PRESENCE HARD_FREEZEXCSO_PRESENCE ///
>     SIZE ROA LEV MB OCF SD_OCF ///
>     BOARD_SIZE GENDER_RATIO BOARD_IND SUS_COMM FUND_STATUS FUND_RATIO ///
>     PLAN_SIZE i.year i.ff_12

. est store m_net

. 
. quietly reg z_DISC HARD_FREEZE CSO_PRESENCE HARD_FREEZEXCSO_PRESENCE ///
>     SIZE ROA LEV MB OCF SD_OCF ///
>     BOARD_SIZE GENDER_RATIO BOARD_IND SUS_SCORE SUS_COMM FUND_STATUS FUND_RATIO ///
>     PLAN_SIZE i.year i.ff_12

. est store m_disc

. 
. quietly suest m_net m_disc, vce(cluster id)

. 
. *------------------------------------------------------------
. * 5) Wald tests: equality of coefficients across the two models
. *------------------------------------------------------------
. test [m_net_mean]HARD_FREEZE = [m_disc_mean]HARD_FREEZE

( 1)  [m_net_mean]HARD_FREEZE - [m_disc_mean]HARD_FREEZE = 0

chi2(  1) =    0.52
Prob > chi2 =    0.4729

. test [m_net_mean]CSO_PRESENCE = [m_disc_mean]CSO_PRESENCE

( 1)  [m_net_mean]CSO_PRESENCE - [m_disc_mean]CSO_PRESENCE = 0

chi2(  1) =    0.43
Prob > chi2 =    0.5137

. test [m_net_mean]HARD_FREEZEXCSO_PRESENCE = [m_disc_mean]HARD_FREEZEXCSO_PRESENCE

( 1)  [m_net_mean]HARD_FREEZEXCSO_PRESENCE - [m_disc_mean]HARD_FREEZEXCSO_PRESENCE = 0

chi2(  1) =    0.00
Prob > chi2 =    0.9533

. 
. 
end of do-file

[Clyde Schechter]

This looks basically correct to me. The only thing I think you should do differently is that I would not run the regressions for -suest- and -suest- itself -quietly-. You really ought to get the output of those and inspect them carefully to make sure everything went according to plan. The output might contain important information, like warnings about omitted observations or technical difficulties with estimation, or even results that are implausible, any of which could cast doubt on the validity of the final results. So I would re-run this without the -quietly-'s just be sure that no problems are being swept under the rug.


But assuming that what comes out of doing that raises no alarms, it looks to me like you have done it correctly.

Disclaimer: From the variable names I'm inferring that this work is in the domain of finance. I have no expertise whatsoever in that area, and I cannot and do not offer any opinion about whether doing these analyses makes sense nor is consistent with broader principles of finance. I am commenting here only on the coding as it pertains to the purposes set out by O.P.

[hussein bataineh]

Hi Clyde, thanks again — I followed your suggested workflow and suest runs fine with clustered SEs. I just want to be 100% sure I’m interpreting the cross-model Wald tests correctly given the different estimation samples.

Workflow (summary):

I run the two main regressions with clustered SEs and report them:

• reg z_NET ... , vce(cluster id) → N = 3,087
• reg z_DISC ... , vce(cluster id) → N = 3,210

Then for suest, I re-estimate both models without vce(cluster ...), store them, and run:

• suest m_net m_disc, vce(cluster id)

Then I test equality of coefficients across equations:

• test [m_net_mean]HARD_FREEZE = [m_disc_mean]HARD_FREEZE
• test [m_net_mean]CSO_PRESENCE = [m_disc_mean]CSO_PRESENCE
• test [m_net_mean]HARD_FREEZEXCSO_PRESENCE = [m_disc_mean]HARD_FREEZEXCSO_PRESENCE

Stata reports chi2(1) statistics and p-values.

My questions:

1. Do these results represent a valid Wald (χ²) test of cross-equation coefficient equality even though the two regressions are estimated on different samples (N=3,087 vs N=3,210)?
2. When writing up results, is it correct to report the two regression Ns (3,087 and 3,210) for the main models, and then report the suest Wald χ² p-values for the coefficient-comparison tests (without forcing a common sample)?
3. Since suest, vce(cluster id) produces χ², should I describe these as Wald chi-square tests (not F-tests)? And do I only report the regression F-statistics for the individual model fits (not for the cross-equation equality tests)?

[Clyde Schechter]

Questions 1 & 2: Yes.

Question 3. Yes, these are Wald chi-square tests. I don't understand the rest of question 3.