Interpreting reghdfe_psacalc outcome

[Catherine Acosta]

I recently learned that Oster’s (2019) diagnostic framework can be used to assess the trade-off between coefficient magnitude and explained variation across different fixed-effects specifications. I estimated two models with different FE structures and would like to compare them.

Could someone help me interpret the Oster diagnostic output below, or point me to relevant guidance, specifically with the Beta and Alt. sol 1? Based on these results, does the constrained model appear to be the more appropriate specification?

Thank you in advance for your help.

Treatment Effect Estimate ----
| Estimate Sq. difference Bias changes
| from controlled beta direction
-------------+----------------------------------------------------------------
Beta | -2.3370e+05 5.46e+10
Alt. sol. 1 | 9.40977 0.179
Alt. sol. 2 |
-------------+----------------------------------------------------------------

---- Inputs from Regressions ----
| Coeff. R-Squared
-------------+----------------------------------------------------------------
Constrained | 9.22048 0.160
Full | 8.98635 0.394
-------------+----------------------------------------------------------------

---- Other Inputs ----
-------------+----------------------------------------------------------------
R_max | 1.000
Delta | 1.000
Models | e1 e2
-------------+----------------------------------------------------------------

[George Ford]

That range on Beta is huge--the results are extremely sensitive to unobservables. Not good.

If you're just interested in which FE to use, Papke and Wooldrige have a test.

I coded this to run it. It should work.

capture program drop papwool
program papwool , rclass
    syntax [, altfe(varlist min=1 max=1) time(varlist min=1 max=1) datatype(string)] 
    qui estimates store reghdfemodel
    preserve
    if "`datatype'" == "" {
        di "You must specify data type (panel or pooled)."
        exit
    }
    quietly {
    macro drop TTERMS MTERMS XTERMS
    capture drop *_xxx
    ** GET TIME VARIABLES FOR POOLED DATA
    egen tobs_xxx = count(`altfe') , by(`altfe')
    global TTERMS 
    if "`datatype'" == "pooled" {
        summ year
        *local start_xxx = r(min)-1
        forv y = `r(min)'/`r(max)' {
            egen y`y'_xxx = total(year==`y'), by(schid)
            replace y`y'_xxx = y`y'_xxx/tobs_xxx
            global TTERMS $TTERMS y`y'_xxx 
        }
    }
    global XTERMS 
    global MTERMS 
    local varnum =  wordcount(`"`e(indepvars)'"')-1 //-1 excludes constant
    forv i = 1/`varnum' {
        local var = `'"`: word `i' of `e(indepvars)''"'
        global XTERMS $XTERMS `var'
        egen `var'_xxx = mean(`var') , by(`altfe')
        global MTERMS $MTERMS `var'_xxx 
    }
    reghdfe `e(depvar)' $XTERMS $MTERMS $TTERMS , absorb(`e(absvars)') cluster(`e(clustvar)')
    qui estimates store reghdfemodel_aug
    test $MTERMS $TTERMS
    }
    local star = (cond(r(p)<.01, "***", cond(r(p)<.05, "**", cond(r(p)<.10, "*", ""))))
    di `"{hline 50}"'
    di "PAPKE-WOOLDRIDGE FIXED EFFECT TEST"
    di `"{hline 50}"'
    di "Main model stored as reghdfemodel"
    di "Augmented model stored as reghdfemodel_aug"
    di `"{hline 50}"'
    di "Null:  Higher level fixed effects is OK."
    di "F-stat = " %6.2f `r(F)' "`star'" 
    di "Prob Level = " %6.4f r(p)
    if r(p) < 0.10 {
        di "Null is REJECTED."
    }
    else {
        di "Null is ACCEPTED"
    }
    di `"{hline 50}"'
    restore
    qui estimates restore reghdfemodel
end

** RUN MODEL AND DO TEST
reghdfe (blah blah blah)

papwool , altfe(alternatefe) time(year) datatype(pooled)
papwool , altfe(alternatefe) time(year) datatype(panel)

[Catherine Acosta]

Many thanks, George ! I will run it.