Coefplot p-values when there are more than one p-value per coefficient

[Adrij Chakraborty]

I run rdrobust, and then plot the estimates using the coefplot command. I plug in options that allow the significance stars to be shown on top of the point estimates. Now rdrobust has three p-values (conventional, bias-corrected and robust), but I can only see the significance stars in the figure based on the first (conventional) p-value. This is the code I use:

local outcomes Y1 Y3aplus Y3bplus
local subgroups "if imp_m==0" "if UR_reg==1 & imp_m==0" "if UR_reg==0 & imp_m==0" "if imp_m==0 & poor==1" "if imp_m==0 & poor==0"
local labels "Total" "Urban" "Rural" "Poor" "Non-poor"
local i=1
foreach outcome in Y1 Y3aplus Y3bplus {
    local j=1
    foreach subgroup in "if imp_m==0" "if UR_reg==1 & imp_m==0" "if UR_reg==0 & imp_m==0" "if imp_m==0 & poor==1" "if imp_m==0 & poor==0" {
        if strpos("`subgroup'", "UR_reg") >= 0 {
            rdrobust `outcome' X `subgroup', kernel(triangular) vce(cluster X) covs(region_dummy*)
        }
        else {
            rdrobust `outcome' X `subgroup', kernel(triangular) vce(cluster X) covs(UR_reg region_dummy*)
        }
        //estimates store result`i' , title("`outcome' `labels[`j']'")
        estimates store X_result`i'
        estimates restore X_result`i'
        estadd local title "`outcome' `labels[`j']'"
        local i = `i' + 1
        local j = `j' + 1
    }
}
* Use coefplot to plot the stored estimates
coefplot (X_result1, label("Utilization Total (1)")) (X_result2, label("Utilization in Urban (1)")) (X_result3, label("Utilization in Rural (1)")) (X_result4, label("Utilization by Poor (1)")) (X_result5, label("Utilization by Non-poor (1)")) ///
         (X_result6, label("Utilization Total (3a)")) (X_result7, label("Utilization in Urban (3a)")) (X_result8, label("Utilization in Rural (3a)")) (X_result9, label("Utilization by Poor (3a)")) (X_result10, label("Utilization by Non-poor (3a)")) ///
         (X_result11, label("Utilization Total (3b)")) (X_result12, label("Utilization in Urban (3b)")) (X_result13, label("Utilization in Rural (3b)")) (X_result14, label("Utilization by Poor (3b)")) (X_result15, label("Utilization by Non-poor (3b)")), ///
         drop(_cons) xline(0) ///
         title(Pooled Cutoff) ///
         mlabel(cond(@pval<.001, "***", ///
cond(@pval<.01, "**", ///
cond(@pval<.05, "*", ///
cond(@pval>=.05, " ", ///
string(@pval)))))) 
note("p-values shown alongside markers" "* p<0.05, ** p<0.01, *** p<0.001")

Can I modify the coefplot command such that I can get the significance stars based on the bias-corrected or robust p-values?

[George Ford]

This appears to work, but you'd have to code a lot more since you have multiple effects.

use rdrobust_senate.dta , clear

rdrobust vote margin if class==1
matrix S = e(V_rb_r)[1,1]
matrix list S
estadd matrix S

coefplot , mlabel(cond(@pval<.01, "***", cond(@pval<.05, "**", cond(@pval<.10, "*", ""))))
coefplot , se(S) mlabel(cond(@pval<.01, "***", cond(@pval<.05, "**", cond(@pval<.10, "*", ""))))

[Adrij Chakraborty]

Hi George! Thanks for the help.

However, I still see that the desired p-value is not being added to the estimation. That is, the number that appears when I request

matrix list S

is not the robust p-value that I can see in my output table.

Could you please clarify as to why you chose e(V_rb_r) and not e(V_rb_l) or something else from the documentation?

[George Ford]

it's stored in e(pv_rb) . i added another approach that may be better suited to your needs. it would take a bit of work to do more than outcome.

use rdrobust_senate.dta , clear

rdrobust vote margin if class==1
matrix P = e(pv_rb)
matrix list P
local star = cond(P[1,1]<.01, "***", cond(P[1,1]<.05, "**", cond(P[1,1]<.10, "*", "")))
matrix S = e(V_rb_r)[1,1]
matrix list S
estadd matrix S

coefplot , mlabel(cond(@pval<.01, "***", cond(@pval<.05, "**", cond(@pval<.10, "*", "", string(@pval)))))
coefplot , se(S) mlabel(cond(@pval<.01, "***", cond(@pval<.05, "**", cond(@pval<.10, "*", ""))))
coefplot , mlabel("`star'")

[George Ford]

Two models

eststo e1: rdrobust vote margin if class==1
matrix P = e(pv_rb)
matrix list P
local star1 = cond(P[1,1]<.01, "***", cond(P[1,1]<.05, "**", cond(P[1,1]<.10, "*", "")))
eststo e2: rdrobust vote margin if class==2
matrix P = e(pv_rb)
matrix list P
local star2 = cond(P[1,1]<.01, "***", cond(P[1,1]<.05, "**", cond(P[1,1]<.10, "*", "")))
coefplot (e1, label("Class 1") mlabel("`star1'")) (e2, label("Class2") mlabel("`star2'"))

[Adrij Chakraborty]

Two models

eststo e1: rdrobust vote margin if class==1
matrix P = e(pv_rb)
matrix list P
local star1 = cond(P[1,1]<.01, "***", cond(P[1,1]<.05, "**", cond(P[1,1]<.10, "*", "")))
eststo e2: rdrobust vote margin if class==2
matrix P = e(pv_rb)
matrix list P
local star2 = cond(P[1,1]<.01, "***", cond(P[1,1]<.05, "**", cond(P[1,1]<.10, "*", "")))
coefplot (e1, label("Class 1") mlabel("`star1'")) (e2, label("Class2") mlabel("`star2'"))

Hi George, I believe with your way of coding it, I have managed to extract the robust p-value instead of the conventional one. Many thanks!

[Adrij Chakraborty]

When I use this block of code:

    eststo e1: rdrobust ue_dau_TM X42 if source == 42, kernel(triangular) vce(cluster X42) h(10)
        matrix P = e(pv_rb)
        matrix list P
        local star1 = cond(P[1,1]<.01, "***", cond(P[1,1]<.05, "**", cond(P[1,1]<.10, "*", "")))
        eststo e2: rdrobust ue_dau_TM X44 if source == 44, kernel(triangular) vce(cluster X44) h(10)
        matrix P = e(pv_rb)
        matrix list P
        local star2 = cond(P[1,1]<.01, "***", cond(P[1,1]<.05, "**", cond(P[1,1]<.10, "*", "")))
        eststo e3: rdrobust ue_dau_TM X49 if source == 49, kernel(triangular) vce(cluster X49) h(10)
        matrix P = e(pv_rb)
        matrix list P
        local star3 = cond(P[1,1]<.01, "***", cond(P[1,1]<.05, "**", cond(P[1,1]<.10, "*", "")))
        matrix CL = e(ci_l_rb)
        matrix CH = e(ci_r_rb)
        matrix mat3 = (CL \ CH) 
        matrix list mat3
        coefplot ///
            (e1, label("at 42") mlabel("`star1'")) (e2, label("at 44") mlabel("`star2'")) (e3, label("at 49") mlabel("`star3'") ci(mat3)), drop(_cons) xline(0) title(UE with 10 months window)

Then the result that appears is the following:
coefplot ///
> (e1, label("at 42") mlabel("`star1'")) (e2, label("at 44
> ") mlabel("`star2'")) (e3, label("at 49") mlabel("`star3'") ci(mat3)), drop(_con
> s) xline(0) title(10 months window)
(e3: e(mat3) not found)
(e3: e(mat3) not found)
(e3: could not determine CI1)

As per coefplot help, there seems to be nothing wrong with the command. I just want coefplot to show the robust confidence intervals and not the conventional (non-robust) ones (in this case for regression stored as e3, but also for e1 and e2). What am I doing wrong here?

[Sebastian Calonico]

Dear Adrij,

I believe your issue comes from the requirements in coefplot, where the CI matrix has to be an e-class. It should work if you add "estadd matrix mat3 = mat3" after "matrix mat3 = (CL \ CH)".

I hope this helps.

Sebastian.