Metan – override pooled effect with user-defined pooled estimate in forest plot

[Sophia Backhaus]

Dear All,

I ran a meta-analysis using robust variance estimation. To display the forest plot, I used the metan command (Stata 17.0).
I would like to override the pooled effect on the plot produced using metan with the actual pooled estimate and confidence interval from my RVE analysis.
Unfortunately, I have some troubles doing this and could not find the information in the metan help file.

This is the code:

foreach var of varlist emo_phy_violence_fifty {
robumeta cohensd if `var'==1, variance(cohensd_var) study(studyid) weighttype(random) rho(0.8)
dis e(QE)
sort author year measure
metan cohensd cohensd_sd if `var'==1, nowt nobox nohet nokeep notable lcols(author year measure int_post_n ctrl_post_n) texts(105) astext(60)
graph export `var'.pdf, as(pdf) replace
}

The RVE produced pooled estimate that I want to display in the forest plot is d = -0.46, and the CI: -0.60, -0.33

I would be most grateful for your help

Thank you!

All the best,
Sophia

[David Fisher]

Hi Sophia,

This is difficult to answer properly without knowing more about your dataset. For instance, if I use the example from the robumeta help-file, I find that there are 179 observations clustered within 66 studies, which cannot possibly be displayed on a forest plot. In particular, I guess my question is: which estimate(s) do you intend to use to represent each individual study? In the paper by Hedges et al, the idea is raised of "synthetic" study-specific pooled estimates, obtained by a simple average of all the information for each study. Is this the procedure that you also wish to use?

For the sake of a simple example, let's take the first 4 studies (12 observations) from the example dataset hedgesexample.dta. Replacing the default pooled estimate with user-defined values is actually very straightforward and has been a feature of metan for many years. With the current version (v4.06 12oct2022), one way to proceed is as follows:

use hedgesexample.dta
robumeta effectsize k1 if study<=4, study(study) variance(vareffsize) weighttype(random) rho(.8)
lincom _cons
local user `r(estimate)' `r(lb)' `r(ub)'

gen wgt = 1/vareffsize
gen stderreffsize = sqrt(vareffsize)
metan effectsize stderreffsize if study<=4, study(study) wgt(wgt) model(`user', label(Robust) extralabel(Random, {&rho}=0.8))
​​​​​​​

Note that, unfortunately, there is a small bug here: with user-defined values you are currently obliged to use the "extralabel()" option whether you want to or not -- if you do not, the command fails. I will make sure to correct this in the next update.

Hopefully this example gives you something to work with. Note that although I currently maintain the metan package, I am not familiar with the technique described by Hedges et al, so I may have gotten some details wrong.
Best wishes,
David.

[Tiago Pereira]

A possible solution - while -metan- is under its way to updating, is to use -meta-

Observation: When we have non-normal-based confidence intervals in the primary studies, we need to use the option meta set mean lower upper, civartolerance(1) and then
replace _meta_es = mean
replace _meta_cil = lower
replace _meta_ciu = upper

When we don't have this issue, we can use something like this:

    *! estimates from robumeta
    local mean =  -0.46
    local lower = -0.60
    local upper = -0.33

    set seed 123456
    clear
    set obs 10
    gene es = rnormal(-0.4,0.5)
    gene se = runiform()+0.005
    meta set es se
    gene author = _n
    gene year = 2010+_n
    gene measure = cond(runiform()>0.5,"A","B")
    
    meta forestplot author year measure _plot _esci,    cibind(parentheses)  noohetstats noohomtest noosigtest noghetstats nogwhomtests nogbhomtests nowmarkers bodyopts(justification(left))   customoverall(`mean' `lower' `upper')   xline(`mean', lpattern(shortdash) lcolor(black)) nullrefline(favorsleft("Intervention" "better") favorsright("Control" "better")) xlabel(-2(1)2) crop(-2,2)
    
    
    *! omit the REML summary estimate - diamond
    gr_edit plotregion1.column4.group[1].draw_view.setstyle, style(no)
    gr_edit .set_spositions
    gr_edit .set_graphwidth
    
    *! omit the REML summary estimate - point estimate and lower/upper bounds
    forvalues i  = 6/8 {
        gr_edit .plotregion1.column`i'.group_items[1].draw_view.setstyle, style(no)
        gr_edit .set_spositions
        gr_edit  set_graphwidth

    }
    
    *! omit the message
    gr_edit .note.draw_view.setstyle, style(no)
    gr_edit .set_spositions
    gr_edit .set_graphwidth

It is not perfect, but is a possible solution. You will need to manually find the column numbers to omit the summary estimates calculates by -meta-

[Claire YUMEI]

Hi Sophia,

Thanks for your previous post. I also met the similar issue for my research project. I'm wondering how you solve the problems and get the final version of the forest plot out. If it is possible to share this part of code for reference. Thanks again!

Best regards,

Claire

[David Fisher]

Hi Claire (et al.),

Just a note to say that the bug in metan mentioned in post #2 is now fixed (version 4.08 17jun2024), so some possible code using the Hedges example might be:

use "hedgesexample.dta", clear

* Carry out RVE and store estimates
robumeta effectsize k1, study(study) variance(vareffsize) weighttype(random) rho(.8)
lincom _cons
local RVE `r(estimate)' `r(lb)' `r(ub)'

* Collapse to one observation per study
* (not sure how best to do this; I'm using common-effect meta-analysis)
gen stderr = sqrt(vareffsize)
statsby effectsize=(r(eff)) stderr=(r(se_eff)), by(study) clear : metan effectsize stderr, nograph

* Construct forest plot
gen weight = 1/stderr^2
metan effectsize stderr, model(.277 .165 .389, label(RVE)) wgt(weight) nowt forestplot(nobox nonote xlabel(-2(2)4, force) astext(40))

Additionally, built-in Stata meta forestplot has also been updated since post #3, and now also supports user-defined overall estimates whilst suppressing the default REML estimate, so you could instead use:
(run code as above up until "construct forest plot")

meta set effectsize stderr
meta forestplot _id _plot _esci, nooverall customoverall(.277 .165 .389, label(RVE)) noohetstats noohomtest noosigtest noghetstats nogwhomtests nogbhomtests nowmarkers nonotes

[Claire YUMEI]

Hi, David,

Thanks for so much for you kind support! I would like to give it a try based on my dataset. Could I please ask where I can get the sample database "hedgesexample.dta"? Thanks again!

Best,

Claire

[Claire YUMEI]

Hi, David,

Thanks! it shows like this:

use hedgesexample.dta
file hedgesexample.dta not found
r(601);

Thanks for your further guidance!

Claire

[Tiago Pereira]

Claire, you can find the ancillary file here:

findit robumeta

or

net get robumeta.pkg

[Claire YUMEI]

Hi, Tiago, David,

Thanks! It shows "all files already exist and are up to date".

I am currently encountering an issue with my robumeta code: " robumeta lnHR, study(robust) variance(variance) weighttype(random) rho(.80) eform". This code successfully calculates the overall estimates using the robust variance estimates, considering the overlap in populations from the included studies.

My goal for this part of the analysis is to generate a forest plot that displays each subgroup estimate (cancer types) as well as the overall estimates for total cancer. It is important that these estimates use the robust variance estimates for each subgroup and overall estimates, as some subgroups (specific cancer type) also have overlapping populations from the included studies.

I have been stuck on this issue for a while and would greatly appreciate any further assistance and guidance you can provide regarding the creation of this forest plot.

Thank you in advance for your help!


Claire

[Tiago Pereira]

Claire, you can plot the individual estimates normally. They will not be affected - unless you use an ad-hoc approach to inflate their standard errors. However, if you have conducted a robust meta-analysis, via -robumeta-, you can replace the traditional summary estimate (IV approach assuming no clustering) with the one obtained via -robumeta-. I am not sure if -metan- or -meta- machineries allow to replace subgroup specific estimates calculated using an external program.

[Claire YUMEI]

Thanks again! Could I please ask the code that could combine the figures of Standard and contour-enhanced funnel plots? Thanks in advance!

[Claire YUMEI]

Thanks! I just followed the manual code to do the meta regression: https://www.stata.com/manuals/metametameregress.pdf However, it did not work.


. meta merregress lnHR || Studydesign:, essevariable(se_new)
subcommand meta merregress is unrecognized
r(199);

If I use other way, it shows like this :

. meta regress Studydesign
no observations
r(2000);



Thanks for your further guidance!

[Tiago Pereira]

Older versions of Stata do not include the -meta merregress- command. Which version of Stata are you using?

[Claire YUMEI]

Older versions of Stata do not include the -meta merregress- command. Which version of Stata are you using?

17.0. Thanks

[Claire YUMEI]

Hi, Professor. Can I please ask how to let the following code show the exponential results? Thanks!

robumeta lnHR, study(robust) variance(variance) weighttype(random) rho(.80) eform