IPDMETAN Treatment covariate interaction with predictive intervals

[Alex Wright-Hughes]

Hi,

I am exploring treatment x covariate interactions using IPDMETAN and run into a problem when including prediction intervals. The resulting forest plot repeats the presentation of the prediction interval where the confidence interval should be presented. I wonder if this is a bug in the code and if there is a relatively easy solution to this?

Many thanks for any help on this!
Alex

ipdmetan, study(StudyName) re(reml,hksj) truncate(zovert) keepall by(_Int,force) rfdist or interaction forest( astext(65) textsize(90) boxsca(30) xlab( 0.01 0.1 1 10 100  ) favours("Favours greater treatment effect" "with older age" # "Favours greater treatment effect" "with younger age")  : logistic Outcome Trt Trt##c.Age Age if Time == 4

[Tiago Pereira]

Beautiful graph!

You can use graph editor to omit the last 95% predictive interval, in case you don't want to repeat the same numbers. That happened because the estimated between-trial heterogeneity is zero or close to zero. Hence, the 95% predictive interval will be very similar to the 95% confidence interval - especially when the number of estimates is large.

Hope this helps!

Tiago

[Alex Wright-Hughes]

Thanks for your reply Tiago!

Sorry I think I should have been a bit clearer on the issue here. The estimated confidence intervals and predictive intervals do differ within this dataset, as shown by the estimates provided to the right of the lines of the forest plot. For example, for the fifth subgroup where a predictive interval is provided, the predictive interval is (0.05, 17.68) and the CI is (0.35, 2.36), however the lines on the plot indicating the CI and the PI are the same and in line with the PI estimate.

Note, I only seem to run into this problem when using IPDMETAN in interaction mode.

Many thanks!
Alex
-

[Tiago Pereira]

Alex, hi.

Yes, indeed. It is a bug. Hopefully David Fisher will see this thread and will be able to fix the problem in the upcoming weeks.

An immediate solution is to get those estimates, and use -meta- to plot them. You can manually replace squares with circles - to emphasize that the estimates refer to interaction effects.

if you have time, you can play around the options, and see the problem persists after changing the estimator of tau², etc.

All the best,

Tiago

[David Fisher]

Hi Alex and Tiago,

Firstly, apologies to Alex for the late response, I hope this will still be helpful. Thankyou to Tiago for responding at the time and providing some context.

I have finally had time recently to release updates (via SSC) to both ipdmetan (v4.03) and metan (v4.06) -- this is something I know has needed doing for a while, but I could not find the time!

Whilst preparing this update, I looked into the issue that Alex describes. It is a bug, of sorts -- but not necessarily in the way you might expect. Two things are happening here:

(a) Firstly, both the confidence interval andthe prediction interval are plotted, simultaneously, over the top of the summary effect circles. You can see this by giving the two lines different characteristics (colours, thicknesses, dashes, etc.) -- but on Alex's plot, it appears as if there is only the prediction interval. The reason for this behaviour makes sense in the context of diamonds. In that case, the diamond represents the confidence interval, with the prediction interval represented by additional horizontal lines either side. But, of course, this does not work for representations other than diamonds! This is what the code did not take into account, and why I said it was only a bug "of sorts".

(b) Then, underneath, the prediction interval is plotted (again), this time alone. This behaviour is a bug, since by design there should not be any lines appearing in those locations.

My resolutions are as follows:

(a) Because of the difficulties of coding to detect the various possibilities and react accordingly, I have basically left this behaviour as it is. There is now additional text in the help file, which explains that these two lines (confidence and prediction intervals) might be overlaid in some contexts, and may therefore be difficult to distinguish without additional line options (e.g. colours) being applied.

BUT I have also introduced the new option sepline, which basically turns (b) into a feature rather than a bug. In other words, under this option the prediction interval line is only plotted underneath, and does not appear overlaid on the confidence interval and summary effect circle.


I hope this is helpful.

Best wishes,

David.