Confidence intervals from estimation versus from margins after qreg

[Sebastian van Baal]

Dear Stata users,

I need to apologize for not having a working code example, but I do not think that I can easily replicate my problem with respect to the details that matter. That written, I would be thankful even for the most general suggestions.

I analyze how the prices charged by hospitals depend on certain characteristics of the hospitals. Since I am interested in the conditional median, I use quantile regression:

qreg rate i.revisedcompliance_score i.code i.i_county, vce(robust)

The confidence interval for "Medium" does not include zero. The base category for comparison is "Low". Thus, I conclude that there is a significant difference between "Low" and "Medium" at the 95% level.

To show the results graphically, I use marginsplot. Before I do so, I run

margins i.revisedcompliance_score

With margins, the confidence interval for "Low" begins at 1013.94, and the confidence interval for "Medium" ends at 1017.187. Hence, there is no significant difference between "Low" and "Medium" at the 95% level. This is also suggested by the marginsplot:

I suppose that there is a difference between how qreg and margins determine standard errors. Can I get margins (or marginsplot) to use the standard errors and confidence intervals from qreg?

Boy, how much I enjoy finally working with Stata again. After a long break due to teaching obligations, I almost forgot how much fun it is - and I am not joking!

Best regards,
Sebastian

[Clyde Schechter]

It is true that -qreg- and -margins- use different methods to calculate standard errors. But that is not the issue here.

You are confusing overlap of confidence intervals with lack of statistically significant difference. The standard error (radius/1.96 of the CI) of the difference between low and medium is the denominator for a significance test of the null hypothesis that the difference is zero. By contrast, the overlap of the two confidence intervals is based on the standard errors of the expected values of rate themselves. These are different things, and while they are related to each other, it is entirely possible to see two confidence intervals overlap greatly and yet the difference between their means can be estimated with great precision in the same data. It all depends on correlations between variables: the virtual indicator variables representing low and medium in your analysis are necessarily correlated fairly strongly with each other because they are indicators for two levels of a three-level variable. That sets the stage for the kind of "paradox" (it isn't really paradoxical) you are seeing.

See https://blog.minitab.com/en/real-wor...u-should-avoid for a more detailed explanation.

[Sebastian van Baal]

Dear Clyde,

On the one hand, awesome, because I learned something new today. On the other hand, terrible, because now I have to question an issue I used to believe in. Thank you! I found some further insight in the (supposedly) more fundamental paper at https://www.researchgate.net/publica...ence_Intervals. The author offers this advice: "When there are 3 or more levels of a factor, multiple comparison procedures are more appropriate to determine whether means are significantly different."

Enjoy your day!
Sebastian