Interpretation of Rosenbaum Bounds

[Felix Baumann]

Dear Statalist Community,

I am testing the sensitivity of my propensity score matching analysis using Rosenbaum Bounds, I am yet struggling to interpret the results. Which one of the values actually shows whether the matching is sensitive to unobserved variables?

.         rbounds diffexq0, gamma(1 (0.2) 3)

Rosenbaum bounds for diffexq0 (N = 217 matched pairs)

Gamma           sig+      sig-    t-hat+    t-hat-       CI+       CI-
----------------------------------------------------------------------
    1        .003697   .003697   .075985   .075985   .019084   .133585  
  1.2        .063237   .000059   .040587   .109437  -.011993   .170511  
  1.4        .285774   5.7e-07   .013905   .139393  -.039594   .200364  
  1.6        .604479   3.9e-09  -.007144   .164094  -.064919   .225683  
  1.8        .840999   2.2e-11  -.027059   .187539  -.089539   .245695  
    2        .951309   1.1e-13  -.046161   .206667  -.105036   .263847  
  2.2        .988022   4.4e-16  -.062986   .223736  -.124032   .280321  
  2.4        .997524         0  -.078089    .23775   -.14306   .295171  
  2.6        .999555         0  -.093383   .249851   -.15955    .30874  
  2.8        .999928         0  -.104309   .261534  -.175378   .320645  
    3        .999989         0  -.116965   .274114  -.190518   .331041  

* gamma  - log odds of differential assignment due to unobserved factors
  sig+   - upper bound significance level
  sig-   - lower bound significance level
  t-hat+ - upper bound Hodges-Lehmann point estimate
  t-hat- - lower bound Hodges-Lehmann point estimate
  CI+    - upper bound confidence interval (a=  .95)
  CI-    - lower bound confidence interval (a=  .95)

Your advice is much appreciated!

Best,
Felix

[Felix Baumann]

Ok, let me rephrase my question:
Can I conclude from calculating the Rosenbaum Bounds that my analysis is insensitive to a bias that would increase the odds of treatment by 20% but sensitive to a bias that would increase them by 40%? Is it furthermore viable to read the results in a way that the analysis is completely insensitive to underestimating but rather sensitive to overestimating the impact of the treatment on diffexq0?

Despite working through the relevant literature (e.g. Rosenbaum, 2010) I am still struggling to make sense out of this.

[frederick lim]

have u found an answer to this yet?

[Nikos Korompos]

I have the same problem (i.e. not very familiar with the interpretation). Any ideas?

Many thanks

Nikos

[Panos Markou]

Hi Felix,

The way I understand this is as follows: The two columns sig+ and sig- are p-critical values, t-hat+ and t-hat- (Hodges-Lehmann) have to do with the median.

Now, consider your hypothesized effect direction. This influences whether you should be looking at the sig+ or sig- column. If your hypothesized direction is positive, then focus on the sig+ level.

Can I conclude from calculating the Rosenbaum Bounds that my analysis is insensitive to a bias that would increase the odds of treatment by 20% but sensitive to a bias that would increase them by 40%?

If your level of significance is 0.10, then yes. If your significance level is 0.05, however, then you cannot reach this conclusion since at Gamma = 1.2, you have surpassed the threshold.

Is it furthermore viable to read the results in a way that the analysis is completely insensitive to underestimating but rather sensitive to overestimating the impact of the treatment on diffexq0?

I am not sure I understand your question, but I do not think this is a correct interpretation. If your hypothesized direction is positive, then what does it even mean to underestimate it?

Finally, let me refer you to two further sources that are quite thorough on the matter:

Bharath, S. T., Dahiya, S., Saunders, A., and Srinivasan, A. (2011), Lending Relationships and Loan Contract Terms, The Review of Financial Studies, 24(4): 1141-1203. https://doi.org/10.1093/rfs/hhp064

DiPrete, T. A. and Gangl, M. (2004), Assessing Bias in the Estimation of Causal Effects: Rosenbaum Bounds on Matching Estimators and Instrumental Variables Estimation with Imperfect Instruments. Sociological Methodology, 34: 271–310. http://onlinelibrary.wiley.com/doi/1...4.00154.x/full