Non-parametric tests in difference-in-difference (rank-test)

[Kasper Nygaard]

Hi Statalist,

I have conducted a difference-in-difference using ROA on firms undertaking a SEO matched with control firms using propensity score matching. A simple parametric t-test has been used to test for significance in ROA (and other different operating performance meaures). I am considering to use multiple statistical tests (in particular non-parametric tests due to assumption-problems with t-test) on my analysis and have read about the Wilcoxon rank test.

My questions is; does it make sense to implement Wilcoxon rank test with panel data using difference-in-difference? I'm not sure how it can measure the dependent variable in panel data. Or can I implement the test to the propensity score matching to test for difference in the treated and untreated group?

In addition to that, I search for recommendations to implement it in STATA.

Hope anyone can help me. Thanks in advance.

Best regards,
Kasper Nygaard

[Carlo Lazzaro]

Kasper:
when dealing with continuous variable, rank-based tests should really be the last resort.
If you are worried about assumption problems with -ttest-, you can consider a -bootstrap- version of the same test (see -help bootstrap- and related entry in Stata .pdf manual).
An interesting (but not free-dowloadable) article on bootstrap procedure at large is: https://www.ncbi.nlm.nih.gov/pubmed/11113956.

[Kasper Nygaard]

Thank you so much, Carlo!

[Kasper Nygaard]

Carlo, I have an additional question to your comment a week ago. Why is rank based tests a problem when dealing with continous variables? Can you link to a relevant article within this field? I am searching for a strong argument to avoid rank-based tests. Thanks in advance!

Best regards,
Kasper

[Carlo Lazzaro]

Kasper:
https://www.ncbi.nlm.nih.gov/pubmed/11113956 is the link to my favourite article on this topic.
In sum, the main argument against rank-based test when dealing with continuous variable rests on the fact that researchers/decision-makers are interested in the difference between two continuous variables (ie, operating on their original metric), not in ranking them. Moreover, rank-based tests can give opposite results to those obtained from the difference in arithmetic means.