Testing whether differences between two groups are different in different sub-samples

[Weiting Hong]

We are working with stock return data and have a conditional double-sort first on firm size (characteristic 1) into terciles and then on whether an event has occurred to this firm within the past year (characteristic 2). Event occurrence is not evenly distributed across size terciles.

How can we test whether the average return difference between states of event occurrence is itself different in small vs large firm terciles? In another word, how can we test whether the return difference along characteristic 2 is different across terciles of characteristic 1?

For example, in the tercile with the biggest firms, 500 firms experienced the event, and 300 firms did not; in the tercile with the smallest firms, 600 firms experienced the event, and 700 firms did not. Say an unpaired t-test among the biggest firms produces an average return difference of 3% with a standard error of 0.4%, and an unpaired t-test among the smallest firms produces an average return difference of 2% with a standard error of 0.5%. How can we test whether the 3% is statistically significantly different from the 2%?

Thank you very much for your time and help in advance!

[Clyde Schechter]

So let me assume that your data set has a variable for size tercile called characteristic1 and one for the event called characteristic2. I assume that you also have variables named firm_id and year and return. You want to contrast the average return with and without characteristic, separately in each of the three terciles.

xtset firm_id year
xtreg return i.characteristic1##i.characteristic2 i.year, fe // AND PROBABLY CLUSTERED ERRORS, MAYBE SOME COVARIATES IF NEEDED
margins characteristic1, dydx(characteristic2)
margins characteristic1, dydx(characteristic2) pwcompare

The output from -margins- will have three rows, one for each tercile, and will give the mean difference based on characteristic2, along with confidence intervals and test statistics. The output from -margins, ... pwcompare- will show you differences among those differences and test statistics.

In the future, when asking for help with code, please use the -dataex- command and show example data. Although sometimes, as here, it is possible to give an answer that has a reasonable probability of being correct, this is usually not the case. Moreover, such answers are necessarily based on experience-based guesses or intuitions about the nature of your data. When those guesses are wrong, both you and the person trying to help you have wasted their time as you end up with useless code. To avoid this, a -dataex- based example provides all of the information needed to develop and test a solution.

If you are running version 18, 17, 16 or a fully updated version 15.1 or 14.2, -dataex- is already part of your official Stata installation. If not, run -ssc install dataex- to get it. Either way, run -help dataex- to read the simple instructions for using it. -dataex- will save you time; it is easier and quicker than typing out tables. It includes complete information about aspects of the data that are often critical to answering your question but cannot be seen from tabular displays or screenshots. It also makes it possible for those who want to help you to create a faithful representation of your example to try out their code, which in turn makes it more likely that their answer will actually work in your data.