Sensitivity analysis to assess selection bias

[Jessica Berrett]

Hi there! Is there a sensitivity analysis or test in Stata that can be used to estimate the direction and magnitude of the bias? Here's the situation with our data. We have a large panel data set that is gathered through surveys each year. Organizations can opt to take the survey or not, so the sample doesn't include the entire population of organizations. And then after data cleaning, the sample gets reduced significantly furthering the selection bias.

[Clyde Schechter]

You can assess the extent of the selection bias that results from the data cleaning by comparing key variables in the final sample with the same variables in the part of the original data set that got eliminated in data cleaning.

There is no real way to assess the selection bias that results from organizations not participating. The best you could do is try to get some estimates of key variables of the non-participants from some other source and then compare those to the corresponding variables in your data set. But the "other source" may have measured those variables in some different way from what the surveys underlying your data used, and in that case you can't distinguish the effects of selection bias from the effects of different measures being used. (Even worse, the measurement difference might cancel out the selection bias effect if you are really unlucky.) No good solution for this.

[Jessica Berrett]

Thank you so much Clyde, I appreciate your thoughts on this!