difference in differences model with small sample size

[Marry Lee]

Hello,
I have a small dataset with number of individuals=900.
These individuals live in different counties and are observed between 2000 and 2005. This is a pool of cross sections, hence I do not have the same number of individuals in each year in every county.


There is a policy implemented in 2003 at the county level. So I have the following model (difference in differences):

 
 probit Outcome i.Treat##i.post_Treat  (other X vars)  i.provinceXyearFE i.county, cluster(county)

I am confused about a problem.

Given the small sample size, I have few individuals per county observed in a given year.
I know that this may induce the incidental parameter problem if we include county fixed effects by hand as I did (i.county), this can be corrected by using a correlated random effects model.

But is there also another problem that results are being driven by certain observations in a certain county in the post period (so this problem cannot be adressed using the correlated random effects model right?) OR is this not a problem if there is a randomness in whether they are observed or not ?



Thank you.

[William Rossi]

I believe that your question is more concerned about outliers driving the results. I would try other estimators that are "robust" to outliers like leave-one-out or quantile regression.