How do you control for time-varying unobserved heterogeneity in panel data?

[John Adler]

I am completing a longitudinal analysis of parental unemployment shocks on children's weight across 3 waves.

clogit kidsweight i.parentsunemployed i.urban_or_rural i.year i.parents_age_y i.Parents_Educa i.Parents_Marital cluster (id) group(id) nolog
margins, dydx(parentsunemployed) post

Weight and unemployment are binary variables ==1 if true. Thus, I make use of the fixed effects conditional logit estimator to control for time-invariant unobserved heterogeneity.

I also use the period of a recession to provides more exogenous assignment mechanism than traditional employment change.

However, although I adjust for observed covariates in my analysis, it is possible that unobservable are not balanced, i.e. that transitions from employment to unemployment may be related to other unobserved shocks that are correlated with both the probability of job loss and the probability of excess weight in children.

What can I do about this? Is there a simple test in Stata of whether the treatment XT is correlated with another unknown time-invariant variable XU, which also has a causal impact on y, but is unobserved?

If not, is there a simple way to alleviate potential reviewer concern that an unobserved time-varying effect is biasing the results of an analysis of the impact of XT on y away from it's true effect?

Is this just a problem people live with?

Please help!

John

[Felix Scholl]

What can I do about this? Is there a simple test in Stata of whether the treatment XT is correlated with another unknown time-invariant variable XU, which also has a causal impact on y, but is unobserved?

If not, is there a simple way to alleviate potential reviewer concern that an unobserved time-varying effect is biasing the results of an analysis of the impact of XT on y away from it's true effect?

Is this just a problem people live witth?

No, there is no simple test and definitely no simple way to alleviate potential reviewer concern. If there was causal analysis would be easy and every social scientist would use follow that method, right? Unobserved factors are unobserved by defintion. How can you control or test something you do not observe? The only situation, where you can completely sure that you have controlled for all variables is experiments. If you have observational data, you need solid knowledge of your data and good theoretical arguments instead

[John Adler]

Thanks for your feedback Felix.

So what is the done thing in published studies? Do researchers just make an argument that there is not time-varying unobserved heterogeneity in their data? How would one support such a claim?

Thanks again,

John

[Felix Scholl]

Again, there is not "one thing" that is done in studies. It really depends on the subject at hand. Apart from experimental studies there is no gold standard. Causal analysis is a true science in itself and I would say that scientist spent a significant amount of time dealing with those question (at least social scientists).

In general you are right, they would just make an argument. Unobserved hererogeneity is a problem (and only then it is), if a third variable is correlated with both the treatment and the dependent variable. So you could argue:

1. There are other variables that fulfil that condition (perhaps we know that from previous studies or from simple reasoning), but we control for these factors. If that variable is not observed, you run into a problem, that is what we call ommitted variable bias (OVB). Your estimates would be biased. But you can still argue if that bias is severe or rather not.

2. There are unobserved factors that affect the dependent variable, but not the treatment. For example, when the parents smoke that would be bad for the kidsweights, but it is unlikely to have an effect on unemployment. Then your estimator is not biased by that. Effectively, if your treatment is assigned completely random, you are good. This is what experiments do. Usually you cannot fully achieve that, but you can come closer to that.

3. There are unobserved factors that affect your treatment, but not the dependent variable. Then you have no problem. For example, children in some cities have lower birth weight due to environemntal pollution. But pollution is unrelated to the probability of becoming unemployed. In that case your estimator will be unbiased even if you don't observe that variable. You just have more residual variance that remains unexplained.


That was a very quick summary. That question is too complex to be answered comprehensively. This is a very interesting introdoctury paper about causal inference from the field of sociology: https://www.annualreviews.org/doi/fu....012809.102702

I suggest you draw on the extensive, existing literature on research designs and causal analysis in your field.

[John Adler]

Dear Felix,

Thank you for your help with this.

I am working on my arguments but was wondering about your third argument, that unobserved factors are affecting the treatment but not the dependent variable. The example you give on pollution seems to have the same affect as the example in argument 2, i.e. it is affecting the dependent variable, but not the treatment. Although maybe you are making the point that these children's birthweight has been affected by pollution such that later they will not be affected as much by changes in the treatment (employment) than children who do not have a lower initial birthweight due to pollution?

I'm sorry if I misunderstand, could you please clarify?

Thanks so much for your help!

John

[Felix Scholl]

You are totally right. I made a mistake. My example in 3. was wrong. This time I try to give a correct example:

3. There are unobserved factors that affect your treatment, but not the dependent variable. Then you have no problem. For example, economic fluctuations/ recessions (e.g. measured by GDP growth) affect the probability of becoming unemployed. But it is unlikely that GDP growth has a direct impact on birth weight except if the parents loose their job. Or do you think that unborn babies respond to internationl trade? In that case your estimator will be unbiased even if you don't observe that variable. You just have more residual variance that remains unexplained.

Of course you can challenge these examples. I just made them up. You have to know your data well. What is plausible? What did other studies find? If there are important variables you cannot control for, how severe will it bias your results?

[John Adler]

Dear Felix,

Thank you, that certainly makes things clearer! I appreciate you taking the time to respond.

Thanks again,

John

[Jeff Wooldridge]

John: You can account for time varying unobservables that are correlated with you key explanatory variable if you have a time-varying instrumental variable. But that’s often a tall order.