Why do different fixed effects change some coefficients during robustness tests?

[Marco Obrador]

Hello everybody,

I am currently investigating the effects of a host country's domestic war on donor ODA payments in a dyadic dataset. My basic model has the following form:

ODA = war + population + oil + ... + slave fixed effects (all controll variables are logged and lagged by one year)

and now I want to further enhance the robustness of my results by adding different fixed effects.

Now the signs change with some control variables, e.g. In the above example, oil first entered into the model positively and becomes negative if I additionally control for annual fixed effects, while my independent variable remains significant.

For example, how can I explain this change of sign when adding fixed annual effects and / or donors fixed effects? What causes this? I'm really unexperienced in this field and would be very happy for any help.

Many thanks
Marco

[Clyde Schechter]

There is no surprise here. Whenever you add new variables to a regression model, everything is subject to change. The phenomenon is known by various names, most commonly it is called Simpson's paradox. The Wikipedia page on Simpson's paradox is quite good and I recommend it to you. There it is presented in the context of simple contingency tables, but exactly the same logic applies to the regression setting. (It is sometimes called Lord's paradox in the regression setting.)

[Marco Obrador]

Hi Clyde,

thank you so much for your quick response. Can I ask a question to clarify:

If I understand my research on this topic correctly, there is no way to find out which of the contradictory conclusions is the right one - so you have to understand the causal pathways to justify your decision about the "right" model.
How do I approach this understanding of the causal path correctly to derive a proper justification?

Many thanks for your help.
Marco

[Carlo Lazzaro]

Marco:
the literature in your research field can support you in this respect. Just skim through it and see what others did in the past when presented with the same research topic.
At the top of that, your regression models should give a fair and true view of the data generating process (eg, no omitted predictor that, in turn, may cause model misspecification and endogeneity).