Effect size in fixed effects regression

[Alex Mueller]

Hello together!

I am performing a fixed effects Regression with Panel data (several Managers over several Points of time) and try to evaluate the effect of Promotion on work Performance (I already discussed possible reverse causality issues).

My Regression basically Looks like this:

 xtreg Performance_indicator Promotion(a binary indicator) controlvariables, fe

The Overall model is significant and also the binary Promotion explanatory variable is.

To improve my model i also want to estimate the effect size of Promotion:
So i tried:

 esize twosample performance, by(Promotion) 

this delivered a Cohens d of roughly 1.1 indicating a strong effect.

1) Now I am wondering whether Stata really calculated what i am interested in. Is this the effect i want to measure?
2) is esize correct in a Panel fixed effects Regression Setting? (what is about esizei,and estat esize?)
3) As i understand esize compares the mean of 2 Groups. Does my command divide the Group into Promotion == 0 and Promotion == 1? In this case The two Groups are not the same size

I am looking Forward to your thoughts on this.

Cheers, Alexander-Florian

[Carlo Lazzaro]

Alex:
my concern is that you do not have independent observations within each panel (and -estat esize- does not allow a -cluster- option). Nor I've seen application of -estat esize- after panel data regression (but this may obviously be my fault).

[Alex Mueller]

Carlo,
hello again and thanks for your reply!

Addendum to my previous post: I use the "robust specificator" so the code is

 xtreg Performance_indicator Promotion(a binary indicator) controlvariables, fe (robust)

Carlo, why do you think the observations in my Panel (i only have one Panel) are not independent? And Independent from what?

Best,
Alex

[Carlo Lazzaro]

Alex:
panels are the groups of the observations with the same panel identifier. It's reasonable that observations belonging to the same panel (say Carlo taking maths tests on three different sessions during the same year and related covariates) are more similar than those belonging to another student (who actually is another panel of observations). That's why observations belonging to the same panel are not independent (all in all, they relate to the same panel unit).
When you state that you have one panel only, I assume that you mean you have one dataset only: if you had one panel only (that is, different ids with one observation each), you would have one wave of data only and you should switch to -regress-.
The -robust- option (you could have used the -cluster- option as well: under -xtreg- they do the very same job, that is taking heteroskedasticity and/or autocorrelation into account) adds nothing to what above.

[Alex Mueller]

Carlo,

thanks for that clarification. Yes i have one Panel i.e. one dataset with several Managers at several Points in time.

1) Is the consequence that (analogous to your math test example) the observations over time for a single Manager are more related than between different Managers?

2) Is it possible to calculate a meaningful effect size in my dataset then? I am trying to get additional Information on the influence of my explaining variable Promotion on the dependent variable performance

Cheers, A.

[Carlo Lazzaro]

Alex:
1) yes, this is exactly what I meant;
2) not that I know; I would consider interactions (if feasible).

[Alex Mueller]

Carlo,

many thanks for your help. The interaction Terms of Promotion and other control variables are insignificant.

I guess i try to argue using the coefficient of "Promotion" and its significance Level.

Best, Alexander-Florian

[Carlo Lazzaro]

Alex:
wise choice.