Regression within-subject data

[Katerina Novakova]

Hi,

I am currently conducting a study trying to research the effect of a security incident (IV) on a company's reputation (DV) (+ certain moderating effects and control variables; however, those are not the important part here).
I have used the within-subject method and measured responses on a Likert scale. First, I measured the reputation without mentioning a security incident, then I introduced the scenario to the participants (a security incident has occurred), and then I measured the reputation again (same questions).

Which regression would you choose (and how would you formulate it) if you wanted to measure the effect of a security incident on a company's reputation?
I presume I will need to use deltaReputation and somehow use it in a regression. But would this delta effectively be used as a measure of the security incident (replacing IV in the regression command - e.g., reg Reputation deltaReputation ...)?
Or is there something else that I am not considering?

Thank you in advance

[Carlo Lazzaro]

Katerina:
welcome to this forum.
I would consider it as a fixed effect panel data regression exercise.
You have two wave of data (companies' reputation before and after the incident) and the a categorical variable (the incident), that can be plugged in the right-hand side of your regression equation as a two-level predictor (no incident=0; incident=1; see -fvvarlist- notation for further details) + other predictors and controls.
In Stata language:

xtset firms waves
xtreg reputation i.incident <otherpredictors> <controls>, fe vce(cluster firms)

Cluster-robust standard errors are Ok if your sample is composed of 30 firms at least.

[Katerina Novakova]

Hi Carlo,
Thank you for your quick response.

Your expertise really helped, and I managed to make it work once I started looking at the dataset as panel data.
The only thing I did differently is to use the random effects model (instead of fixed effects) as the fixed effects model seems to be omitting some of my variables (I also used Hausman to confirm that it should be re).

Thank you for your help.
Katerina

[Carlo Lazzaro]

Katerina:
as we know, the -fe- machinery wipes out time-invariant variables.
Testing -fe- vs -re- specification via -hausman- is correct unless you should go cluster-robust standard errors.
If that were the case, you should switch from -hausman- to the community-contributed -xtoverid-.
Lastly, it would be not correct to test -fe- vs -re- via -hausman- with default standard errors and adding cluster-robust standard errors after the -hausman- outcome because you detected heteroskedasticity and/or autocorrelation.

[Katerina Novakova]

Dear Carlo,

I have just gotten feedback that it is "methodologically incorrect" to use a within-subject design with a random effects panel analysis (as the RE model does not answer an RQ that is interested in a within-subject design). However, the feedback did not mention why this would be the case. Do you see any reason why this combination could be considered methodologically incorrect?

/The RQ: What is the relationship between a security incident (IV) and a company's reputation (DV) and how do moderators x,y, and z influence this relationship?/

Thank you.

Katerina

[Carlo Lazzaro]

Katerina:
the only methodological reason I see is that while the -fe- estimator focuses on within panel variation, the -re- one considers a mix between within and between panel variation.

[Katerina Novakova]

Dear Carlo (and anyone else with the proper knowledge),

I looked into it, and the only reason I can find is the fact that a random effects model is better for studying differences across entities rather than within an entity over time (which is the point of a within-subject experiment). Nevertheless, I still did not find why using it in my case would be inappropriate.

Nevertheless, I am now looking into a mixed-effects model (considering both FE and RE) as a possible better fit. After running the mixed code, I got the same results as my random effects code.

xtreg DV IV Control1 Control2, re vce(cluster Id)

mixed DV IV Control1 Control2 || Id:, vce(cluster Id)

Would it be correct to assume that (since the results for these are virtually identical) the RE model can still be kept as the main analytical tool, and the mixed model could be used as a robustness check?

Moreover, am I using the mixed code correctly? I am quite new to the mixed method.

Lastly, the mixed model also provides var(const) and var(residual). Are these important for the interpretation of the model? If so, could you help me understand how to interpret them (or which paper would be talking about this)?

Thank you!

Katerina