fracreg logit and fixed-effect OLS show different results

[Anna Pak]

Dear all,

I have an unbalanced panel data and my DV is a proportion bonded between 0 and 1.
My goal is to examine the moderating effect of M1 on two curvilinear relationships I suggest.
When I tried fractional logit and fixed-effect OLS for robustness checks, the curvilinear relationships are robust to two models, but I have opposite results for the moderating effects.

Please note that my main independent variables are PS and PA. PS and PA are split from a variable measuring the difference between the focal firm's financial performance and industry average performance (relative performance) depending on whether its value is below or above 0. PS is negative relative performance, and PA is positive relative performance. PS equals to 0 when the firm has positive relative performance, whereas PA equals to 0 when the firm has negative relative performance.

My hypotheses are as following:
H1: Inverted U-shaped relationship between PS and DV (supported)
H2. U-shaped relationship between PA and DV (supported)
H3: moderating effect of M1 on H1
H4: moderating effect of M1 on H2

First of all, I tried a fractional logit model.

fracreg logit DV c.PS##c.M1 c.PS2##c.M1 c.PA##c.M1 c.PA2##c.M1 M2 C1 C2 C3 i.C4 C5 C6 C7 i.year, vce(cl firm) nolog


Fractional logistic regression Number of obs = 15,939
Wald chi2(61) = 2800.86
Prob > chi2 = 0.0000
Log pseudolikelihood = -8066.3057 Pseudo R2 = 0.1451

(Std. Err. adjusted for 19,964 clusters in firm)
------------------------------------------------------------------------------
| Robust
DV | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
PS | .0889003 .0261946 3.39 0.001 .0375598 .1402407
M1 | -.0041093 .0040971 -1.00 0.316 -.0121394 .0039208
|
c.PS#c.M1 | -.0093061 .0026438 -3.52 0.000 -.0144878 -.0041244
|
PS2 | -.0136717 .0031683 -4.32 0.000 -.0198815 -.007462
|
c.PS2#c.M1 | .0006044 .000306 1.98 0.048 4.62e-06 .0012041
|
PA | -.0157287 .0043009 -3.66 0.000 -.0241583 -.0072991
|
c.PA#c.M1 | -.0000702 .0001327 -0.53 0.597 -.0003304 .0001899
|
PA2 | .000078 .0000241 3.24 0.001 .0000309 .0001251
|
c.PA2#c.M1 | 5.68e-09 4.43e-07 0.01 0.990 -8.62e-07 8.73e-07
|
M2 | .0477159 .0496802 0.96 0.337 -.0496554 .1450872
C1 | -.9551749 .0743328 -12.85 0.000 -1.100864 -.8094854
C2 | .5121476 .1313358 3.90 0.000 .2547343 .7695609
C3 | -.04401 .0036149 -12.17 0.000 -.051095 -.036925
|
C4 |
2 | -.7148166 .2516482 -2.84 0.005 -1.208038 -.2215951
3 | -.8099169 .2570871 -3.15 0.002 -1.313798 -.3060354
4 | -1.395783 .2511466 -5.56 0.000 -1.888021 -.9035446
5 | -2.199169 .2540965 -8.65 0.000 -2.697189 -1.701149
|
C5 | -.0058189 .0025845 -2.25 0.024 -.0108846 -.0007533
C6 | -.0991881 .0170875 -5.80 0.000 -.1326791 -.0656971
C7 | -.0088507 .001748 -5.06 0.000 -.0122768 -.0054246


Then, I tried a fixed-effect OLS.
xtreg DV c.PS##c.M1 c.PS2##c.M1 c.PA##c.M1 c.PA2##c.M1 M2 C1 C2 C3 i.C4 C5 C6 C7 C8 i.year, fe vce(cl firm)

Fixed-effects (within) regression Number of obs = 15,939
Group variable: firm Number of groups = 4,470

R-sq: Obs per group:
within = 0.1311 min = 1
between = 0.0595 avg = 3.6
overall = 0.0874 max = 37

F(55,4469) = 22.80
corr(u_i, Xb) = -0.3723 Prob > F = 0.0000

(Std. Err. adjusted for 4,470 clusters in firm)
------------------------------------------------------------------------------
| Robust
DV | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
PS | .0152938 .0053779 2.84 0.004 .0047505 .0258371
M1 | -.0007989 .0006537 -1.22 0.222 -.0020806 .0004827
|
c.PS#c.M1 | .0002284 .0003305 0.69 0.490 -.0004196 .0008763
|
PS2 | -.0015522 .0005156 -3.01 0.003 -.002563 -.0005414
|
c.PS2#c.M1 | -.0000821 .0000337 -2.43 0.015 -.0001483 -.0000159
|
PA | -.0019203 .0006831 -2.81 0.005 -.0032596 -.000581
|
c.PA#c.M1 | .0000247 .0000113 2.18 0.029 2.53e-06 .000047
|
PA2 | 9.19e-06 2.18e-06 4.21 0.000 4.91e-06 .0000135
|
c.PA2#c.M1 | -7.38e-08 3.29e-08 -2.24 0.025 -1.38e-07 -9.23e-09
|
M2 | .0217506 .0131305 1.66 0.098 -.0039917 .0474929
C1 | -.1811204 .0148444 -12.20 0.000 -.2102228 -.152018
C2 | -.6921941 .0339614 -20.38 0.000 -.7587753 -.6256128
C3 | -.0159252 .0018845 -8.45 0.000 -.0196199 -.0122306
|
C4 |
2 | -.0198844 .1248094 -0.16 0.873 -.2645727 .2248038
3 | .0355759 .1262371 0.28 0.778 -.2119114 .2830632
4 | -.0718508 .1246124 -0.58 0.564 -.3161528 .1724512
5 | -.1170256 .1245552 -0.94 0.348 -.3612155 .1271643
|
C5 | -.0008741 .0002293 -3.81 0.000 -.0013237 -.0004246
C6 | -.0090821 .0030168 -3.01 0.003 -.0149966 -.0031676
C7 | -.000516 .0003196 -1.61 0.107 -.0011426 .0001107
C8 | -8.38e-06 7.30e-06 -1.15 0.251 -.0000227 5.94e-06

The only difference between the two models is that C8 (8th control variable) is added in OLS, but not in a fractional logit model due to a convergence problem.

H3 is supported, whereas H4 is not supported according to the first result. However, this becomes exactly the opposite when OLS is used.

I am not sure what causes this problem and how to address this.
I am also aware of the very small magnitude of coefficients. DV ranges between 0 and 1, but main independent variables range between 0 and 5,000 even after I divide some of them by 10^6. I will scale them.

I would greatly appreciate it if anyone can give me some suggestions and comments.
Thank you in advance for your help.



Best regards,


Anna

[Clyde Schechter]

You are using two radically different models. The inclusion/exclusion of C8 alone could cause changes like this. But even putting that aside, one model is hierarchical and the other is flat. One model is linear and the other is fractional logistic. One model estimates only within-firm effects, the other measures a weighted average of within- and between- firm effects. Any one of these differences alone could cause the magnitudes of the coefficients to change radically or the signs to reverse. So why would you expect the results to look similar when you have made all of these changes simultaneously?

[Anna Pak]

Dear Clyde,

Thank you very much for your comments. I understand that the different nature of models could cause the signs to change. Since my dependent variable is a proportion, linear regression may not be suitable in some cases. So, I was recommended to employ a multimodel approach to confirm the robustness of my results. It's not difficult to find studies (at least in my field) that employ these two models for robustness checks of their findings when the dependent variable is a proportion. Since the base hypotheses are supported by both models, I expected similar signs for the moderating effects.

If I have to choose one model, would a fractional logit model be a better option?
I prefer a fixed-effect OLS because my data is an unbalanced panel data and the fixed-effect fractional logit model requires a balanced data, to the best of my knowledge.

Thank you always for your comments and suggestions.

[Clyde Schechter]

I would calculated predicted values for both models and compare each to the observed outcomes and pick the one that shows the better fit.

[Guest]

Can anyone explain what would be the explanatory variables for a fractional response model in an unbalanced framework?
Any reply would be of great help ..