Breusch and Pagan LM test for random effects

[Farid Osmanov]

Dear community,

I am deciding among Pooled OLS, Fixed and Random Effects panel models in the presence of first-order autocorrelation ( null hypothesis of Wooldridge test for autocorrelation in panel data is not rejected) . Thus, i have found out that the command . xtregar gives reliable estimates in the presence of AR1. So I run the FE and RE tests ( e.g., . xtregar, re) and then the default Hausman test . hausman fixed random. The null hypothesis is rejected, meaning FE is to be chosen.

1) Am i doing everything correctly till now?

An now i want also to exclude Pooled OLS by running the Breusch and Pagan Lagrangian multiplier test for random effects. Usually it is done with the comand of . xttest0 but it returns an error message last estimates not found r(301)

2) Which command/ procedure would be appropriate in order to exclude Pooled OLS?
I want to mention that there is no heteroscedasticity problem, only the autocorrelation is the issue.

Thank you a lot for support!

Kind Regards
Farid

[Carlo Lazzaro]

Farid:
welcome to this forum.
1) -xttest0- works after -xtreg,re- estimation only (please see the following dope toy-example, not to be followed for serious reserch):

. use "https://www.stata-press.com/data/r16/nlswork.dta"
(National Longitudinal Survey.  Young Women 14-26 years of age in 1968)

. reg ln_wage age

      Source |       SS           df       MS      Number of obs   =    28,510
-------------+----------------------------------   F(1, 28508)     =   2391.00
       Model |   504.26878         1   504.26878   Prob > F        =    0.0000
    Residual |  6012.42137    28,508  .210902953   R-squared       =    0.0774
-------------+----------------------------------   Adj R-squared   =    0.0773
       Total |  6516.69015    28,509  .228583611   Root MSE        =    .45924

------------------------------------------------------------------------------
     ln_wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |   .0198485   .0004059    48.90   0.000     .0190529    .0206441
       _cons |   1.098442   .0120996    90.78   0.000     1.074726    1.122158
------------------------------------------------------------------------------

. xttest0
last estimates not found
r(301);

. xtreg ln_wage age, re

Random-effects GLS regression                   Number of obs     =     28,510
Group variable: idcode                          Number of groups  =      4,710

R-sq:                                           Obs per group:
     within  = 0.1026                                         min =          1
     between = 0.0877                                         avg =        6.1
     overall = 0.0774                                         max =         15

                                                Wald chi2(1)      =    3140.35
corr(u_i, X)   = 0 (assumed)                    Prob > chi2       =     0.0000

------------------------------------------------------------------------------
     ln_wage |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |   .0185667   .0003313    56.04   0.000     .0179174    .0192161
       _cons |   1.120439   .0112038   100.01   0.000      1.09848    1.142398
-------------+----------------------------------------------------------------
     sigma_u |  .36972456
     sigma_e |  .30349389
         rho |  .59743613   (fraction of variance due to u_i)
------------------------------------------------------------------------------

. xttest0

Breusch and Pagan Lagrangian multiplier test for random effects

        ln_wage[idcode,t] = Xb + u[idcode] + e[idcode,t]

        Estimated results:
                         |       Var     sd = sqrt(Var)
                ---------+-----------------------------
                 ln_wage |   .2285836       .4781042
                       e |   .0921085       .3034939
                       u |   .1366963       .3697246

        Test:   Var(u) = 0
                             chibar2(01) = 28235.40
                          Prob > chibar2 =   0.0000

.

2) the evidence of a group-wise effect is enough to switch from pooled OLS to -xt- suite.

[Farid Osmanov]

Dear Carlo,

Thank you for your explanation. However, i want to clarify the following:

Regarding my question

1) Am i doing everything correctly till now?

i was asking if my procedure of running pooled OLS first in order to check for OLS assumptions ( e.g., Normality, Heteroscedasticity , Autocorrelation) and then running the FE and RE by considering the detected violations ( in this case Autocorrelation ) and then using the default Hausman test is correct?

and regarding your point

2) the evidence of a group-wise effect is enough to switch from pooled OLS to -xt- suite.

: why do you mean under "evidence of groupwise effect" ? what would be the correct command from Stata to test that "groupwise effect" after running the RE and FE with . xtregar command?

I am very new to the panel models, that is why i might need more clarifications.

Thank you once more!

Kind Regards
Farid

[Carlo Lazzaro]

Farid:
1) put differently, pooled OLS is not the 1st approach to follow when you have a panel dataset. In addition, if you detect heteroskedasticity after OLS you should go -robust- (.vce(cluster clusterid) if serial correlation comes alive). Normality is a weak requirements for rsidual distribution only. Hence, -xtreg- is not the answer if you detect autococorelation in your residual distribution, because you can handle this issue in -regress-, too, just invoking -vce(cluster clusterid)- standard errors.
Hence, I would start with -xtreg- and, since non-default standard errors have to be invoked due to serial correlation of the epsilon, -hausman- should be replaced by the community-contributed module -xtoverid- (just type -search xtoverid- from within Stata to spot and install it).
2) as far as your second point, hopefully the following toy-example can shed some light (-xttest0- outcome tells us that there's a panel-wise effect; -xtoverid- outcome tells us that -fe- is the way to go):

use "https://www.stata-press.com/data/r16/nlswork.dta"
. xtreg ln_wage c.age c.tenure, re vce(cluster idcode)

Random-effects GLS regression                   Number of obs     =     28,101
Group variable: idcode                          Number of groups  =      4,699

R-sq:                                           Obs per group:
     within  = 0.1291                                         min =          1
     between = 0.2010                                         avg =        6.0
     overall = 0.1499                                         max =         15

                                                Wald chi2(2)      =    1954.25
corr(u_i, X)   = 0 (assumed)                    Prob > chi2       =     0.0000

                             (Std. Err. adjusted for 4,699 clusters in idcode)
------------------------------------------------------------------------------
             |               Robust
     ln_wage |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |   .0116567   .0006696    17.41   0.000     .0103442    .0129691
      tenure |   .0254571   .0011261    22.61   0.000       .02325    .0276643
       _cons |    1.25258   .0174563    71.76   0.000     1.218366    1.286794
-------------+----------------------------------------------------------------
     sigma_u |  .33849709
     sigma_e |  .29808194
         rho |  .56323316   (fraction of variance due to u_i)
------------------------------------------------------------------------------

. xttest0

Breusch and Pagan Lagrangian multiplier test for random effects

        ln_wage[idcode,t] = Xb + u[idcode] + e[idcode,t]

        Estimated results:
                         |       Var     sd = sqrt(Var)
                ---------+-----------------------------
                 ln_wage |   .2282906       .4777976
                       e |   .0888528       .2980819
                       u |   .1145803       .3384971

        Test:   Var(u) = 0
                             chibar2(01) = 24019.62
                          Prob > chibar2 =   0.0000

. xtoverid

Test of overidentifying restrictions: fixed vs random effects
Cross-section time-series model: xtreg re  robust cluster(idcode)
Sargan-Hansen statistic 318.394  Chi-sq(2)    P-value = 0.0000

.

[Farid Osmanov]

Your explanation is highly appreciated, Carlo! I have implemented the procedure that you have suggested already(e.g., . fe/re cluster (country) ), however, only when both autocorrelation and heteroscedasticity are violated, since the option with .vce(cluster clusterid) or . fe/re cluster (country) controls for both .

1) I was wondering if there is a command which would control only for autocorrelation and then it is possible to test for a panel-wise effect after command? theoretically according to Hoechle, the command for controlling only for autocorrelation is . xtregar

2) The issue with .cluster (country) is that the command uses robust standard errors, which is needed in the case of heteroscedasticity. Is using robust errors is also justified when there is only the autocorrelation problem and no heteroscedasticity?

Thanks again!

Kind Regards
Farid

[Farid Osmanov]

Dear Carlo Lazzaro

I have another question regarding Ramsey RESET Test. My regression model using the collected raw data has failed the test, implying functional misspecification. However, i have adopted some data transformation, by using square roots of the original variables. And now the null hypothesis of the Ramsey's Test cannot be rejected, meaning the structural form is correct.

So my question is: is such a solution appropriate or not? Is it acceptable to use data transformation in this case?

Thank you in advance!

Kind Regards,
Farid

[Carlo Lazzaro]

Farid:
yes, it is.

[Farid Osmanov]

Dear Carlo Lazzaro

After running Pooled OLS model i detected autocorrelation and heteroscedasticity violations. Therefore, i have opted for xtreg, fe/re cluster() and then in order to decide between them i used rhausman and xtoverid commands. After implementing the rhausman and xtoverid commands, i receive contradicting results.

Thus, my question is which results should i trust and go for?


rhausman fixed random, cluster
bootstrap in progress
----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5
.................................................. 50
.................................................. 100
--------------------------------------------------------------------------------
Cluster-Robust Hausman Test
(based on 100 bootstrap repetitions)

b1: obtained from xtreg sqrtROA sqrtSTD sqrtLTD SIZE GROWTH, fe cluster( companyID)
b2: obtained from xtreg sqrtROA sqrtSTD sqrtLTD SIZE GROWTH, re cluster( companyID)

Test: Ho: difference in coefficients not systematic

chi2(4) = (b1-b2)' * [V_bootstrapped(b1-b2)]^(-1) * (b1-b2)
= 7.76
Prob>chi2 = 0.1010




xtoverid

Test of overidentifying restrictions: fixed vs random effects
Cross-section time-series model: xtreg re robust cluster(companyID)
Sargan-Hansen statistic 16.515 Chi-sq(4) P-value = 0.0024

Thank you in advance!

Kind Regards,
Farid Osmanov

[Carlo Lazzaro]

Farid:
this happens from time to time.
Take a look at: https://www.stata.com/statalist/arch.../msg00557.html

[Farid Osmanov]

Carlo Lazzaro

I have read the link where the robust Hausman Test is run manually, however, i get the contradicting results after the default commands responsible for the respective test. What would you suggest me to do?

Use xtoverid command or rhausman command?

[Carlo Lazzaro]

Farid:
comparing their underlying theoretical assumptions, I would trust -xtoverid- outcome.

[Farid Osmanov]

Carlo Lazzaro

the test is also a good choice the panel data is unbalanced?

Thank you so much for your help till now!

Kind Regards
Farid Osmanov

[Carlo Lazzaro]

Farid:
yes, it is.

[Farid Osmanov]

Dear Carlo Lazzaro

Could you, please, suggest how the estimated coefficient values can be interpreted when sqrt transformation of the original variables are used running the xtreg, fe/re cluster() ?

I know, for example, how interpretation can be made following a logarithmic transformation to one or more of the dependent or independent variables ( however this is not applicable in my research) . But what about the square root? Or both, logarithm and square root?

The models are as below:


sqrtROE(dependent variable) = sqrtSTD (independent variable)

sqrt-sqrt model

or

lnROE = sqrtSTD

log-sqrt model

Any suggestions?

Thank you a lot!

Kind Regards,
Farid Osmanov

[Carlo Lazzaro]

Farid:
I've never used -sqrt- transformation, so I cannot help.
See, if interested, https://quantifyinghealth.com/square...ransformation/