How to compare model performance of SUR, RE and Random Coefficient in Stata

[Melody Brown]

Hi,

I have a question about comparing the model performance of the random intercept model, random coefficient model as well as a seemingly unrelated regression model in Stata.

Below are three examples of random intercept model, random coefficient model and SUR model. In random intercept model, there is R-sq value. However, in the random coefficient model, we don't have this value. The random intercept model doesn't have log likelihood value either. So in this case, how can we compare the performance of these three models?

[Carlo Lazzaro]

Melody:
set aside -sureg- for a while, you can compare -mixed- and -xtreg- with random effect specification this way:

use https://www.stata-press.com/data/r17/pig.dta
. mixed weight i.week||id:

Performing EM optimization ...

Performing gradient-based optimization:
Iteration 0:   log likelihood = -1007.0675 
Iteration 1:   log likelihood = -1007.0675 

Computing standard errors ...

Mixed-effects ML regression                     Number of obs     =        432
Group variable: id                              Number of groups  =         48
                                                Obs per group:
                                                              min =          9
                                                              avg =        9.0
                                                              max =          9
                                                Wald chi2(8)      =   26412.22
Log likelihood = -1007.0675                     Prob > chi2       =     0.0000

------------------------------------------------------------------------------
      weight | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
        week |
          2  |   6.760417   .4187014    16.15   0.000     5.939777    7.581056
          3  |   13.84375   .4187014    33.06   0.000     13.02311    14.66439
          4  |     19.375   .4187014    46.27   0.000     18.55436    20.19564
          5  |   25.13542   .4187014    60.03   0.000     24.31478    25.95606
          6  |   31.42708   .4187014    75.06   0.000     30.60644    32.24772
          7  |    37.4375   .4187014    89.41   0.000     36.61686    38.25814
          8  |   44.28125   .4187014   105.76   0.000     43.46061    45.10189
          9  |   50.19792   .4187014   119.89   0.000     49.37728    51.01856
             |
       _cons |   25.02083   .6298893    39.72   0.000     23.78627    26.25539
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects parameters  |   Estimate   Std. err.     [95% conf. interval]
-----------------------------+------------------------------------------------
id: Identity                 |
                  var(_cons) |   14.83704    3.12421      9.819998     22.4173
-----------------------------+------------------------------------------------
               var(Residual) |   4.207462   .3036474      3.652498    4.846747
------------------------------------------------------------------------------
LR test vs. linear model: chibar2(01) = 484.84        Prob >= chibar2 = 0.0000

.

. xtreg weight i.week, mle

Fitting constant-only model:
Iteration 0:   log likelihood = -1827.2124
Iteration 1:   log likelihood = -1827.2118

Fitting full model:
Iteration 0:   log likelihood = -1008.0493
Iteration 1:   log likelihood = -1007.0894
Iteration 2:   log likelihood = -1007.0675
Iteration 3:   log likelihood = -1007.0675

Random-effects ML regression                        Number of obs    =     432
Group variable: id                                  Number of groups =      48

Random effects u_i ~ Gaussian                       Obs per group:
                                                                 min =       9
                                                                 avg =     9.0
                                                                 max =       9

                                                    LR chi2(8)       = 1640.29
Log likelihood = -1007.0675                         Prob > chi2      =  0.0000

------------------------------------------------------------------------------
      weight | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
        week |
          2  |   6.760417   .4187015    16.15   0.000     5.939777    7.581056
          3  |   13.84375   .4187015    33.06   0.000     13.02311    14.66439
          4  |     19.375   .4187015    46.27   0.000     18.55436    20.19564
          5  |   25.13542   .4187015    60.03   0.000     24.31478    25.95606
          6  |   31.42708   .4187015    75.06   0.000     30.60644    32.24772
          7  |    37.4375   .4187015    89.41   0.000     36.61686    38.25814
          8  |   44.28125   .4187015   105.76   0.000     43.46061    45.10189
          9  |   50.19792   .4187015   119.89   0.000     49.37728    51.01856
             |
       _cons |   25.02083    .629889    39.72   0.000     23.78627    26.25539
-------------+----------------------------------------------------------------
    /sigma_u |   3.851886   .4055422                      3.133686    4.734688
    /sigma_e |    2.05121   .0740167                      1.911151    2.201533
         rho |   .7790719    .038439                      .6968047    .8468207
------------------------------------------------------------------------------
LR test of sigma_u=0: chibar2(01) = 484.84             Prob >= chibar2 = 0.000

.

As you can see, both codes give back identical results.

[Melody Brown]

Melody:
set aside -sureg- for a while, you can compare -mixed- and -xtreg- with random effect specification this way:

use https://www.stata-press.com/data/r17/pig.dta
. mixed weight i.week||id:

Performing EM optimization ...

Performing gradient-based optimization:
Iteration 0: log likelihood = -1007.0675
Iteration 1: log likelihood = -1007.0675

Computing standard errors ...

Mixed-effects ML regression Number of obs = 432
Group variable: id Number of groups = 48
Obs per group:
min = 9
avg = 9.0
max = 9
Wald chi2(8) = 26412.22
Log likelihood = -1007.0675 Prob > chi2 = 0.0000

------------------------------------------------------------------------------
weight | Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
week |
2 | 6.760417 .4187014 16.15 0.000 5.939777 7.581056
3 | 13.84375 .4187014 33.06 0.000 13.02311 14.66439
4 | 19.375 .4187014 46.27 0.000 18.55436 20.19564
5 | 25.13542 .4187014 60.03 0.000 24.31478 25.95606
6 | 31.42708 .4187014 75.06 0.000 30.60644 32.24772
7 | 37.4375 .4187014 89.41 0.000 36.61686 38.25814
8 | 44.28125 .4187014 105.76 0.000 43.46061 45.10189
9 | 50.19792 .4187014 119.89 0.000 49.37728 51.01856
|
_cons | 25.02083 .6298893 39.72 0.000 23.78627 26.25539
------------------------------------------------------------------------------

------------------------------------------------------------------------------
Random-effects parameters | Estimate Std. err. [95% conf. interval]
-----------------------------+------------------------------------------------
id: Identity |
var(_cons) | 14.83704 3.12421 9.819998 22.4173
-----------------------------+------------------------------------------------
var(Residual) | 4.207462 .3036474 3.652498 4.846747
------------------------------------------------------------------------------
LR test vs. linear model: chibar2(01) = 484.84 Prob >= chibar2 = 0.0000

.

. xtreg weight i.week, mle

Fitting constant-only model:
Iteration 0: log likelihood = -1827.2124
Iteration 1: log likelihood = -1827.2118

Fitting full model:
Iteration 0: log likelihood = -1008.0493
Iteration 1: log likelihood = -1007.0894
Iteration 2: log likelihood = -1007.0675
Iteration 3: log likelihood = -1007.0675

Random-effects ML regression Number of obs = 432
Group variable: id Number of groups = 48

Random effects u_i ~ Gaussian Obs per group:
min = 9
avg = 9.0
max = 9

LR chi2(8) = 1640.29
Log likelihood = -1007.0675 Prob > chi2 = 0.0000

------------------------------------------------------------------------------
weight | Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
week |
2 | 6.760417 .4187015 16.15 0.000 5.939777 7.581056
3 | 13.84375 .4187015 33.06 0.000 13.02311 14.66439
4 | 19.375 .4187015 46.27 0.000 18.55436 20.19564
5 | 25.13542 .4187015 60.03 0.000 24.31478 25.95606
6 | 31.42708 .4187015 75.06 0.000 30.60644 32.24772
7 | 37.4375 .4187015 89.41 0.000 36.61686 38.25814
8 | 44.28125 .4187015 105.76 0.000 43.46061 45.10189
9 | 50.19792 .4187015 119.89 0.000 49.37728 51.01856
|
_cons | 25.02083 .629889 39.72 0.000 23.78627 26.25539
-------------+----------------------------------------------------------------
/sigma_u | 3.851886 .4055422 3.133686 4.734688
/sigma_e | 2.05121 .0740167 1.911151 2.201533
rho | .7790719 .038439 .6968047 .8468207
------------------------------------------------------------------------------
LR test of sigma_u=0: chibar2(01) = 484.84 Prob >= chibar2 = 0.000

.

As you can see, both codes give back identical results.

Hi Carlo,

Thank you so much for your kind reply. The method helps a lot.

Another side question: Do you know if it's possible to combine random coefficient model and SUR model in Stata? I checked there is a xtsur command, but not sure if it's random coefficients+sur or random intercept+sur.

[Carlo Lazzaro]

Melody:
sorry, but I'm unfamiliar with the community-contributed module -xtsur-.

[Melody Brown]

Melody:
sorry, but I'm unfamiliar with the community-contributed module -xtsur-.

No problem Carlo. Thank you all the same.