Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • mixed model postestimation using predict

    Hello,

    I am using Stata 14. I have a panel dataset - firms and years. I am running a mixed model which includes both fixed and random effects.
    My total sample size is 52,801. The data set has missing values so entire dataset is not used when I run mixed model. The mixed model output shows the #observations used is 40595.

    I ran the "predict ebX1,reffects" command to obtain BLUPS for random component of independent variable X1. Now since the mixed model analysis used 40,595 observations out of the total of 52,801 observations, I assumed that BLUPS would be created for exactly 40,595 observations.
    However, when I summarize ebX1 (command used: summarize ebX1), I get #obs =51032 in the summary table. This #obs is what I do not understand. So now if I run the command as "summ ebX1 if e(sample)" then I get summary statistics for the 40,595 observations. So I am able to get the correct summary statistics but what I do not understand is why were BLUPS calculated for 51,032 (>40,595) observations in the first place?
    Also, if I now want to say plot a histogram of the BLUPS using command "hist ebX1, freq", the graph is plotted for 51,032 observations which are incorrect for me. And I can not use "if e(sample)" with the histogram command. I can create an indicator variable to mark the 40,595 observations and go from there but I am curious why BLUPS were calculated for more observations than that.

    thanks.


  • #2
    Well, you don't say which variable defines your higher level in the model, but let's assume for the sake of discussion that it's firm.

    Remember that the random effect at the firm level is a constant attribute of the firm; it does not vary among observations within firm. Now, if a firm has any observations in the estimation sample at all, a firm-level random effect can be calculated for that firm. And Stata does that and records that same value in every observation for that firm, even if that observation is not, itself, part of the estimation sample. So the only circumstance under which Stata will not calculate the random effect in an observation is if that observation's firm has no observations in the estimation sample. I'm guessing that this doesn't happen very often in your data (although apparently it does happen occasionally since you don't get 52,801 observations for ebX1.)

    Actually, if you want to do a histogram of ebX1, even restricting to e(sample) is not appropriate because each firm is then counted as many times as it appears in the estimation sample. The actual distribution of ebX1 has to be viewed at the firm level. So something like this:

    Code:
    egen flag = tag(firm)
    histogram ebX1 if flag
    In fact pretty much anything you do with ebX1 should be conditioned on -if flag- so that each firm is counted (at most) once.


    Comment


    • #3
      Take a look at the patterns of missing values in your dataset. (help missing and then scroll down to "Useful commands".) You can still get a BLUP under some patterns of missing data for a cluster. See the illustration of that below, which shows examples of circumstances under which you will and under which you won't get a prediction.

      .ÿversionÿ14.2

      .ÿ
      .ÿclearÿ*

      .ÿsetÿmoreÿoff

      .ÿsetÿseedÿ1372490

      .ÿ
      .ÿquietlyÿsetÿobsÿ200

      .ÿgenerateÿintÿpidÿ=ÿ_n

      .ÿgenerateÿdoubleÿuÿ=ÿrnormal()

      .ÿ
      .ÿquietlyÿexpandÿ2

      .ÿbysortÿpid:ÿgenerateÿbyteÿtimÿ=ÿ_n

      .ÿ
      .ÿgenerateÿdoubleÿrspÿ=ÿuÿ+ÿrnormal()

      .ÿ
      .ÿquietlyÿreplaceÿrspÿ=ÿ.ÿinÿ1

      .ÿquietlyÿreplaceÿtimÿ=ÿ.ÿinÿ3

      .ÿquietlyÿreplaceÿtimÿ=ÿ.ÿinÿ5/6

      .ÿ
      .ÿmixedÿrspÿi.timÿ||ÿpid:ÿ,ÿnolrtestÿnolog

      Mixed-effectsÿMLÿregressionÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿÿÿÿÿ=ÿÿÿÿÿÿÿÿ396
      Groupÿvariable:ÿpidÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿgroupsÿÿ=ÿÿÿÿÿÿÿÿ199

      ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿObsÿperÿgroup:
      ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿminÿ=ÿÿÿÿÿÿÿÿÿÿ1
      ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿavgÿ=ÿÿÿÿÿÿÿÿ2.0
      ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿmaxÿ=ÿÿÿÿÿÿÿÿÿÿ2

      ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿWaldÿchi2(1)ÿÿÿÿÿÿ=ÿÿÿÿÿÿÿ5.56
      Logÿlikelihoodÿ=ÿ-675.86184ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿProbÿ>ÿchi2ÿÿÿÿÿÿÿ=ÿÿÿÿÿ0.0184

      ------------------------------------------------------------------------------
      ÿÿÿÿÿÿÿÿÿrspÿ|ÿÿÿÿÿÿCoef.ÿÿÿStd.ÿErr.ÿÿÿÿÿÿzÿÿÿÿP>|z|ÿÿÿÿÿ[95%ÿConf.ÿInterval]
      -------------+----------------------------------------------------------------
      ÿÿÿÿÿÿÿ2.timÿ|ÿÿÿ.2412021ÿÿÿ.1023253ÿÿÿÿÿ2.36ÿÿÿ0.018ÿÿÿÿÿ.0406481ÿÿÿÿ.4417561
      ÿÿÿÿÿÿÿ_consÿ|ÿÿÿ.0155886ÿÿÿ.1017302ÿÿÿÿÿ0.15ÿÿÿ0.878ÿÿÿÿ-.1837989ÿÿÿÿ.2149761
      ------------------------------------------------------------------------------

      ------------------------------------------------------------------------------
      ÿÿRandom-effectsÿParametersÿÿ|ÿÿÿEstimateÿÿÿStd.ÿErr.ÿÿÿÿÿ[95%ÿConf.ÿInterval]
      -----------------------------+------------------------------------------------
      pid:ÿIdentityÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
      ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(_cons)ÿ|ÿÿÿ1.009803ÿÿÿ.1621702ÿÿÿÿÿÿ.7371185ÿÿÿÿ1.383363
      -----------------------------+------------------------------------------------
      ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvar(Residual)ÿ|ÿÿÿÿ1.03397ÿÿÿ.1041779ÿÿÿÿÿÿ.8486823ÿÿÿÿ1.259711
      ------------------------------------------------------------------------------

      .ÿ
      .ÿpredictÿdoubleÿebe,ÿreffects
      (2ÿmissingÿvaluesÿgenerated)

      .ÿ
      .ÿlistÿinÿ1/6,ÿnoobsÿsepby(pid)

      ÿÿ+-------------------------------------------------+
      ÿÿ|ÿpidÿÿÿÿÿÿÿÿÿÿÿÿuÿÿÿtimÿÿÿÿÿÿÿÿÿÿrspÿÿÿÿÿÿÿÿÿebeÿ|
      ÿÿ|-------------------------------------------------|
      ÿÿ|ÿÿÿ1ÿÿÿÿ.41822853ÿÿÿÿÿ1ÿÿÿÿÿÿÿÿÿÿÿÿ.ÿÿÿ1.0644975ÿ|
      ÿÿ|ÿÿÿ1ÿÿÿÿ.41822853ÿÿÿÿÿ2ÿÿÿÿ2.4112619ÿÿÿ1.0644975ÿ|
      ÿÿ|-------------------------------------------------|
      ÿÿ|ÿÿÿ2ÿÿÿ-.32213358ÿÿÿÿÿ.ÿÿÿ-.80480298ÿÿÿÿ.0886456ÿ|
      ÿÿ|ÿÿÿ2ÿÿÿ-.32213358ÿÿÿÿÿ2ÿÿÿÿ.43620344ÿÿÿÿ.0886456ÿ|
      ÿÿ|-------------------------------------------------|
      ÿÿ|ÿÿÿ3ÿÿÿÿÿ.6285637ÿÿÿÿÿ.ÿÿÿÿÿ1.388014ÿÿÿÿÿÿÿÿÿÿÿ.ÿ|
      ÿÿ|ÿÿÿ3ÿÿÿÿÿ.6285637ÿÿÿÿÿ.ÿÿÿÿ.43723975ÿÿÿÿÿÿÿÿÿÿÿ.ÿ|
      ÿÿ+-------------------------------------------------+

      .ÿ
      .ÿexit

      endÿofÿdo-file


      .

      Comment


      • #4
        Thank you Clyde for the elaborate explanation!
        Thank you Joseph for the hands on example, it so helps!

        Comment

        Working...
        X