Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • mlogit Postestimation Stata 14 vs Stata 15 (bug?)

    Hello,

    Is anyone aware of changes Stata has made from v14 to v15 in terms of post-estimation (margins command) after running mlogit? The SEs for the estimated marginal effects change when I run the exact same code on Stata 14 and 15. The point estimates don't change.

    To be specific, I'm estimating the marginal effects of different time periods after running mlogit where the response has 4 levels. In Stata 14, the SEs for the marginal effects consistently jump up and down in an orderly fashion and as a result my confidence intervals do this:

    Time period #1: larger SE --> wide interval
    Time period #2: shorter SE --> short interval
    Time period #3: larger SE --> wide interval
    Time period#4: shorter SE --> short interval
    ...

    It is a very obvious pattern, and can't be correct. The N's time periods are similar. When I run the exact same code in Stata 15, this issue appears to be fixed.

    Here is some skeleton code:

    Step 1: Run mlogit (SEs in this output look fine)
    Code:
     mlogit response i.t#treatment controls [pw=weight], cluster(clusters) robust
    Step 2: Run margins (SEs in this output exhibit alternating pattern in Stata 14, not Stata 15)
    Code:
     margins t if treatment==1 & e(sample)==1, at(treatment=(0 1))
    Any hints would be greatly appreciated.

    Thanks!
    Alex

  • #2
    Short reply

    It appears that margins in Stata 14 has a bug that was fixed in
    Stata 15. We believe this bug was fixed in the 06nov2017 update where
    margins was made faster as a result of adding support for
    analytical derivatives of nonlinear predictions after more estimation
    commands. Before this update, mlogit, ologit,
    oprobit, logit, and probit were the only commands
    that supported analytical derivatives for their probability predictions.
    In the 06nov2017 update, as part of the speed-up, the code using
    analytical derivatives was changed to better handle when factor
    variables were specified in the at() option.


    Longer reply

    margins, in this case, is using the delta method to compute
    standard errors. In short, the VCE computed by margins is

    Code:
    J * e(V) * J'
    where J is the Jacobian matrix (matrix of first derivatives of
    the predictive margins with respect to each model parameter).

    The mlogit point estimates and their standard errors are the
    same, so there must be a difference in the Jacobian matrix computed by
    margins. A quick matrix list of r(Jacobian) after
    margins will show that Stata 14 is miscomputing some of the
    columns of the Jacobian.

    margins has special code to compute analytical derivatives for
    mlogit. You can turn this off with option numerical (not
    documented). margins with the numerical option in Stata
    14, will be slower, but should reproduce the correct results reported in
    Stata 15.

    The bug appears to happen under the following circumstances:

    1. A factor variable participates in one interaction term and in no
    other lower order terms (such as main effects).

    2. The factor is specified in an at() option instead of in
    marginlist.

    An alternative to using option numerical is to put the factor
    variable in marginlist.

    Code:
    margins t#i(0 1).treatment if treatment==1

    Comment


    • #3
      Thanks for the response, Jeff. Is mlogit the only affected command? I tested logit with similar code and the SEs appear to be fine.

      Comment


      • #4
        My mistake.

        margins' original support for analytical derivatives of
        predictions was limited to predicted probabilities after mlogit,
        ologit, and oprobit.

        Support for logit and probit was added in the 06nov2017
        update, and is also when the above mentioned margins bug was
        fixed.

        Comment

        Working...
        X