Maximum likelihood estimation with weights

[Llorenç Quintó]

Dear statalisters

I implemented my own conditional logit maximum likelihood estimator (MLE) using Stata’s ML language. To compute the overall value of the log likelihood, I used the following trick mentioned in the technical note in the Stata Manuals (page 1336) and also in the book "Maximum Likelihood Estimation with Stata", 3rd Edition, Gould, W. J. Pitblado, and W. Sribney. 2006, Stata Press:

tempvar last
by groupid: gen byte `last' = (_n == _N)
mlsum `lnL' = ... if `last' == 1

It works fine and results agree with those obtained with command clogit. However, it seems that it does not work with weights and produces the following error:

mlsum must be over entire estimation sample when weights
are specified
r(499);


See the example below:

My likelihood evaluator is

program myclog
args todo b lnL
version 13
tempvar denom xb p last
mleval `xb' = `b', eq(1)
qui {
bysort $MY_panel: egen double `denom' = sum(exp(`xb'))
gen double `p' = exp(`xb')/`denom'
by $MY_panel: gen byte `last' = (_n == _N)
mlsum `lnL' = $ML_y1*log(`p') if `last' == 1
}
end


And the result is

. webuse lowbirth2, clear
(Applied Logistic Regression, Hosmer & Lemeshow)

. global MY_panel pairid

. ml model d0 myclog (xb: low = lwt smoke ht, noconstant)

. ml maximize, eform(Odds Ratio) nolog

Number of obs = 112
Wald chi2(3) = 10.06
Log likelihood = -31.794758 Prob > chi2 = 0.0180

------------------------------------------------------------------------------
low | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
lwt | .9850332 .0070493 -2.11 0.035 .9713132 .9989469
smoke | 3.280854 1.525329 2.56 0.011 1.319009 8.160674
ht | 5.468604 4.540713 2.05 0.041 1.074245 27.83874
------------------------------------------------------------------------------

generate weights for the example (the same for all observations in a group) and estimate the model with weights

. gen sweights = uniform()

. bysort pairid: replace sweights = sweights[1]
(56 real changes made)

. ml model d0 myclog (xb: low = lwt smoke ht, noconstant) [iw=sweights]

. ml maximize, eform(Odds Ratio) nolog
mlsum must be over entire estimation sample when weights
are specified
r(499);


Any help would be greatly appreciated.
Thank you very much

[Jeff Pitblado (StataCorp)]

In the help file for mleval you should see that it takes the option noweight.

This means that you will have to apply the weights directly in the likelihood calculation.
Here is a modified version of your evaluator that does this.

program myclog
        version 13
        args todo b lnL
        tempvar denom xb p last
        mleval `xb' = `b', eq(1)
qui {   
        bysort $MY_panel: egen double `denom' = sum(exp(`xb'))
        gen double `p' = exp(`xb')/`denom'
        by $MY_panel: gen byte `last' = (_n == _N)
        * directly weighting the group level likelihoods
        mlsum `lnL' = $ML_w*$ML_y1*log(`p') if `last' == 1 , noweight
}       
end

[Llorenç Quintó]

Thank you very much for your help