Issues in convergence of fractional logit model

[Jaya Jumrani]

Dear Stata users,

I'm working on a survey dataset in Stata 11.2 with around 50,000 households. My dependent variable is a proportion variable and as a result I'm using the fractional logit model i.e. the command glm Y X, family(binomial) link(logit) robust where Y is a proportion ranging between 0 and 1 with majority being zeroes. X consists of continuous, dummy variables and some variables which are interactions among those variables. The squared terms of some of these variables are also included.

The problem that I'm facing is that the model is not able to converge and the log pseudolikelihood gets backed up. The independent variables whose squared terms have been incorporated have also been centered. The problem persists even after changing the ml technique to bhhh.

Any help in this regard will be highly appreciated.

Thanks in anticipation.

Best,
Jaya

[Richard Williams]

The -difficult- option sometimes works miracles. Other than that I would suggest starting with a very simple model and then building it up. Maybe you can identify a variable that is causing all the grief.

[Andrew Lover]

Sometimes cycling through multiple ml algorithms helps: eg, -technique(bhhh nr dfp bfgs)- or adding the -difficult- option

Also, take a look at section 4 here: Cummings P (2009) Methods for estimating adjusted risk ratios. Stata Journal 9: 175–196.
However, this might be problematic with svy data....

[Jeff Wooldridge]

This comment may be uninformed, but I wonder if the problem is due to perfectly predicting some zero outcomes based on certain combinations of the covariates. For binary responses, logit, probit, and glm handle this problem and drop the relevant observations. I don't know whether glm has a way of recognizing perfect predictions with fractional responses. I haven't encountered this problem in the examples I've done.

[Jaya Jumrani]

Thanks Dr. Williams. The -difficult- option unfortunately doesn't solve the problem. I did start from a simple model and then started building up on it and the grief causing variables are continuous and interaction (continuous*dummy) variables, which are also theoretically important variables for the model.

[Jaya Jumrani]

Thanks Mr. Andrew.. The command as suggested in "Cummings P (2009) Methods for estimating adjusted risk ratios. Stata Journal 9: 175–196" is applicable but is also leading to the iterations getting backed up. The cycling through multiple ml algorithms is in progress yet.

[Jaya Jumrani]

Thanks Dr. Woolridge. I don't get any such warning messages when I use the glm command. However, when I run the normal logit command I do get the error message of some of the independent variables perfectly predicting the data.

[Maarten Buis]

I don't get any such warning messages when I use the glm command. However, when I run the normal logit command I do get the error message of some of the independent variables perfectly predicting the data.

I suspect that glm won't check for perfect prediction, so I suspect that that is the cause of your problem. I have no idea how to solve it though.

[Jaya Jumrani]

Thanks Dr. Buis. If you happen to come across any solution for the same, please do share.

[Jeff Wooldridge]

As I wrote to Jaya in an email, one should not use the logit (or probit) commands in Stata for fractional response. Those commands automatically transform the dependent variable into a zero-one variable. One should always use the glm command for fractional response, which leaves the Y variable in its original form. Jaya's problem seems to go away when glm is used, which means there is sufficient variation in the outcome that cannot be perfectly predicted by Y.

[Richard Williams]

Jaya's problem seems to go away when glm is used, which means there is sufficient variation in the outcome that cannot be perfectly predicted by Y.

However, Jaya said in her first post that she was using glm. So I assume there was also some other change that solved the problem? If so, what was it?

[Richard Williams]

Jaya's problem seems to go away when glm is used, which means there is sufficient variation in the outcome that cannot be perfectly predicted by Y.

However, Jaya said in her first post that she was using glm. So I assume there was also some other change that solved the problem? If so, what was it? (Just in case I ever need this solution myself!)

[Maarten Buis]

As I wrote to Jaya in an email, one should not use the logit (or probit) commands in Stata for fractional response.

The way I understand the problem is that with glm the fractional logit model does not converge. If logit is used there is an warning message that says there are 0s that are exactly predicted.

Jeff is (obviously) right that logit is not the correct way to estimate a fractional logit. However, the warning message that 0s are exactly predict can be useful: The 0s in logit match the 0s in glm (the 1s in logit don't, they match any value larger than 0 in glm). So the exact prediction of 0s in logit should match the exact prediction of 0s in glm, which could be the reason why glm does not converge.

[Jeff Wooldridge]

Here is what Jaya wrote in her recent email to me: "I don't get any such warning messages when I use the glm command. However, when I run the normal logit command I do get the error message of some of the independent variables perfectly predicting the data. " I assume she made a mistake earlier in describing the problem. I agree that using glm could still have led to a problem, but it doesn't seem to now in Jaya's particular application.

[Richard Williams]

To me that sounds like glm doesn't give an error about perfect prediction, but whatever problem caused her to post in the first place is still there or has been solved some other way. But Jaya is the ultimate authority on this!