Logit does not achieve convergence. Is my solution sensible and are there alternatives?

[Joe Duffy]

My first post, so sorry if anything is unclear.

My problem is that logit and probit models are failing to converge. I have a solution and wanted to check why this worked, as well as get a better of idea of why I have this problem in the first place. I searched the forum archives, and couldn't find anything very helpful.


I am running logit and probit models on a pooled UK dataset of households (a combination of the Survey of English Housing and House Price Index data). My dependent variable is a dummy variable indicating whether the household owns their house or not (the alternative being to rent). The mean of this dummy variable is 0.87 (so 87% are 1's and 13% are 0) and my sample size is 103,821. I have 29 controls and a constant.

This is my probit input, logit is analagous:

 logit own hpl1 rnl1 crrl1 voll1 rntl1 w_inc savings age old npers ndepch R2 R3 R4 R5 R6 R7 R8 R9 R10 G2 G3 G4 G5 G6 G7 G8 G9 St_5

When entering this code I get many 100s of iterations, get bored and press break after a while. If I use instead

logit own hpl1 rnl1 crrl1 voll1 rntl1 w_inc savings age old npers ndepch R2 R3 R4 R5 R6 R7 R8 R9 R10 G2 G3 G4 G5 G6 G7 G8 G9 St_5, iter(20)

the following message is received:

"Note: 0 failures and 119 successes completely determined.
Warning: convergence not achieved"


I assume the latter line is the more relevant one. My (shaky) understanding of why this would happen is that one or more of the coefficients estimated in my regression is not converging to a single value. In other words when stata is using whatever numerical method it uses to maximise likelihood, there is no single value that maxmises likelihood (or rather there are several). Question 1 - have I got this completely wrong/is there something I'm missing here?


I played around with iterations a bit, and found that the coefficients never changed between 10-50 iterations (and all the others in between), but I always got the error message above. My understanding is, however, that this does not make these results robust in any way. I mention it anyway in case it's helpful.


I was advised to play around with my regression to see if I could identify the variable that 'caused' this convergence. I found that when I omitted the weekly income variable (w_inc) there were no problems with convergence. As I wanted to include weekly income, and essentially on a whim, I thought it might help to get rid of some of the extreme values of weekly income. On removing the top 250 people - who all earn above £6000 ($8952) a week and of whom 22 have a value of 0 and 228 have a value of 1 for my dependent variable - I found that my logit and probit worked fine. It confirmed the sign and significance of coefficients estimated from a linear probability model, as well as when I pre-defined the number of iterations. Note I still get 16 successes pre-determined.

Question 2: Is this a good fix? What likely problems might this cause? Are there better solutions I might try?

Sorry if anything is unclear, and if it would be helpful for me to give any more information, more detail on my commands, upload some graphs/distributions or data then I can do so.

Thanks in advance,
Joe

[Carlo Lazzaro]

Joe:
my take is that the first line of the message

"Note: 0 failures and 119 successes completely determined.

is more important than the second one, because it points you out to a necessary scrutiny of your dataset.
As far as your Question 2 is concerned, you are seemingly intended to throw away some information conveyed by your data, Whether this is a good fix for your problem is difficult to say.

[Joe Duffy]

Thanks for replying Carlo,

You would suggest that I should scrutinise my data set in more detail, in particular to try and identify what linear combination of my covariates perfectly predicts the 119 successes? I suppose any action taken on this will depend on what I find.



Re Question 2, I suppose it might be valuable for me to speculate what direction any bias caused by their exclusion might be, just in order to put any results I do find in context (i.e. that I have thrown away data). Do you think it would add anything to look at how the dependent variable is correlated with all of the independent variables for those earning above and below the cut-off for income.

Say if the correlation between age and my dependent variable was higher for the excluded group I might suggest that my estimated coefficient on age could be downward biased compared with the 'true' population value? As I type this it seems flimsy - simple correlations by themselves won't take into account the way the different independent variables are correlated with one another (which the regression would).

An alternative might be to compare the coefficients of a simple OLS regression on both groups? Although this has other problems.

Failing this, my results should at least be representative of those earning less than £6000 a week (so less than £312,000 a year), in the same way that if I excluded males from my analysis my results would be representative of females - for example?


Thanks again,
Joe

[William Lisowski]

Joe - I seem to recall having found it useful to use logarithms when using income in a model. Income is typically so skewed, as you found, that the high values exert undue leverage on the estimates. Also, the distribution of all but the highest incomes are lognormal. And it can be argued that income's effects are proportional, not additive: the effect of an additional £60 a week is more to one with £600 a week than it is to one with £6000 a week. So you might try replacing w_inc with something like ln(w_inc+1). Or abandon theory entirely and use lnskew0 to find a zero-skew transformation for your w_inc data. One or the other of those approaches may render problem less ill-conditioned.

[Richard Williams]

You may wish to look at

http://www.stata.com/support/faqs/st...ic-regression/

[Carlo Lazzaro]

Joe:
as an aside to Others' helpful recomendations, my take is that the main issue there has more to do with theory than statistics, in that your research strategy depends on its original goal. Put differently, when you wrote out your statistical plan, did you forecast to focus on a subsample of your data (i.e.,

those earning less than £6000 a week (so less than £312,000 a year)

) or to undertake sub-group analyses (i.e. affluent males vs affluent females)? Otherwise, I would recommend you to go back to square one and consider the main outcome of your research endeavour and analyze your dataset accordingly. Again, you should have selected statistic tools and related analysis strategy from the very start in your statistical plan.
Conversely, In case you are doing exploratory analysis (whatever this may mean) and your are trying different approaches to analyse your data, you should randomly split your dataset and use one of those sections to generate hypotheses and test models and the other one to validate your research strategy on the remaining part of your dataset. As far as exploratory data analysis is concerned, you may be interested in David C. Hoaglin (Editor), Frederick Mosteller (Editor), John W. Tukey (Editor). Understanding Robust and Exploratory Data Analysis. Wiley, 2000.
http://eu.wiley.com/WileyCDA/WileyTi...471384917.html

[Joe Duffy]

Thanks all, you've given me a lot of different things to look over. I'll post back if I'm able to make any progress/find anything interesting.

Happy Easter!

[Carlo Lazzaro]

Thanks Joe:
the same to you and yours.

[Joe Duffy]

Just to post what I found in case someone else has a similar issue.


Conveniently - if somewhat embarrassing - using log weekly income as a control instead of weekly income meant that I had no convergence problems.

Thanks all again, this problem has certainly helped me learn a lot more about how MLE works and exploratory data analysis!