exlogistic limits

[Phil Bromiley]

I'm thinking of using exlogistic but it seems to be restricted to extremely small problems. Two gig is the maximum the program will accept which accords with the documentation. I've tried running this:

clear
set max_memory 5g 

set obs 28
g x1=rnormal()
g x2=rnormal()
g e=rnormal()

g y1= x1 - x2 +   e
g y2=(y1>0)
reg y2 x1 x2
logit y2 x1 x2

exlogistic y2 x1 x2,  memory(2g)

And I get:


. exlogistic y2 x1 x2, memory(2g)

Enumerating sample-space combinations:
observation 1: enumerations = 2
observation 2: enumerations = 4
observation 3: enumerations = 8
observation 4: enumerations = 16
...
observation 24: enumerations = 10965204
observation 25: enumerations = 18126760
exceeded memory limit of 2048.0M bytes; use the memory() option to increase the memory
limit


It will run if I keep the observations very low (say 24). Am I reading this as suggesting that the maximum possible data is in the mid 20's?

If this is the case, it might be worth a comment in the documentation.

[Richard Williams]

The manual has examples with 47 cases. I don't really understand exlogistic but I think the number of values the xs can have may be as or more important than the N. Here is my tweaking of your code:

clear all

set seed 123
set obs 28
g x1=rnormal()
g x2=rnormal()
g e=rnormal()

g y1= x1 - x2 +   e
g y2=(y1>0)

egen xx1 = cut(x1), group(10)
egen xx2 = cut(x2), group(10)

reg y2 xx1 xx2
logit y2 xx1 xx2

exlogistic y2 xx1 xx2,  memory(2g) coef

My cut approach probably isn't the best, but one way or another think about a good way to reduce the number of possible values (e.g. if you have values like 9.11, 9.13, 9.14, 9.17) consider rounding to 9.1 and 9.2).

Also consider using the convars option. Use it multiple times if necessary. See the help (the manual entry explains it pretty well).

[Phil Bromiley]

Thanks! A logit with small sample properties seemed like a useful estimator.

I hadn't thought about the number of values in each explanatory variable.

Phil

[Richard Williams]

There are other methods besides exlogistic you may want to consider. I summarize some of them in

http://www3.nd.edu/~rwilliam/stats3/RareEvents.pdf