Problems with qreg and reg, with vce(r), and lots of FE variables - Stata Crashing

[Ben Hoen]

Hello,

I have a dataset of ~ 22,000 cases of home selling prices that I am regressing on ~ 5 independent variables and two vectors of FE variables, one for the sale year-quarter, and another denoting the location of the home (the census block group in which the home is located). The latter consists of approximately 2,400 different "locations". Normally I use areg to absorb the geographic FE, but I have reason to use qreg, with which I enter the FE using the following i.<geographic FE variable>.

Despite that areg runs in seconds on my computer, qreg stalls and never resolves. I have left it chugging away for many hours with no result. Moreover, when I press "X" to stop the process i can sometimes not return and I have to forcibly close Stata. Similarly, if I try to estimate a model with reg, using the same dataset and estimating robust standard errors - vce(r), the same thing happens.

I am running Windows 7 64 bit Pro, with 6 cores and 32 MB of RAM and Stata MP for 6 cores (13.1). My memory settings are as follows:



I have also tried to drop all of the variables in the dataset that are not needed for the regression to ensure the smallest amount of memory is being used when the command is called, but that has not helped.

So, any ideas as to why this is happening and if there is a way to adjust my settings to better accommodate running the qreg or reg command (preferred).

Thanks, in advance,

Ben Hoen
Berklely Lab

[Ben Hoen]

Problems with qreg and reg, with vce(r), and lots of FE variables - Stata Hanging Up Indefinitely...

Hi all,

Trying again on this. Does anyone know what limit I am hitting when I run qreg or reg with a FE variable that has ~ 2,400 values?

Also, to be clearer, The subject line says "Stata Crashing", when, to be clearer, it simply hangs up indefinately (i.e., does not complete the command) and then requires me to close the program via. the Windows Task Manager, because it shows "not responding" when I click on it or try to press the "X" for stop/quit command.

Ben

[Joao Santos Silva]

Dear Ben,

I am not a Stata expert and therefore I have no idea of what limit you may be hitting. Note, however, that estimation of quantiles with such large number of regressors is a very demanding problem and it may take a long time. So, the question I have is this: does Stata really hangs up or is it still working on the problem? When do you open Task Manager, do you see Stata using CPU when it says it is not responding? On a side note, I do not think it is very sensible to estimate over 2000 parameters with just 22000 observations; the number of observations per parameter is quite small.

Best of luck with it,

Joao

[Ben Hoen]

Thanks Joao,

This does help. I will check the things you have suggested/questioned and get back to the list.

Ben