Computation Problem for Fitting Fixed Effects Regression with Large Data Set

[Mingyu Qi]

Hello everyone, I am trying to fit a fixed effects regression using reghdfe on a data set with 40 million observations with around 2000 fixed effects. The regression has been running for over 3 hours so far, does it seem normal to you? Are there any tips that you could provide to make the regressions fitting faster?

Additionally, I currently using a binary outcome in the regression. I think it should be fine since I have a large sample. I do want to try other function forms (e.g., logit) though. Is there any suggestion on the command that I should use for fitting logistic regression with a large number of fixed effects?

Thank you very much!

[Andrew Musau]

reghdfe is from https://github.com/sergiocorreia/reghdfe, as you are asked to explain (FAQ Advice #12). This thread may help: https://www.statalist.org/forums/for...dfe-10x-faster.

[Andrew Musau]

You could also update Stata and try xtreg with the -fe- and -absorb()- options.

absorb(varlist[, method]) is part of StataNow. It specifies the categorical variables to be absorbed. The results are adjusted as if indicator variables for each level of each variable in varlist were included in the regression.

[Mingyu Qi]

You could also update Stata and try xtreg with the -fe- and -absorb()- options.

Thank you Andrew, and I apologize for not explaining the package I used. The two suggestions you offered are very helpful, however, I am using Stata on a Linux server. Neither using Julia nor updating to StataNow is possible.