AIC & BIC questions Fitstat output, differences of +/-2

[Julii Brainard]

I am putting together a negative bin. regression model. (And some mlogit models, too, actually)
Example from nbreg below.

(nbreg) I thought to report an indicator of goodness-of-model fit. AIC seemed to do the job.
Fitstat reports 3 different types of AIC.
I read often that a difference of +/- 2 in AIC is not important when comparing models.
QUESTION 1) But which version of AIC does that +/-2 rule pertain to? The largest one in the fitstat printout?

QUESTION 2) Same query about different versions of BIC reported in fitstat, which one does +/-2 rule apply to?

ALSO, I read about different ways to report AIC (see http://www.stanfordphd.com/AIC.html).
QUESTION 3) Is there a correct usual standard way to say I am using the version of AIC that divides by N (6.417 in below example), and not one of the other versions of AIC? Does this smallest value version of AIC have a special name?

QUESTION 4) I read that BIC is preferable when using 'large' datasets. My dataset has ~27,000 observations. Is ~27,000 = 'large'?

Here's hoping for simple answers...
Thanks in advance for any help! -Julii

[Julii Brainard]

Boohoo. I thought this might have a simple easy answer. I may just go for the biggest AIC as the one that +/- 2 rule applies to. Anything else seems unlikely.

[Richard Williams]

With BIC it shouldn't make any difference which formula you use. Further, if you are comparing two models using fitstat, it will tell you how much the BIC stat favors one model over the other.

. webuse nhanes2f, clear

. quietly ologit health weight i.female i.black

. quietly fitstat, save

. quietly ologit health weight i.female i.black height

. fitstat, diff ic

                         |     Current        Saved   Difference 
-------------------------+---------------------------------------
AIC                      |                                       
                     AIC |   30879.519    31333.790     -454.271 
          (divided by N) |       2.988        3.032       -0.044 
-------------------------+---------------------------------------
BIC                      |                                       
           BIC(df=8/7/1) |   30937.465    31384.493     -447.028 
 BIC (based on deviance) |  -64591.952   -64144.924     -447.028 
    BIC' (based on LRX2) |    -628.302     -181.275     -447.028 

Difference of  447.028 in BIC provides very strong support for current model.

.

For AIC, I haven't heard that rule of thumb, but I agree that it probably refers to the larger AIC value.

Incidentally, your fitstat output looks different than mine. I suspect your version is out of date. Make sure that you have the version that comes with spost13_ado. Then update using adoupdate.

Anyone else who wants fitstat (and a lot of other great routines) should type

findit spost13_ado

[Julii Brainard]

Thanks Richard, your replies on this forum are immensely appreciated. I'll try to remember to check which fitstat I have when I get back to work. I only found out about findit & fitstat b/c of one of your posts here. If you're interested, here's one of a gazillion pages giving this +/- 2 rule of thumb about AIC differences when choosing a model. Sorry link is to stackexchange. Not a friendly place.