Area Under Curve (AUC) after random Effects models

[Nabil Aly]

I would like to know how can I calculate AUC after I run a logit model using runmlwin command through STATA.

Kind regards

[Clyde Schechter]

Well, I am not familiar with the -runmlwin- command; it is not part of official Stata. But to get an ROC curve all you need to do is apply whatever facilities -runmlwin- offers you to obtain predicted values from your random effects model. Then the -roctab- command will do the job. -help roctab-.

[Nabil Aly]

Dear Clyde,
Thank you for your reply.
Actually, I tried to run the roctab command: roctab refvar classvar , but I'm not sure what the classvar variable is or how it can be created.

Kind regards

[Nabil Aly]

I tried to run the lroc command that works very well after the simple logistic regression, but after I included the random effects term to the logistic regression I got the following error
"last estimates not found"

[Leonardo Guizzetti]

Nabil,

Clyde has pointed you in the right direction. Stata's built-in facilities for computing discrimination statistics (AUC, ROC tools, lroc and lsens, etc) are all intended to work with logistic regression. The same idea is straightforwardly applied in a multi-level logistic regression framework. However, you will need to do the work to compute the individual predicted outcome probabilities. Together with the actual outcome, you can then use -roctab- to get the AUC values.

Note that you may read the help documentation for -roctab- (and any other topic) by typing -help roctab- (or whatever). Here, the -refvar- is the true reference value which is the same as your dependent variable from the regression model. The -classvar- is the assigned/predicted outcome, which correspond to the predicted probabilities from the model.

As I am not familiar with the MLwIN software, I cannot tell you how the calculate out those probabilities using that tool, so you will need to do some investigation there.

[Nabil Aly]

Dear Leonardo,

Thank you for the detailed explanation. I managed to do it. I appreciate it.

kind regards,
Nabil