Firthlogit

[Bill Smith]

Using StataMP 14.1 under Win7E. I'd like to run ROC curves (http://www.stata.com/manuals14/rlroc.pdf) after firthlogit but I get:

. lroc
last estimates not found
r(301);

I'd also like to obtain derivatives using predict, scores (http://www.stata.com/manuals14/rpredict.pdf), but I get:

. predict,scores
newvarlist required
r(100);

Does anyone know any work arounds?

[Rich Goldstein]

first, with your predict statement: I have never used -firthlogit- but in all other cases predict requires a varname; see "h predict"; the predict command may or may not work after -firthlogit-; if not, you can always calculate your own based on the coefficients

second, once you have your predicted values, type "h roc" to see other built-in commands for ROC analysis

[Bill Smith]

My mistake:

. predict sc,scores
scores cannot be produced with method d0
r(301);

But these are not the predicted values. This is the derivative with respect to the linear predictor

score calculates the equation-level score, partial(lnL)=partial(x*betaj). Here lnL refers to the log-likelihood

function.

scores is the ME model equivalent of the score option, resulting in multiple equation-level score

variables. An equation-level score variable is created for each equation in the model; ancillary

parameters—such as ln(sigma) and atanh(rho)—make up separate equations.

[Joseph Coveney]

From the help file for lroc:

lroc requires that the current estimation results be from logistic, logit, probit, or ivprobit.

firthlogit, being a user-written command, isn't among them, but maybe you could

firthlogit . . .
matrix define B = e(b)
logistic . . ., from(B, copy) iterate(0)
lroc

You could try an analogous work-around for the scores, but be aware that the likelihoods' score equations are different, and so the scores wouldn't represent those of the Firth logistic regression model.

[Bill Smith]

Very nice. Thanks for that solution.

[Andrew Lover]

Just to echo Joseph's point, as Firth is a penalized likelihood model, and -lroc- etc expect/assume ML estimates, you could get strange results (?).

Some info here http://prema.mf.uni-lj.si/files/Angelika_654.pdf

[Bill Smith]

So here's a parallel question: what's the best way of judging the fit of Firth regressions?

[Joseph Coveney]

So here's a parallel question: what's the best way of judging the fit of Firth regressions?

Have you considered a calibration plot? An example with firthlogit can be seen here. A post in the latter thread also mentions Tjur's R², if you're looking for a number to provide.