Optimal cutoff lroc curve after probit

[Arbnor Gashi]

Does anyone know how to get the optimal cutoff point of lroc curve after a probit estimation?

[Arbnor Gashi]

I think my question was wrong. I want to find the threshold cutoff which gives the lowest missclasfied error (missed events+false alarms). Maybe this is given by lsens command. However my question is how to have a table of these results where I can see the specificity and sensitivity that corresponds to each cutoff.

[Joseph Coveney]

my question is how to have a table of these results where I can see the specificity and sensitivity that corresponds to each cutoff.

Doesn't -lsens- have options to generate new variables with probability cutoffs and the corresponding sensitivity and specificity values?

[Leonardo Guizzetti]

However my question is how to have a table of these results where I can see the specificity and sensitivity that corresponds to each cutoff.

This can be done by obtaining the predicted probability from the model, then using -roctab, detail-.

* get predicted probabilities
predict prhat, pr
* compute the diagnostic statistics table for all thresholds
* Note: change true_outcome with your actual outcome variable.
roctab true_outcome prhat, detail

[Leonardo Guizzetti]

Doesn't -lsens- have options to generate new variables with probability cutoffs and the corresponding sensitivity and specificity values?

You are correct. Arbnor Gashi can read the output of -help lsens- to see that there are options to generated variables for probability cutoffs, sensitivity and specificity.

[Arbnor Gashi]

I actually want to calculate the model with lowest mistakes (False + rate for true ~D Pr( +|~D) + False - rate for true D Pr( -| D) ). And the command you mentioned doesn't give these two ratio.

[Joseph Coveney]

You can use a combination of the -probit- postestimation command -estat classification- and a function-minimizing command like -minbound- to search for the so-called optimum cutoff. You'd wrap the postestimation command and call the wrapped command with the function minimizer in a manner like that illustrated below. (Start at the "Begin here" comment; the beginning just sets up the illusration.)

version 16.0

clear *

set seed `=strreverse("1531837")'
quietly set obs 10000

generate double sco = runiform()
generate double xb = invnormal(sco) + rnormal()
generate byte dis = rbinomial(1, normal(xb))

probit dis c.sco, nolog

*
* Begin here
*
program define maxim, rclass
    version 16.0

    estat classification, cutoff(`=`1'')
    return scalar fx = 100 - r(P_corr) // or (100 - r(P_corr))^2
end

minbound maxim, range(0.01 0.99) from(0.5)
display in smcl as text char(34) + "optimum" + char(34)+ " cutoff: " %05.3f r(x)

exit

There might even be a user-written command that does this for you. You might want to search SSC using relevant keywords.

You can google for pitfalls of this kind of activity and decide for yourself whether it's worthwhile.