estat classficaition after logit lasso

[Adrien Bouguen]

Hi,

I am interested in classifying households into poverty status (poor non poor) using lasso.

I ran a couple of logit lasso using several selection mechanisms and predicted their respective probability. I would have liked to draw the roc and use the command estat classification to compare the different predictions but the command lasso logit does not seem to allow neither lroc or estat classification. Do you know an alternative?

To circonvent the isssue, I extract from the lasso the selected variables and run a simple logit using these variables but I don''t know if this is entirely equivalent (as I use a prediction is not penalized as it should). A minimal example would be:

xi: lasso probit poor  $var_lasso     if  sample==1 
global selected_pcv `e(othervars_sel)'
logit poor  $selected_cv   if  sample==1 
lroc 
estat classification

Does this make sense ?



Thanks for your help

[Andrew Musau]

Why not? You use lasso to select a model. Once you estimate the selected model, whatever post estimation tools you choose to employ in relation to that model is up to you.

[Adrien Bouguen]

Thanks for your answer. My problem is that I would prefer to use the predicted value that is penalized by the lasso. Basically I would like to do

 
  xi: lasso probit poor  $var_lasso     if  sample==1 predict pr_lasso  lroc  estat classification

and use the pr_lasso prediction in my roc and estat classficiation. In the code in my previous post, I am using the non penalized prediction.

[Rich Goldstein]

once you have the predicted values you can use other tools including the -roc- series to get ROC curves and the area under the curve; see

h roc

you can duplicate what is from -estat classification- by using the predicted probabilities to classify people and then using -tabulate-

this sounds straightforward to me so I'm afraid maybe I'm missing something; if so, please clarify

[Adrien Bouguen]

Ok thanks for the info. Yes it is straightforward to calculate the stats from estat but maybe less to graph a roc. I believe this fixed my problem for instance to calculate the roc

xi: lasso logit q0_cse_tp   $all      if Year=="BSL" & q0_cse_ptp!=. & sample==1 & eligible==1,  cluster(vill_unique) rseed(29051737)
lassocoef, dis(coef, penalized)
mat a=r(coef)
mat a= a'
lroc q0_cse_tp, beta(a)