Understanding Logit and Probit output

[Kathleen Blackman]

I am currently running Stata14 to analyse a dataset of 228 observations. The dependent variable is binary and so I am trying logit, probit and heckman commands to analyse. When I run the logit command without including the option to 'retain perfect predictor variables, Stata returns the following:

 logit WTPPer Age Income EducationBase Household FishAssocMem CommMem BidAmt

outcome does not vary; remember:
                                  0 = negative outcome,
        all other nonmissing values = positive outcome

Having included 'perfect predictor variables', output I have received so far is baffling because of the amount of missing information. Below are some examples. For the logit and probit, no z, P-values and confidence intervals are given. What does that mean? Listwise deletion has eliminated 10 observations for missing data but I don't know why logit and probit reports 'omitted' variables. The Heckan models' Chi square show a good overall fit but again there are missing z and P-values etc.

.
logit WTPPer Age Income EducationBase Household FishAssocMem CommMem BidAmt, asis

Iteration 0:   log likelihood = -151.10609  
Iteration 1:   log likelihood =          0  
Iteration 2:   log likelihood =          0  

Logistic regression                             Number of obs     =        218
                                                Wald chi2(0)      =          .
Log likelihood =          0                     Prob > chi2       =          .

-------------------------------------------------------------------------------
       WTPPer |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
          Age |          0  (omitted)
       Income |   5.62e-15          .        .       .            .           .
EducationBase |   6.00e-14          .        .       .            .           .
    Household |          0  (omitted)
 FishAssocMem |  -6.00e-14          .        .       .            .           .
      CommMem |          0  (omitted)
       BidAmt |          0  (omitted)
        _cons |      33.75          .        .       .            .           .
-------------------------------------------------------------------------------
Note: 0 failures and 218 successes completely determined.

. probit WTPPer Age Income EducationBase Household FishAssocMem CommMem BidAmt, asis

Iteration 0:   log likelihood = -151.10609  
Iteration 1:   log likelihood =          0  
Iteration 2:   log likelihood =          0  

Probit regression                               Number of obs     =        218
                                                Wald chi2(0)      =          .
Log likelihood =          0                     Prob > chi2       =          .

-------------------------------------------------------------------------------
       WTPPer |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
          Age |  -2.60e-14          .        .       .            .           .
       Income |  -1.73e-14          .        .       .            .           .
EducationBase |  -2.60e-13          .        .       .            .           .
    Household |   8.66e-15          .        .       .            .           .
 FishAssocMem |  -3.46e-14          .        .       .            .           .
      CommMem |          0  (omitted)
       BidAmt |   1.22e-15          .        .       .            .           .
        _cons |   6.109906          .        .       .            .           .
-------------------------------------------------------------------------------
Note: 0 failures and 218 successes completely determined.
heckman WTPPer Age Income EducationBase Household FishAssocMem CommMem BidAmt, twostep select(WTPPer = Age Income EducationBase Household 
> FishAssocMem CommMem BidAmt) rhosigma

Heckman selection model -- two-step estimates   Number of obs     =        218
(regression model with sample selection)        Censored obs      =          0
                                                Uncensored obs    =        218

                                                Wald chi2(7)      =     198.61
                                                Prob > chi2       =     0.0000

-------------------------------------------------------------------------------
              |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
WTPPer        |
          Age |   .0090785   .0203176     0.45   0.655    -.0307434    .0489003
       Income |    .022491   .0152231     1.48   0.140    -.0073458    .0523277
EducationBase |   .0683711   .0396216     1.73   0.084    -.0092857    .1460279
    Household |   .0116895   .0102223     1.14   0.253    -.0083458    .0317248
 FishAssocMem |   .0082721   .0508522     0.16   0.871    -.0913963    .1079405
      CommMem |  -.0178611   .0760647    -0.23   0.814    -.1669452    .1312229
       BidAmt |  -.0245764   .0018022   -13.64   0.000    -.0281087   -.0210441
        _cons |   1.318616   .2651529     4.97   0.000     .7989256    1.838306
          Age |  -2.89e-15          .        .       .            .           .
       Income |   6.80e-16          .        .       .            .           .
EducationBase |  -1.24e-14          .        .       .            .           .
    Household |  -1.94e-16          .        .       .            .           .
 FishAssocMem |  -3.11e-15          .        .       .            .           .
      CommMem |  -1.07e-14          .        .       .            .           .
       BidAmt |   3.47e-18          .        .       .            .           .
        _cons |   6.072463          .        .       .            .           .
--------------+----------------------------------------------------------------
mills         |
       lambda |          0  (omitted)
--------------+----------------------------------------------------------------
          rho |    0.00000
        sigma |  .33375857
-------------------------------------------------------------------------------

. heckman WTPPer Age Income Household  BidAmt, twostep select(WTPPer = Age Income EducationBase Household FishAssocMem CommMem BidAmt) rhosi
> gma

Heckman selection model -- two-step estimates   Number of obs     =        218
(regression model with sample selection)        Censored obs      =          0
                                                Uncensored obs    =        218

                                                Wald chi2(4)      =     192.56
                                                Prob > chi2       =     0.0000

-------------------------------------------------------------------------------
              |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
WTPPer        |
          Age |   .0024724   .0189079     0.13   0.896    -.0345863    .0395312
       Income |   .0232877    .015266     1.53   0.127     -.006633    .0532085
    Household |   .0134325   .0100786     1.33   0.183    -.0063212    .0331863
       BidAmt |  -.0242228   .0018028   -13.44   0.000    -.0277562   -.0206893
        _cons |    1.48447   .1065087    13.94   0.000     1.275717    1.693223
          Age |  -2.89e-15          .        .       .            .           .
       Income |   6.80e-16          .        .       .            .           .
EducationBase |  -1.24e-14          .        .       .            .           .
    Household |  -1.94e-16          .        .       .            .           .
 FishAssocMem |  -3.11e-15          .        .       .            .           .
      CommMem |  -1.07e-14          .        .       .            .           .
       BidAmt |   3.47e-18          .        .       .            .           .
        _cons |   6.072463          .        .       .            .           .
--------------+----------------------------------------------------------------
mills         |
       lambda |          0  (omitted)
--------------+----------------------------------------------------------------
          rho |    0.00000
        sigma |  .33620952
-------------------------------------------------------------------------------

I have unsuccessfully looked for information to help me understand this output so any help received here will be appreciated.

[Carlo Lazzaro]

Kathleen:
-logit- and -probit- notes explain a part of your output: both these regression models work when there's a reasonable mix of 0 and 1 in the regressand; conversely, your regressand does not seem to vary accordingly to this theoretical requrement. The other reason rests on missing values.
It is also expected that -logit- and -probit- outcome cannot differ (set aside their standard errors).
Eventually, I'm not clear why you seemingly considered -heckman- as an alternative to -logit- and -probit-.

[Richard Williams]

DVs in logit/probit need to be coded 0/1. My guess is that yours are coded 1/2 or something like that. Check the coding.

[Eric de Souza]

To follow up on Richard Williams post, type
sum WTPPer
In fact it would useful proiding us with the summary of all the variables you use in the model above, and not only the dependent variable.

[Richard Williams]

To follow up on Eric's suggestion, if the DV is coded 0/1, it might be good to type

logit WTPPer Age Income EducationBase Household FishAssocMem CommMem BidAmt
sum WTPPer Age Income EducationBase Household FishAssocMem CommMem BidAmt if e(sample)

It is possible that missing data could be losing you all the 0s or 1s in the DV.

[Kathleen Blackman]

Thank you Carlo, Eric and Richard for your immensely helpful comments. @Carlo, your comments tied in with Richard's regarding the coding of the DV (I had it coded as 1/2 instead of 1/0) which I have now corrected. However, Richards suggested code did not produce output and so intuitively I believe the summarization as originally suggested by Eric could not be effected. I don't understand what "outcome = BidAmt > 0 predicts data perfectly" really means so I did a cross tabulation of WTPPer and BidAmt and this is what I got.

Thanks again gentlemen.

logit WTPPer Age Income EducationBase Household FishAssocMem CommMem BidAmt

outcome = BidAmt > 0 predicts data perfectly
r(2000);



. sum WTPPer Age Income EducationBase Household FishAssocMem CommMem BidAmt if e(sample)

    Variable |        Obs        Mean    Std. Dev.       Min        Max
-------------+---------------------------------------------------------
      WTPPer |          0
         Age |          0
      Income |          0
EducationB~e |          0
   Household |          0
-------------+---------------------------------------------------------
FishAssocMem |          0
     CommMem |          0
      BidAmt |          0


tab WTPPer BidAmt, all

  Would you |
 be willing |
 to pay for |
     health |
  insurance |
        for |                     Maximum bid of respondent
  yourself? |         0          8         16         24         31         47 |     Total
------------+------------------------------------------------------------------+----------
         no |        69          0          0          0          0          0 |        69 
        yes |         0         56         47         21         21         12 |       157 
------------+------------------------------------------------------------------+----------
      Total |        69         56         47         21         21         12 |       226 

          Pearson chi2(5) = 226.0000   Pr = 0.000
 likelihood-ratio chi2(5) = 278.1139   Pr = 0.000
               Cramér's V =   1.0000
                    gamma =   1.0000  ASE = 0.000
          Kendall's tau-b =   0.7365  ASE = 0.025

[Richard Williams]

You got no cases when you used my suggestion because the logit command didn't run.

Whenever the bid amount is over 0 the DV = 1. So, BidAmt > 0 predicts the outcome perfectly. BidAmt should not be in the model, at least not as is. It looks like the DV is a collapsed version of BidAmt (0 = 0, else = 1)

[Kathleen Blackman]

Carlo, I forgot to respond to your comment about the similarity of logit and probit outcomes and the reason for implementing Heckman. I expected the outcome for the logit andd probit to be similar but when I was getting such poor results from logit, I decided to try probit. Regarding the use of heckman, it has been touted to be an improvement on logit by treating the self-selection process and the amount WTP as two separate but related processes. So I decided to try it out. Thanks again for your comments.

[Jeff Wooldridge]

Kathleen: Is this supposed to be a contingent valuation study, where people are presented with a price and then they answer yes or no at that price? It appears that's not how it was done, as the Bid Amount seems to be chosen by the respondent.

As Richard points out, as things stand, you can't have BidAmt as predictor of WTPPer. You seem to want to use a hurdle model of some sort, but then BiDAmt would be a dependent variable, not a predictor of WTPPer. A hurdle model with a Tobit form doesn't really work because of the discreteness in BidAmt. It seems like an ordered probit or ordered logit is more appropriate if your goal is to explain BidAmt.

[Kathleen Blackman]

You got no cases when you used my suggestion because the logit command didn't run.

Whenever the bid amount is over 0 the DV = 1. So, BidAmt > 0 predicts the outcome perfectly. BidAmt should not be in the model, at least not as is. It looks like the DV is a collapsed version of BidAmt (0 = 0, else = 1)

Thank you for clarifying that for me Richard. I will have to give some thought to what is the way forward with BidAmt.

[Kathleen Blackman]

Kathleen: Is this supposed to be a contingent valuation study, where people are presented with a price and then they answer yes or no at that price? It appears that's not how it was done, as the Bid Amount seems to be chosen by the redyspondent.

As Richard points out, as things stand, you can't have BidAmt as predictor of WTPPer. You seem to want to use a hurdle model of some sort, but then BiDAmt would be a dependent variable, not a predictor of WTPPer. A hurdle model with a Tobit form doesn't really work because of the discreteness in BidAmt. It seems like an ordered probit or ordered logit is more appropriate if your goal is to explain BidAmt.

Thank you Jeff. In fact, this is a CV study but the bid question was not open ended. I used a bidding game where bids were offered iteratively from lowest to highest until a 'no' response was received and then the process stopped. The highest bid for which a 'yes' response was received was taken as the maximum willingness to pay. The goal of the exercise is to determine the factors that significantly influence willingness to pay. I was investigating BidAmt along with other variables to see their impacts on WTPPer.

[Richard Williams]

Like Jeff says, I could see BidAmt as a dependent variable. With over 200 cases, I am curious how you wind up with the distinct values 8, 16, 24, 31, 47. What was the actual question or questions?

[Kathleen Blackman]

Jeff and Richard, after contemplating your suggestion I do see now why BidAmt would be better as a regressand. @Richard - the values are the USD conversion of the local currency. See question below.


[img]
[/img]

[Jeff Wooldridge]

So it seems you know the range of their WTP, correct? You know it's above something and below something else (unless it is is above the largest amount, which is okay). I would suggest an ordered interval regression where you code the variable {0,1,2,3,4,5}. You know the cut points, which makes this interval regression.

[Kathleen Blackman]

So it seems you know the range of their WTP, correct? You know it's above something and below something else (unless it is is above the largest amount, which is okay). I would suggest an ordered interval regression where you code the variable {0,1,2,3,4,5}. You know the cut points, which makes this interval regression.

Yes I do know the ranges. I never considered using interval regression. Thank you for the suggestion.