Problems with simple adjusted logistic regression on small sample-size (n=13)

[Rasmus Hansen]

Hi
I am trying to make a simple logistic regression adjusted for one potential confounder ("Antibtiotics") on a small sample size (n=13).
All variables are binary. I need to adjust for one potential confounder; whether or not the person took ("Antibiotics").

I am "investigating" whether a bloo-value above/below a givet cut-off affects outcome, when adjusting for Antibiotics (yes/no).


My code:
logit Outcome i.Bloodvalue i.Antibiotics, or

The output:
Logistic regression Number of obs = 13
LR chi2(1) = 2.22
Prob > chi2 = 0.1361
Log likelihood = -6.9135667 Pseudo R2 = 0.1384

----------------------------------------------------------------------------------------------------------------------------------------------------
Outcome | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval]
---------------------------+-----------------------------------------------------------------------------------------------------------------------
1.Bloodvalue | 1 (empty)
1.Antibiotics | .125 .1822172 -1.43 0.154 .0071793 2.176401
_cons | 2 2.44949 0.57 0.571 .1813532 22.0564




I am aware that an adjusted regression of such a small sample size is problematic, but my question is:
- Why can't I get Stata to compute the Odss Ratio, Standard Error, P-value and CI for the bloodvalue
- All people with a value above the blood-value (n=3) cut-off experienced "Outcome", is that the problem? It worked fine with Fishers Exact Test

Stata IC 15.1

Kind Regards
Rasmus

[Rasmus Hansen]

Sorry, the output looks absolutely terribly. But key point is that it wont calculate on the depentent variable (Bloodvalue)

[Carlo Lazzaro]

Rasmus:
if the predictor actually predicts the outcome perfectly, as I read it from the description of the problem you experienced, the coefficient is left uncalculated.
However, Stata should have warned you about that issue.
Since you have such a small dataset, you may want to share it with the list via -dataex-, so that interested listers can investigate where the problem lurks.

[Richard Williams]

You may want to try exlogistic.

Also, see pt 12 of the FAQ, especially the section on asking Qs effectively. Code tags would make the output much more readable. And, like Carlo says, using dataex could really help.

[Eric de Souza]

With so few observation, you should forget about using logit models and the like. A simple linear probability model is the furthest you can go; And even then.

[Rasmus Hansen]

Thank you for the replies.
The dataset looks as follows - remember, all variables are binary

* Example generated by -dataex-. To install: ssc install dataex
clear
input byte Outcome double Bloodvalue byte Antibiotics
0 0 1
0 0 1
1 0 1
0 0 0
1 0 1
0 0 1
1 0 1
0 0 1
1 1 1
0 0 1
1 0 1
0 0 1
1 1 1
0 0 1
0 0 1
end

The input:

logit Outcome i.Bloodvalue i.Antibiotics, or

I too think that the problem is that the predictor predicts the outcome perfectly.

[Richard Williams]

This is what I get, which does not match the output you posted earlier. If this is indeed the exact same data as you used in your first post, it makes me wonder if your version of Stata is up to date. Use the -update all- command to make sure you have the latest version.

. logit Outcome i.Bloodvalue i.Antibiotics, or

note: 0.Bloodvalue != 1 predicts success perfectly
      0.Bloodvalue dropped and 2 obs not used

note: 0.Antibiotics != 0 predicts failure perfectly
      0.Antibiotics dropped and 1 obs not used

note: 1.Bloodvalue omitted because of collinearity
note: 1.Antibiotics omitted because of collinearity
Iteration 0:   log likelihood =   -7.63817  
Iteration 1:   log likelihood =   -7.63817  

Logistic regression                             Number of obs     =         12
                                                LR chi2(0)        =       0.00
                                                Prob > chi2       =          .
Log likelihood =   -7.63817                     Pseudo R2         =     0.0000

------------------------------------------------------------------------------
     Outcome | Odds Ratio   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.Bloodvalue |          1  (empty)
             |
 Antibiotics |
          0  |          1  (empty)
          1  |          1  (omitted)
             |
       _cons |         .5   .3061862    -1.13   0.258     .1505628    1.660437
------------------------------------------------------------------------------
Note: _cons estimates baseline odds.

[Carlo Lazzaro]

Rasmus:
got Richard's very same output with Stata 15.1:

. input byte Outcome double Bloodvalue byte Antibiotics

      Outcome  Bloodvalue  Antibi~s
  1.
. 0 0 1
  2.
. 0 0 1
  3.
. 1 0 1
  4.
. 0 0 0
  5.
. 1 0 1
  6.
. 0 0 1
  7.
. 1 0 1
  8.
. 0 0 1
  9.
. 1 1 1
 10.
. 0 0 1
 11.
. 1 0 1
 12.
. 0 0 1
 13.
. 1 1 1
 14.
. 0 0 1
 15.
. 0 0 1
 16.
. end

. logit Outcome i.Bloodvalue i.Antibiotics, or

note: 0.Bloodvalue != 1 predicts success perfectly
      0.Bloodvalue dropped and 2 obs not used

note: 0.Antibiotics != 0 predicts failure perfectly
      0.Antibiotics dropped and 1 obs not used

note: 1.Bloodvalue omitted because of collinearity
note: 1.Antibiotics omitted because of collinearity
Iteration 0:   log likelihood =   -7.63817 
Iteration 1:   log likelihood =   -7.63817 

Logistic regression                             Number of obs     =         12
                                                LR chi2(0)        =       0.00
                                                Prob > chi2       =          .
Log likelihood =   -7.63817                     Pseudo R2         =     0.0000

------------------------------------------------------------------------------
     Outcome | Odds Ratio   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.Bloodvalue |          1  (empty)
             |
 Antibiotics |
          0  |          1  (empty)
          1  |          1  (omitted)
             |
       _cons |         .5   .3061862    -1.13   0.258     .1505628    1.660437
------------------------------------------------------------------------------
Note: _cons estimates baseline odds.

.

[Richard Williams]

Also, in the data posted, N = 15 with 3 cases getting dropped. That differs from the 13 cases claimed in the original post. This makes me wonder if the data posted is the same as the data originally used. dataex is great, but make sure the extract it creates actually reproduces the problem you are writing about.

[Richard Williams]

Incidentally here are the results if you use exlogistic and the user-written firthlogit (available from SSC). I am not sure how much I trust anything done with such a small N, but maybe this is ok or at least as good as it gets. I would read up on exlogistic before using it. Given that the CIs run to positive infinity I suspect exlogistic doesn't help you much!

. exlogistic Outcome Bloodvalue Antibiotics

Enumerating sample-space combinations:
observation 1:   enumerations =          2
observation 2:   enumerations =          3
observation 3:   enumerations =          4
observation 4:   enumerations =          8
observation 5:   enumerations =         10
observation 6:   enumerations =         12
observation 7:   enumerations =         13
observation 8:   enumerations =         13
observation 9:   enumerations =         24
observation 10:  enumerations =         23
observation 11:  enumerations =         20
observation 12:  enumerations =         16
observation 13:  enumerations =         18
observation 14:  enumerations =         12
observation 15:  enumerations =          6
note: CMLE estimate for Bloodvalue is +inf; computing MUE
note: CMLE estimate for Antibiotics is +inf; computing MUE

Exact logistic regression                        Number of obs =         15
                                                 Model score   =    3.62963
                                                 Pr >= score   =     0.1429
---------------------------------------------------------------------------
     Outcome | Odds Ratio       Suff.  2*Pr(Suff.)     [95% Conf. Interval]
-------------+-------------------------------------------------------------
  Bloodvalue |   3.703965*          2      0.3297      .2636157       +Inf
 Antibiotics |   .4444444*          6      1.0000       .011396       +Inf
---------------------------------------------------------------------------
(*) median unbiased estimates (MUE)

. firthlogit Outcome Bloodvalue Antibiotics

initial:       penalized log likelihood = -10.646823
rescale:       penalized log likelihood = -10.646823
Iteration 0:   penalized log likelihood = -10.646823  
Iteration 1:   penalized log likelihood = -9.2735357  
Iteration 2:   penalized log likelihood =  -9.272758  
Iteration 3:   penalized log likelihood =  -9.272758  

                                                Number of obs     =         15
                                                Wald chi2(2)      =       2.03
Penalized log likelihood =  -9.272758           Prob > chi2       =     0.3632

------------------------------------------------------------------------------
     Outcome |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
  Bloodvalue |   2.245427   1.655255     1.36   0.175    -.9988136    5.489667
 Antibiotics |   .4626236   1.733936     0.27   0.790    -2.935829    3.861076
       _cons |  -1.098612   1.632993    -0.67   0.501    -4.299219    2.101995
------------------------------------------------------------------------------

.