F statistic from logit model with survey data

[Ivan Oreskovic]

Hi,

when performing logistic regression with survey data, is the reported F statistic actually result from the adjusted Wald test? I run

 test coeff, accumulate

command after

svy: logit y x

for every coefficient and obtained the exact value as reported F statistic. I am using Stata 15.

[Clyde Schechter]

The results of -test- after -svy: logit- are Wald tests. I do not know what you mean by "adjusted" in this context.

[Ivan Oreskovic]

Clyde Schechter

Thank you for time. I was referring to https://www.stata.com/support/faqs/s...od-ratio-test/ and to https://www.stata.com/manuals13/rtest.pdf (page 3, second paragraph) where it states that "test" command in svy context computes adjusted Wald test. What does it really mean, I did not figure it out yet.

Additionally, what it is calculated/tested by running only -test command without any coeflist and options after -svy: logit?

[Clyde Schechter]

As for as I know, nothing is calculated, and you would just get an error message. If that's not true, please post an example.

[Ivan Oreskovic]

use http://www.stata-press.com/data/r13/nhanes2d
svy: logistic highbp height weight age female
(running logistic on estimation sample)

Survey: Logistic regression

Number of strata   =        31                Number of obs     =       10,351
Number of PSUs     =        62                Population size   =  117,157,513
                                              Design df         =           31
                                              F(   4,     28)   =       368.33
                                              Prob > F          =       0.0000

------------------------------------------------------------------------------
             |             Linearized
      highbp | Odds Ratio   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      height |   .9657022   .0051511    -6.54   0.000     .9552534    .9762654
      weight |   1.053023   .0026902    20.22   0.000     1.047551    1.058524
         age |   1.050059   .0019761    25.96   0.000     1.046037    1.054097
      female |   .6272129   .0368195    -7.95   0.000     .5564402     .706987
       _cons |    .716868   .6106878    -0.39   0.699     .1261491    4.073749
------------------------------------------------------------------------------
Note: _cons estimates baseline odds.

test
( 1)  [highbp]height = 0
 ( 2)  [highbp]weight = 0
 ( 3)  [highbp]age = 0
 ( 4)  [highbp]female = 0

       F(  4,    31) =  407.79
            Prob > F =    0.0000

test height weight age female

Adjusted Wald test

 ( 1)  [highbp]height = 0
 ( 2)  [highbp]weight = 0
 ( 3)  [highbp]age = 0
 ( 4)  [highbp]female = 0

       F(  4,    28) =  368.33
            Prob > F =    0.0000

As it can been seen, running -test command with or without coefflist, computes different values for F statistic and degrees of freedom.

[Clyde Schechter]

This is quite odd, and I don't know what to make of it. I can't figure out what -test- is actually doing here. The denominator degrees of freedom 31, seems to me to be impossible because you only have 31 df in the model to start with, and some of them should be taken up by the variables (apparently) being tested. And it also gives the wrong result for an omnibus test of the variables. It disagrees with the omnibus F-test reported with the -svy: logit- output (and also with the results from -test- with a varlist.)

I strongly suspect this is a bug, but perhaps there is something I'm overlooking.

[Richard Williams]

The first test command is not doing an svy adjustment. It is equivalent to

. testparm height weight age female, nosvy

Unadjusted Wald test

 ( 1)  [highbp]height = 0
 ( 2)  [highbp]weight = 0
 ( 3)  [highbp]age = 0
 ( 4)  [highbp]female = 0

       F(  4,    31) =  407.79
            Prob > F =    0.0000

That explains what is being done, but not why. I am puzzled too.

[Richard Williams]

The help says "test supports svy estimators (see [SVY] svy estimation), carrying out an adjusted Wald test by default in such cases. " So why doesn't do that when test appears without variables? It sounds like a bug to me or an error in the documentation.

[Richard Williams]

I ran the test commands with set trace on. If you just specify -test- the code shown is

- if "`e(cmd)'" == "anova" | "`e(cmd)'" == "manova" {
= if "logistic" == "anova" | "logistic" == "manova" {
  AnovaTestChooser `0'
  exit
  }
- else if "`e(cmd)'" == "mixed" {
= else if "logistic" == "mixed" {
  MixedTestChooser `0'
  exit
  }
- else if _caller() < 8 | replay() {
- _test `0'
= _test 

 ( 1)  [highbp]height = 0
 ( 2)  [highbp]weight = 0
 ( 3)  [highbp]age = 0
 ( 4)  [highbp]female = 0

       F(  4,    31) =  407.79
            Prob > F =    0.0000

- exit

But, if you specify test varnames, the returned code is much longer, and includes a call to is_svy.

So, I think plain old -test- is failing to check if svy is being used. This seems like a bug to me. If the behavior is documented I am not seeing it. If somebody from Stata Corp doesn't comment on this thread I would report the behavior to tech support.

[Ivan Oreskovic]

I came with one question and it turned out that I inadvertently discovered a bug.

I will wait for a while if someone from Stata Corp will comment on this thread. If not, I will report the bug to tech support.

Anyway, thank you both for your effort, you provided answers to my questions.