Unexpected result from test command used with xtgee

[Simon Cousens]

I am using xtgee to estimate risk ratios for a data set with repeated measures. I want to test an interaction between two factors, 1 with 2 levels (arm1) and the other with 3 levels (indication): i.e. 2 interaction parameters. I had expected that using the test command would yield a chi-squared statistic (2df) equal to the difference between the chi-squared statistics for models with and without the interaction. That isn't what I get. The Wald chi(2) (5 df) for the model with the interaction = 10.21. The Wald chi(2) (3df) for the model without the interaction is 8.44 a difference of 1.77. The chi squared value produced by the the test command is 1.60. Can anyone explain the difference? Output below.

xtgee prim_outcome i.arm1##i.indication if arm1!=3, family(binomial) link(log) corr(exch) vce(robust) eform

GEE population-averaged model Number of obs = 6,749
Group variable: participan~m Number of groups = 6,287
Family: Binomial Obs per group:
Link: Log min = 1
Correlation: exchangeable avg = 1.1
max = 4
Wald chi2(5) = 10.21
Scale parameter = 1 Prob > chi2 = 0.0696

(Std. err. adjusted for clustering on participantnum)
-----------------------------------------------------------------------------------------------------------------
| Semirobust
prim_outcome | exp(b) std. err. z P>|z| [95% conf. interval]
------------------------------------------------+----------------------------------------------------------------
arm1 |
Dexa 4x6mg | .8965813 .1559133 -0.63 0.530 .6376286 1.260699
|
indication |
Preterm labor with intact membranes | .6763322 .1428367 -1.85 0.064 .4470871 1.023123
Planned delivery | .9138782 .1248478 -0.66 0.510 .6992025 1.194466
|
arm1#indication |
Dexa 4x6mg#Preterm labor with intact membranes | 1.168356 .3464885 0.52 0.600 .6533442 2.089335
Dexa 4x6mg#Planned delivery | 1.278983 .2505847 1.26 0.209 .8711508 1.877745
|
_cons | .1040274 .0124361 -18.93 0.000 .0822981 .131494
-----------------------------------------------------------------------------------------------------------------

. test 2.arm1#2.indication 2.arm1#3.indication

( 1) 2.arm1#2.indication = 0
( 2) 2.arm1#3.indication = 0

chi2( 2) = 1.60
Prob > chi2 = 0.4493

. xtgee prim_outcome i.arm1 i.indication if arm1!=3, family(binomial) link(log) corr(exch) vce(robust) eform

GEE population-averaged model Number of obs = 6,749
Group variable: participan~m Number of groups = 6,287
Family: Binomial Obs per group:
Link: Log min = 1
Correlation: exchangeable avg = 1.1
max = 4
Wald chi2(3) = 8.44
Scale parameter = 1 Prob > chi2 = 0.0378

(Std. err. adjusted for clustering on participantnum)
------------------------------------------------------------------------------------------------------
| Semirobust
prim_outcome | exp(b) std. err. z P>|z| [95% conf. interval]
-------------------------------------+----------------------------------------------------------------
arm1 |
Dexa 4x6mg | 1.083794 .0823057 1.06 0.289 .9339095 1.257734
|
indication |
Preterm labor with intact membranes | .7305089 .1081895 -2.12 0.034 .5464637 .9765394
Planned delivery | 1.035454 .1013354 0.36 0.722 .8547268 1.254394
|
_cons | .0945754 .009167 -24.33 0.000 .078212 .1143624
------------------------------------------------------------------------------------------------------

[Andrew Musau]

I think you are implicitly thinking in terms of a likelihood-ratio (LR) test, where

\[
LR = 2\left(\ell_{\text{full}}-\ell_{\text{reduced}}\right),
\]

and the difference in log-likelihoods provides a valid test of the additional parameters. In that setting, it is natural to think that

\[
\chi^2_{\text{full}}-\chi^2_{\text{reduced}}
\]

should equal the test statistic for the added terms.

However, that logic applies to LR statistics, not to Wald statistics. There is generally no reason why the Wald statistic for a subset of coefficients should equal the difference between two model-level Wald statistics. In general,

\[
W(\beta_1,\beta_2)\neq W(\beta_1)+W(\beta_2).
\]


While LR, Wald, and score tests are asymptotically equivalent under standard regularity conditions, they are not identical in finite samples. Thus, even in ordinary likelihood-based models, the Wald test for a subset of coefficients need not equal the difference between the model \(\chi^2\) statistics from nested models.

For example:

webuse union, clear
logit union age not_smsa
local chi2= e(chi2)
est sto m1
logit union age grade not_smsa south
lrtest . m1
display "The difference in LR chi2 is `:di %4.2f `e(chi2)'- `chi2''"
test grade south

Res.:

. logit union age not_smsa

Iteration 0:  Log likelihood =  -13864.23  
Iteration 1:  Log likelihood = -13795.841  
Iteration 2:  Log likelihood = -13795.571  
Iteration 3:  Log likelihood = -13795.571  

Logistic regression                                     Number of obs = 26,200
                                                        LR chi2(2)    = 137.32
                                                        Prob > chi2   = 0.0000
Log likelihood = -13795.571                             Pseudo R2     = 0.0050

------------------------------------------------------------------------------
       union | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
         age |   .0125366   .0022983     5.45   0.000      .008032    .0170412
    not_smsa |  -.3597254   .0347456   -10.35   0.000    -.4278256   -.2916253
       _cons |  -1.543606    .072325   -21.34   0.000     -1.68536   -1.401851
------------------------------------------------------------------------------

.
. local chi2= e(chi2)

.
. est sto m1

.
. logit union age grade not_smsa south

Iteration 0:  Log likelihood =  -13864.23  
Iteration 1:  Log likelihood = -13550.791  
Iteration 2:  Log likelihood = -13546.595  
Iteration 3:  Log likelihood = -13546.593  
Iteration 4:  Log likelihood = -13546.593  

Logistic regression                                     Number of obs = 26,200
                                                        LR chi2(4)    = 635.27
                                                        Prob > chi2   = 0.0000
Log likelihood = -13546.593                             Pseudo R2     = 0.0229

------------------------------------------------------------------------------
       union | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
         age |    .011683   .0023419     4.99   0.000     .0070929     .016273
       grade |    .048511   .0064256     7.55   0.000      .035917    .0611049
    not_smsa |  -.2214007   .0355809    -6.22   0.000     -.291138   -.1516634
       south |  -.6470985   .0327323   -19.77   0.000    -.7112527   -.5829443
       _cons |  -1.941974   .1055379   -18.40   0.000    -2.148824   -1.735124
------------------------------------------------------------------------------

.
. lrtest . m1

Likelihood-ratio test
Assumption: m1 nested within .

LR chi2(2) = 497.96
Prob > chi2 = 0.0000

.
. display "The difference in LR chi2 is `:di %4.2f `e(chi2)'- `chi2''"
The difference in LR chi2 is 497.96

.
. test grade south

 ( 1)  [union]grade = 0
 ( 2)  [union]south = 0

           chi2(  2) =  473.33
         Prob > chi2 =    0.0000

.

[Simon Cousens]

Andrew, thanks for the clarification.