Wald tests

[paulvonhippel]

A question about Wald tests. Suppose I fit a model like
xtmixed y ibn.x, noconstant
where x is a factor variable with P levels. If I run the postestimation command
test i.x, equal
I get a Wald chi-square test of the null hypothesis that the different levels of x don't differ in their effect on y. The Wald statistic has P-1 degrees of freedom.

Initially I thought that this Wald test was obtained by a matrix formula that compared b, the estimated Px1 coefficient vector of x, to V, the estimated PxP variance-covariance matrix of b. However, looking at the output, I'm starting to think that the the Wald test is actually obtained by a series of scalar calculations. Each scalar calculation computes the chi-square associated with the hypothesis that two elements of b are equal, which is obtained by dividing the squared difference between those elements of b by the squared standard error of the difference. Then all those chi-square are added up.

The scalar approach is a little different than the matrix approach because it ignores the off-diagonal elements. Am I right? Is the scalar approach used by the test statement, and what would I do to use the matrix approach instead?

Best,
Paul

[Jeff Pitblado (StataCorp)]

If you take a look at the e(V) matrix produced by your call to xtmixed you will notice that
it is diagonal. Testing the equality of coefficients typically involves a matrix calculation, but this case
has zeroes in the off-diagonal elements.

[paulvonhippel]

Good catch. The simplified model I presented --
xtmixed y ibn.x, noconstant
-- does have a diagonal e(V) because x^t x is diagonal. However, as soon as I add a covariate, e.g.,
xtmixed y ibn.x i.race, noconstant
-- e(V) isn't diagonal any more. That's the situation I meant to ask about. With a nondiagonal e(V), does "test i.x, equal" invoke the matrix formula for the Wald test? Or does it add up scalar Wald tests, as described earlier. From the output below, it looks as though it's just walking through a series of pairwise contrasts which wouldn't use a matrix formula. Thanks for any clarification.

Sample output of "test i.x, equal":
(1) - [y]x1 + [y]x2 = 0
(2) - [y]x1 + [y]x3 = 0
<snip>
(80) - [y]x1 + [y]x81 = 0

chi2( 80) = 82.61
Prob > chi2 = 0.3986

[paulvonhippel]

What got me wondering about this is that we get the same chi-square from the testparm command as we do from the equivalent contrast command. But I might understand why. I'm guessing that both testparm and contrast do the big matrix calculation by doing calculations on all the 2x2 submatrices and adding up the results.

Is that about right?

Best,
Paul

[Jeff Pitblado (StataCorp)]

Sometimes it helps to bring in an example we can all play with.

In Stata 13, xtmixed has been official renamed to mixed
and otherwise retains the same functionality.

I feel I should also point out that Paul is specifying a model without
any random effects, but that doesn't affect the following discussion.

Here we use mixed to fit a fixed-effects only model using the
auto data:

. sysuse auto
(1978 Automobile Data)

. xtmixed mpg bn.rep i.for, noconstant

Mixed-effects ML regression                     Number of obs      =        69

                                                Wald chi2(6)       =   1312.43
Log likelihood = -208.02576                     Prob > chi2        =    0.0000

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       rep78 |
          1  |         21   3.488089     6.02   0.000     14.16347    27.83653
          2  |     19.125   1.744044    10.97   0.000     15.70674    22.54326
          3  |   19.07767   .9157814    20.83   0.000     17.28278    20.87257
          4  |   19.88837   1.428395    13.92   0.000     17.08877    22.68798
          5  |    24.4537   2.013849    12.14   0.000     20.50663    28.40078
             |
     foreign |
    Foreign  |   3.556584   1.659456     2.14   0.032     .3041104    6.809058
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
                sd(Residual) |   4.932903   .4199165      4.174875    5.828565
------------------------------------------------------------------------------

We can see that e(V) is not diagonal:

. matrix list e(V)

symmetric e(V)[8,8]
                       mpg:        mpg:        mpg:        mpg:        mpg:
                         1.          2.          3.          4.          5.
                     rep78       rep78       rep78       rep78       rep78
   mpg:1.rep78   12.166764
   mpg:2.rep78           0   3.0416909
   mpg:3.rep78           0           0   .83865551
   mpg:4.rep78           0           0   .13768971   2.0403112
   mpg:5.rep78           0           0   .22531044   1.1265522   4.0555879
mpg:0b.foreign           0           0           0           0           0
 mpg:1.foreign           0           0  -.27537942  -1.3768971  -2.2531044
 lnsig_e:_cons           0           0           0           0           0

                       mpg:        mpg:    lnsig_e:
                        0b.          1.            
                   foreign     foreign       _cons
mpg:0b.foreign           0
 mpg:1.foreign           0   2.7537942
 lnsig_e:_cons           0           0   .00724638

Now let's test the hypothesis that all the coefficients on the level variables
corresponding the factor variable rep78 are the same:

. testparm i.rep, equal

 ( 1)  - [mpg]1bn.rep78 + [mpg]2.rep78 = 0
 ( 2)  - [mpg]1bn.rep78 + [mpg]3.rep78 = 0
 ( 3)  - [mpg]1bn.rep78 + [mpg]4.rep78 = 0
 ( 4)  - [mpg]1bn.rep78 + [mpg]5.rep78 = 0

           chi2(  4) =    7.36
         Prob > chi2 =    0.1179

Here testparm is reporting the null hypothesis being tested, in the form
of individual contrasts, followed by the test statistic and p-value. We can reproduce
this directly with a call to test:

. test 1.rep78 = 2.rep78 = 3.rep78 = 4.rep78 = 5.rep78

 ( 1)  [mpg]1bn.rep78 - [mpg]2.rep78 = 0
 ( 2)  [mpg]1bn.rep78 - [mpg]3.rep78 = 0
 ( 3)  [mpg]1bn.rep78 - [mpg]4.rep78 = 0
 ( 4)  [mpg]1bn.rep78 - [mpg]5.rep78 = 0

           chi2(  4) =    7.36
         Prob > chi2 =    0.1179

We can test each contrast individually:

. test 1.rep78 = 2.rep78
4
 ( 1)  [mpg]1bn.rep78 - [mpg]2.rep78 = 0

           chi2(  1) =    0.23
         Prob > chi2 =    0.6307

. test 1.rep78 = 3.rep78

 ( 1)  [mpg]1bn.rep78 - [mpg]3.rep78 = 0

           chi2(  1) =    0.28
         Prob > chi2 =    0.5940

. test 1.rep78 = 4.rep78

 ( 1)  [mpg]1bn.rep78 - [mpg]4.rep78 = 0

           chi2(  1) =    0.09
         Prob > chi2 =    0.7681

. test 1.rep78 = 5.rep78

 ( 1)  [mpg]1bn.rep78 - [mpg]5.rep78 = 0

           chi2(  1) =    0.74
         Prob > chi2 =    0.3912

The sum of those chi-square statistics is about 1.34, which does not equal
7.36 the chi-square reported by testparm.

The actual formulas used by testparm and test are documented in
the manual entry for test. From within Stata type

help testparm

Then click on the link [R] test within the Title section in the Viewer.

[paulvonhippel]

Thank you -- this is *very* helpful. Continuing your example, I note that I can get the same joint chi-square statistic from the contrast statement -- i.e.,
. contrast gw.rep, post

Contrasts of marginal linear predictions

Margins : asbalanced

------------------------------------------------
| df chi2 P>chi2
-----------------+----------------------------------
mpg |
rep78 |
(1 vs mean) | 1 0.05 0.8195
(2 vs mean) | 1 0.40 0.5281
(3 vs mean) | 1 2.23 0.1357
(4 vs mean) | 1 0.09 0.7615
(5 vs mean) | 1 6.97 0.0083
Joint | 4 7.36 0.1179
------------------------------------------------

Until I saw this example, I thought that the joint chi2(4) was obtained by adding the five chi2(1) values above it. But in this example I see that's not the case. Is the contrast joint chi-square obtained using an equivalent formula to the Wald joint chi-square?

[Jeff Pitblado (StataCorp)]

From the methods and formulas for test, the Wald statistic is computed as

\[
W = (Rb - r)'(RVR')^{-1}(Rb - r)
\]

where matrix R and vector r represent the constraints implied by the null
Hypothesis being tested. In our case, R is a matrix of contrasts and r is
a vector of zeroes.

The methods and formulas for contrast provide examples of the contrasts
implied by the factor variable (FV) operators unique to the contrast (and margins)
command. The 'Joint' test reported by contrast is using the same formula above
and R is determined by the chosen FV operator. The contrast matrix for the g.
operator specifically for 5 levels is shown in this section. The weighted version gw.
is also described.

[G Bhadra]

I just want to confirm that the test command can be used to compare the coefficients of the random effects as well. Or is there a separate check/command for that?

Best,
Bhadra