xtlogit wald statistic

[Drishta Tengur]

I am using a panel dataset of 26 countries for the period 1995-2015. I have a binary dependent variable and I am using a logit estimator. However, upon running the regression, the Wald statistic and the Prob>chi2 are missing. Is this a concern?

 xtlogit reelection patr1 right left govtexp inflation gdp popbelow14 pop65anda
> bove illiteracy i.year, pa
note: 2015.year != 0 predicts success perfectly
      2015.year dropped and 1 obs not used


Iteration 1: tolerance = .2706007
Iteration 2: tolerance = .00823476
Iteration 3: tolerance = .00050117
Iteration 4: tolerance = .00001396
Iteration 5: tolerance = 7.796e-07

GEE population-averaged model                   Number of obs     =        494
Group variable:                    country      Number of groups  =         26
Link:                                logit      Obs per group:
Family:                           binomial                    min =         19
Correlation:                  exchangeable                    avg =       19.0
                                                              max =         19
                                                Wald chi2(26)     =          .
Scale parameter:                         1      Prob > chi2       =          .

-------------------------------------------------------------------------------
   reelection |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
        patr1 |   2.344548   2.219738     1.06   0.291    -2.006058    6.695154
        right |   .0349211   .4202385     0.08   0.934    -.7887312    .8585733
         left |   .5842132   .4160247     1.40   0.160    -.2311801    1.399607
      govtexp |   1.575913   2.965354     0.53   0.595    -4.236074      7.3879
    inflation |  -9.848577   6.235816    -1.58   0.114    -22.07055    2.373398
          gdp |  -9.66e-14   1.67e-13    -0.58   0.563    -4.24e-13    2.31e-13
   popbelow14 |   3.382788   9.922067     0.34   0.733    -16.06411    22.82968
pop65andabove |  -10.20768   9.991401    -1.02   0.307    -29.79046    9.375109
   illiteracy |   3.183667   4.618847     0.69   0.491    -5.869108    12.23644
              |
         year |
        1997  |  -.8072545   1.273658    -0.63   0.526    -3.303578    1.689069
        1998  |   1.224637    .900239     1.36   0.174    -.5397989    2.989073
        1999  |    .612746   .9474458     0.65   0.518    -1.244214    2.469706
        2000  |   .4551383   .9935316     0.46   0.647    -1.492148    2.402424
        2001  |  -.7200783   1.285661    -0.56   0.575    -3.239927    1.799771
        2002  |   1.277485   .9126422     1.40   0.162    -.5112614     3.06623
        2003  |   .8111347   .9625633     0.84   0.399    -1.075455    2.697724
        2004  |  -.6165052   1.291895    -0.48   0.633    -3.148572    1.915562
        2005  |   .6447918   1.019536     0.63   0.527    -1.353461    2.643045
        2006  |   .7293904   1.023305     0.71   0.476    -1.276251    2.735031
        2007  |   1.392523   .9603391     1.45   0.147     -.489707    3.274753
        2008  |    .932563    1.01778     0.92   0.360     -1.06225    2.927376
        2009  |   .0755349   1.120753     0.07   0.946      -2.1211     2.27217
        2010  |  -.5570656   1.326388    -0.42   0.674    -3.156738    2.042607
        2011  |   .8688529   1.038749     0.84   0.403    -1.167057    2.904763
        2012  |   .3958903   1.118712     0.35   0.723    -1.796745    2.588525
        2013  |    1.33006   1.000007     1.33   0.184    -.6299166    3.290037
        2014  |   .9889316   1.044374     0.95   0.344    -1.058005    3.035868
        2015  |          0  (empty)
              |
        _cons |  -2.763799   2.875825    -0.96   0.337    -8.400313    2.872715
-------------------------------------------------------------------------------

[Clyde Schechter]

You have only 26 groups, but 27 predictors. So there are not enough degrees of freedom to do a Wald test on all of these predictors simultaneously. You can do subsets of them, but not the whole bunch because there are too many.

[Drishta Tengur]

But are the results still valid even if the Wald test does not show?

[Clyde Schechter]

Yes, they are perfectly valid. And, if you think about it, the overall model Wald test is usually of no real interest or importance anyway. Many of the variables included are typically nuisance variables incorporated to adjust for their effects on the outcome that might obscure the effects of the variables of interest. Usually one is interested in tests of a small number of key variables that are the focus of research. The overall model Wald test is a test of the hypothesis that all of the coefficients in the model are simultaneously zero. It is an exceptional situation where anybody cares about that. The typical situation where one would be need that statistic is if the model variables were all just indicators for levels of a single category variable ("dummy variables" for a single categorical variable), and the research question is about the association of that category variable with the outcome. But that is clearly not the case in your output. Perhaps your research hypothesis really is about all of these variables (including the year indicators) being zero simultaneously--but it would surprise me.

If your research goals really include testing the hypothesis that all of the coefficients are simultaneously zero (which means that none of your variables are there just as "control variables,") then you cannot attain that goal until you expand your data set to include more countries.

[Clyde Schechter]

You have too few observations per country to do this estimation. Take a look at the results you are getting for the year indicators. Most of these coefficients are ridiculously large. Take 1997. A coefficient of -86.7 corresponds to an odds ratio of about 2 x 10^-38. That clearly doesn't happen in real life. What is causing this is that after Stata weeds out observations with missing values in any variable, and after it drops any countries where there is no variation in the outcome (which it must do in -xtlogit, fe-), it is probably left with nearly all observations in 1997 having reelection == 0. So the maximum likelihood likelihood estimate is "close to negative infinity." Similar things are happening in most of the year indicators. Similar things seem to be happening with several of your other variables as well, though it is difficult to be sure because if the continuous variables are scaled such that their numeric values are very small, these large coefficients might be realistic. If I had to make a guess, I would imagine that in most of the countries there is very little variation in the reelection outcome year to year, so Stata is trying to estimate a bunch of effects that are very close to zero. In linear models this isn't terribly difficult for Stata, but in logistic models, things can easily blow up because the maximum likelihood estimates are close to infinite (and biased away from 0 when the sample is small.) If you had a very large data set, you might be able to get convergence in the face of this problem. But with a small data set you either get non-convergence, or convergence to a set of results that, if given a moment's thought, are not possible in the real world.

You may be wondering why things worked out OK in the original -xtlogit-. The difference is that because of the different patterns of missing data and excluded observations due to non-variation in outcome, your estimation sample size has dropped from the original 26 countries and 494 observations to 20 countries and 101 observations. So you've lost 80% of your sample here and what remains is just too skimpy to support a maximum likelihood estimation.

[Clyde Schechter]

Well, I'm reluctant to recommend other estimators here because your sample size is so small that I doubt you can get really usable results even if you find one that converges. Your loss of sample size here is not just due to the lack of variation in outcome: that only affected 6 countries and 30 observations, but you lost hundreds. So the bulk of them are being lost due to missing data on the variables in your model, and that won't change no matter what estimator you use. I really think the fundamental problem here is that you are asking more of your data than it can give you. As they say, you cannot get blood from a stone.

That said, here are some possibilities:

1. Try a linear probability model, using -xtreg- instead of -xtlogit-. At least you are less likely to run into convergence problems here. Now, given poor variation in outcome, it may be that you have a lot of predicted probabilities close to 0 or 1, and that is a situation where linear probability models perform poorly and do things like predict probabilities less than 0 or bigger than 1 (especially in small samples!)

2. Consider using -logit- instead of -xtlogit-. If your group-level variance is small, then this won't throw things off by all that much. And it may be that there is enough variation in outcome across countries that you get reasonable effect estimates.

But I'm pessimistic that either of these will really solve your problem. You will probably get both of these to converge--these models have much simpler likelihood functions. But I suspect the results will not be very helpful.

3. Greatly simplify your model and stick with -xtlogit, fe-. Eliminate variables that are not crucial. This will increase your sample size, because there will be fewer opportunities for an observation to be dropped for missing values. And it also simplifies the likelihood function so that convergence is easier. Moreover, it brings the amount of information you are trying to extract from the data more in line with the amount of information in the data. I think this is actually your best bet, although it means answering a different question than the one you hoped to.