Chi2 test returns no result

[Dave Velthoff]

Hello,
I ran the Pearson Chi Square test using the command "tab var1 var2, chi2", where var1 and var2 are replaced with the actual variable names.

The results are unclear. The output is:
- Chi2 = .
- p = .

Frankly, for both variables, sufficient data of participants are available.

What is the best way to find out what happened here?

Thank you.

Best, Dave

[Carlo Lazzaro]

Dave:
can you please post an excerpt of your data vi a-dataex-?

[Joseph Coveney]

The output is:
- Chi2 = .
- p = .

What's the rest of the output?

[William Lisowski]

If you are running Stata 15.1 or a fully updated Stata 14.2, it should be part of the installation. But if you're off-internet, the likelihood of that seems remote.

I second Joseph's question: can you post the tabulation output, or are there so many categories of your variables that it is excessively large?

[Nick Cox]

Dave: Given that you can access this forum you can also access its FAQ Advice:

12.2 What to say about your data

We can understand your dataset only to the extent that you explain it clearly.

The best way to explain it is to show an example. The community-contributed command dataex makes it easy to give simple example datasets in postings. It was written to support Statalist and its use is strongly recommended. Usually a copy of 20 or so observations from your dataset is enough to show your problem. See help dataex for details.

As from Stata 15.1 (and 14.2 from 19 December 2017), dataex is included with the official Stata distribution. Users of Stata 15 (or 14) must update to benefit from this.

Users of earlier versions of Stata must install dataex from SSC before they can use it. Type ssc install dataex in your Stata.

The merits of dataex are that we see your data as you do in your Stata. We see whether variables are numeric or string, whether you have value labels defined and what is a consequence of a particular display format. This is especially important if you have date variables. We can copy and paste easily into our own Stata to work with your data.

If your dataset is confidential, then provide a fake example instead.

In this instance, however, seeing the tabulate output should be enough to allow comment.

[Carlo Lazzaro]

Dave:
as there are no male (female) patients in the Comparison (Intervention) Group, as you suspect chi2 has nothing to say.

[Nick Cox]

I can't reproduce this report of missing chi-square or P-value, with either the official distribution or community-contributed commands.

The P-value is very small but it's not missing. Statistically, the cross-tabulation is as far from independence as you can get -- so there's a research question of whether there is anything to learn -- but the set-up is not pathological.

Here I ran tabchii from tab_chi on SSC as well as official commands (tabi leaves variables in memory when applied with an empty dataset).

. tabchii 35 0 \ 0 33

          observed frequency
          expected frequency

--------------------------
          |      col      
      row |      1       2
----------+---------------
        1 |     35       0
          | 18.015  16.985
          | 
        2 |      0      33
          | 16.985  16.015
--------------------------

         Pearson chi2(1) =  68.0000   Pr = 0.000
likelihood-ratio chi2(1) =  94.2092   Pr = 0.000

. tabi 35 0 \ 0 33

           |          col
       row |         1          2 |     Total
-----------+----------------------+----------
         1 |        35          0 |        35 
         2 |         0         33 |        33 
-----------+----------------------+----------
     Total |        35         33 |        68 

           Fisher's exact =                 0.000
   1-sided Fisher's exact =                 0.000

. l

     +-----------------+
     | row   col   pop |
     |-----------------|
  1. |   1     1    35 |
  2. |   1     2     0 |
  3. |   2     1     0 |
  4. |   2     2    33 |
     +-----------------+

. tab row col [fw=pop]

           |          col
       row |         1          2 |     Total
-----------+----------------------+----------
         1 |        35          0 |        35 
         2 |         0         33 |        33 
-----------+----------------------+----------
     Total |        35         33 |        68 


. tab row col [fw=pop], chi2

           |          col
       row |         1          2 |     Total
-----------+----------------------+----------
         1 |        35          0 |        35 
         2 |         0         33 |        33 
-----------+----------------------+----------
     Total |        35         33 |        68 

          Pearson chi2(1) =  68.0000   Pr = 0.000

Here a P-value of 0.000 just means <0.0005. Here, it's about 2e-16.

[William Lisowski]

My experience was similar to Nick's, in that before my post #5 I tried to confirm my intuition of zeroes in the table, but could not find a way to make the tab or tabi commands generate a missing value for chi2.

Dave, what version of Stata are you using? Can you show us the first two lines out of the about command?

. about

Stata/SE 15.1 for Mac (64-bit Intel)
Revision 13 Feb 2018

And also confirm that you don't have some other tab command installed that overrides the built-in command.

. which tab
built-in command:  tabulate

It would be instructive to learn where this unintuitive behavior is coming from. The only way I can see to get an undefined result in the manual calculation of chi squared is to have a row or column with a zero margin, which is not going to happen in Stata. I think.

[Nick Cox]

Dave: Sorry, but I don't fully understand how your extra information bears on the question you asked. Conversely, if your set-up doesn't allow you to tell us what we need to know, that's understood, but you'll need to seek support in-house.

For your chi-square example, missing chi-square statistics make no sense and can't be reproduced.

You could easily get missing t-statistics for a comparison of means if there is no variability in what you're comparing. Here's a silly example below.

You can run this on your system.

. sysuse auto, clear
(1978 Automobile Data)

. ttest foreign, by(foreign)

Two-sample t test with equal variances
------------------------------------------------------------------------------
   Group |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
---------+--------------------------------------------------------------------
Domestic |      52           0           0           0           0           0
 Foreign |      22           1           0           0           1           1
---------+--------------------------------------------------------------------
combined |      74    .2972973    .0534958    .4601885    .1906803    .4039143
---------+--------------------------------------------------------------------
    diff |                  -1           0                      -1          -1
------------------------------------------------------------------------------
    diff = mean(Domestic) - mean(Foreign)                         t =        .
Ho: diff = 0                                     degrees of freedom =       72

    Ha: diff < 0                 Ha: diff != 0                 Ha: diff > 0
 Pr(T < t) =      .         Pr(|T| > |t|) =      .          Pr(T > t) =      .

In your case, if all one group was 19 years old and all the other group was 21 years old, there is no variability to act as context.

[Chen Samulsion]

Was something deleted between #5 and #6? I just cannot follow the sudden advice gave in #6 without a data example before it.

[William Lisowski]

All of David's posts after his first post have been deleted.

In one of them he indicated that he could not post any Stata output without prior approval from the source of the data, and that he is working by remote access on a server rather than on his desktop system. I'm guessing that somewhere along the line he posted something he realized was sensitive and deleted them.

Between the current #5 and #6 David posted the results of his tabulation. The current posts #6 and #7 capture the salient feature of the deleted tab: it was a 2-by-2 tab with two cells of zero. And the problem is, neither Nick nor I (nor perhaps others who tried) have been able to make the current (Stata 15.1) version of tab report missing values for the chi2 and its p statistic.

Between the current post #7 and #8 David reported doing a t-test on similarly distributed data and having the t-test report missing values for the difference in the means. It is that to which Nick responds in the current post #9, and the key point he makes there is that the t-test is irrelevant to the underlying issue:

For your chi-square example, missing chi-square statistics make no sense and can't be reproduced.

The only hypothesis for this problem currently on the table are the questions I asked in the current post #8, focused on the idea that on David's system the tab is not the tab he is looking for; alternatively, although he keeps using that tab, I do not think it does what he thinks it does.

That is, perhaps the server he is forced to use is not kept well up-to-date, or has some other things going on that mean the tab output he is receiving is not from the tab command built into Stata 15.1.

If David returns and sees this, my advice is that he should try the commands I recommended and see if they report anything suspiciously different, and perhaps try the following version of Nick's final example in post #7 to see if his results match Nick's. That will arm him with some facts to share with his local Stata support.

clear
input byte(row col pop)
1 1 35
1 2  0
2 1  0
2 2 33
end
tab row col [fw=pop], chi2

. tab row col [fw=pop], chi2

           |          col
       row |         1          2 |     Total
-----------+----------------------+----------
         1 |        35          0 |        35
         2 |         0         33 |        33
-----------+----------------------+----------
     Total |        35         33 |        68

          Pearson chi2(1) =  68.0000   Pr = 0.000

[Chen Samulsion]

William Lisowski, thank you. It must be hard for me to memory those posts that have disappeared. But without sufficient information or with perhaps wrong information about Stata's results and command operate, what helpers to do is like to attack straw man or tilt at windmills.

[Nick Cox]

The forum administrators will on request delete posts that are sensitive. Meanwhile the advice for those in Dave's situation has been there all along:

If your dataset is confidential, then provide a fake example instead.

Stata version is I suggest unlikely to be an issue here. For example, Stata 10.1 reproduces the same chi-square result.

I don't think there's anything else we can say unless and until Dave adds more detail.