Alternative to Chi squared and Fisher's Exact

[Jens Moller]

Hello.

I have some problems investigating differences between five groups with categorical outcome (four values). The number of observations varies between 250 and 400. The expected values are smaller than five, and when performing Fisher's exact, an error occurs: "exceeded memory limits using exact(1); try again with larger #; see help tabulate for details"

Are there any ways to analyze the differences without collapsing categories?

[Marcos Almeida]

You just need to add a larger number under the - exact() - option.

Please keep in mind that Fisher's exact test is time consuming.

[Jens Moller]

I have tried increasing the number a bit without solving the problem.

Can I just raise the number without any concerns?

[Marcos Almeida]

This is the information you get in the Stata Manual

exact[(#)] displays the significance calculated by Fisher's exact test and
may be applied to r x c as well as to 2 x 2 tables. For 2 x 2 tables,
both one- and two-sided probabilities are displayed. For r x c
tables, two-sided probabilities are displayed. The optional positive
integer # is a multiplier on the amount of memory that the command is
permitted to consume. The default is 1. This option should not be
necessary for reasonable r x c tables. If the command terminates with
error 910, try exact(2). The maximum row or column dimension allowed
when computing Fisher's exact test is the maximum row or column
dimension for tabulate (see limits).

[Jens Moller]

Thank you. I still not quite understand the impact of increasing the number. Some of the commands can now be performed using 10. However, I do not know if increasing the number is "allowed" or will compromise the p value.

[Marcos Almeida]

Please read the text above. It says a)that it is an issue related to the memory; b) the maximum number allowed.

[Mike Lacy]

You might well have a problem too large for an exact test, even with a smart algorithm that avoids looking at the really low probability arrangements of the table. A simple and reasonable alternative here is to use -permute-. If you use, say, 100,000 reps, you'd get a p-value accurate to about 1e-3. (-permute- in fact offers a confidence interval on the p-value if it matters). You'd need to use Chi-Squared or something else fast as the test-statistic, but that should not be a problem:

permute x r(chi2), reps(100000): tab2 y x, chi2

[Jens Moller]

Thank you both. I will try to read a bit about the possibilities.