Power analysis - experiment with binary outcome

[Owen Gallupe]

Hi,

I'm interested in conducting a power analysis for an experiment with a 4x3 factorial design (so, 12 groups). The outcome is dichotomous (did participant cheat on task? 0=no, 1=yes). I'm wondering if my approach seems reasonable (and if not, I'm hoping someone would be kind enough to provide guidance on the proper approach).

A few points based on previous work:

• the cell with the highest proportion of cheaters is expected to have a proportion of 0.15 (15% cheat).
• the lowest expected proportion is 0.00 (0% cheat).
• the overall proportion expected to cheat is 0.05 (5%) (standard deviation = 0.21).
• there is no longitudinal component to the experiment (no before/after comparison).

Between-group differences will be analysed either by a simple chi-square test or Fisher's exact test.

My initial reaction was to use the user-written fpower command (from http://stats.idre.ucla.edu/stat/stata/ado/analysis and using Stata 14) but since this is for ANOVA power analysis, I'm doubting my initial reaction. Nonetheless, this was my approach.

Delta for use in fpower was calculated by the following:
Delta = (largest mean – smallest mean)/SD
= (0.15 – 0)/0.21
= 0.714

fpower, a(4) b(3) delta(0.714) alpha(0.05)

The results show that 15 cases per group is required to achieve a power of 0.8.

Is this approach justifiable or would it better if I approached this another way?

Thanks!

Owen

[Note: Apologies if this has been asked previously...I wasn't able to locate it if has.]