Verify that differences across countries are not driven by sample size

[Sabina Avdagic]

I am analyzing data from a survey experiment in three countries. The sample size differs across countries - roughly 2000 in country A, 3200 in country B and 4500 in country C. The aim of the study is to see how the framing of a salient political issue influences support for an expansion of policy X.

Ordinal logit was used to analyze the effect of the treatment as the dependent variable ranges from strongly agree to strongly disagree. There are two treatment groups and one control group. Separate models were run for each country.

The expected effect of framing is evident in all countries, but the predicted probabilities suggest that the effect of the treatment was stronger in countries A and B than country C. In addition, when the treatment is interacted with some pre-existing attitudes, some differences emerge across countries, i.e. not all interactions are significant in all countries.


I was asked to verify that differences between countries are not partly driven by differences in sample size. Is there a simple way to do this in Stata?

Thank you.

[Marcos Almeida]

If I understood correctly your query, you may simulate with a (random) fraction of the sample, and for this we can use - sample - command, or you may simulate with an expanded version of the data, and for this we can use - expand - command.

[Sabina Avdagic]

Thank you, Marcos. Using -sample- is probably the best way to do this.

Best regards,
Sabina

[Phil Bromiley]

To add to Marco's suggestion, you'll increase your chances of a useful answer by following the FAQ on asking questions - provide Stata code in code delimiters, readable Stata output, and sample data using dataex. Knowing exactly what you've got and what you're estimating would help us help you.

While statistical significance of individual parameters varies with sample size, the parameters don't in general. I think you can set up the tests across the countries without worrying much about sample size. Often, it is easiest to do this by estimating all the data in one model using dummy variables (see factor variable notation) to allow separate parameters for each country.