Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Comparing proportions in two independent samples

    Hi Statalisters,

    I have two (probably) independent samples taken from the same population under two different surveys conducted a few years apart. I'd like to check whether the frequencies for a particular indicator match between the samples. I don't expect these proportions to change much over time (though I can't rule it out). The indicator uses multiple categories which are mutually exclusive and not ordinal, e.g.:

    What is your car's color?
    Red
    Green
    Blue
    Black
    etc.

    I want to check that survey A captured, basically, the same proportions of red/green/blue that survey B did. I *think* I should be using a Chi-square goodness of fit for this (csgof); except that I'm comparing two samples, rather than the sample versus the population. Anyone know what other statistical test I should be using? I'm a bit stuck, and am considering just eyeballing it: using svy: tab on both and just comparing the confidence intervals. But I feel like this is a crude way of doing things.

    Thanks very much!

    a

  • #2
    Hi Angela,
    It's OK to use an ordinary chi-square test of a 2xK crosstable if you hypothesis is that the relative distribution of the K categories are the same in both samples.
    Valter Sundh

    Comment


    • #3
      Angela,

      If you were comparing the distribution on that categorical variable between two groups in the same survey, a chi-squared test should be straightforward. If each of your two surveys has its own weights, the situation is more complicated. I have not done that sort of comparison, and perhaps another member with more survey experience will explain how to proceed.

      Comparing the confidence intervals is probably not a good idea. Some people mistakenly think that the difference between two estimates is statistically significant if and only if the two confidence intervals overlap. The correct general idea is that the difference is significant if the confidence intervals do not overlap, and it may also be significant if the confidence intervals do not overlap too much. The appropriate way to assess the significance of a difference is to ask whether the confidence interval for the difference includes zero. In your example the categories are not ordered, so it's not clear what confidence intervals you might compare.

      David Hoaglin

      Comment


      • #4
        Thanks very much, Valter and David. This is reassuring. I'll investigate the weight issue and see how I can incorporate that into csgof.

        For eyeballing confidence intervals, I've run svy: tab [var], ci, and it's returned lower and upper bounds for each proportion. So, for example, the 95% interval of people owning a red car was 15% - 17%, with the reported proportion being 16%. For blue cars, it was 11% - 12%, reported proportion being 11.65%, and so on. My second-best idea, then, was to run svy: tab on the carcolor variable for each survey, and then check to see that the confidence intervals overlap. If they do, then I "conclude" that survey A is probably, mostly just as good as survey B. If they don't overlap, then I assume that something is up with survey A - and I may be worried about its data quality, measurement error, survey design, or whatever. As I said, though, I'd like to avoid doing it this way and instead rely on a t-test-type comparison.

        a

        Comment


        • #5
          hello

          Please I construct an political index with mean and median approach ponderation. after it, I want to do a test to compare a similarity of this two index.

          Please somebody can help me with the name and the stata synthax test that it is good to me to do. Please

          Comment

          Working...
          X