Foreach: testing each value of one variable against the corresponding value of another variable

[Jennifer Hauschildt]

Hello,
I am using StataMP 18 on windows 10.

I have a variable "citypart" that describes from 1 to 99 in which city part of my city participants live in. I also have a variable called "propfem" that describes the proportion of females in each city part (so correspondingly coded from 1 to 99). Furthermore I have the variable "sex" that tells me if a participant is 1 = female.

Now I want to check for each city part if the proportion of females in my dataset is approx. equal to the actual proportion of females in that city part.

I came up with the following code:

levelsof citypart, local(citypartlevels)
 foreach 1 of local citypartlevels{
     sum sex if citypart== `1' 
     local N_1 = r(N)
     local av_1 = r(mean)
     local sd_1 = r(sd)
    
      levelsof propfem, local(propfemlocal)
 foreach 1 of local propfemlocal{
     sum propfem if propfem== `1'
     local meanpropfem = r(mean)
 
 ttesti `N_1' `av_1' `sd_1' `meanpropfem' 

 }
 
 }

This technically gives me my desired results but this way the ttest is computed not merely for the corresponding value of propfem but for all values of propfem. Since I have 99 city parts this takes a lot of time and giving me a lot of unnecessary results. Is there a way that I can get the ttest only for the corresponding values of citypart & propfem?

Any advice is highly appreciated

[Jennifer Hauschildt]

I just realised that what I was doing does not make sense. What I imagined the propfem variable to be was a variable that was coded 1-99 and at the same time has the value of the proportion of females in it. Obviously that is not possible, it was just a brain fart.

Nevertheless, my question remains: Is there a way to automate that Stata is always only testing the two corresponding values in the loop? Maybe with an if condition or something?

[Clyde Schechter]

Well, the reason you are getting so many redundant comparisons is that you have nested one loop inside another. There is no need for two loops here. If the variable propfem really represents the proportion of females in the observation's citypart, then it must be constant within that group of observations. So you could do this with a single loop

 levelsof citypart, local(cityparts)
 foreach c of local cityparts {
     summ propfem if citypart == `c', meanonly
    assert r(min) == r(max) // VERIFY propfem CONSTANT WITHIN CITYPART
    local reference_value = r(min)
    ttest sex = `reference_value' if citypart == `c'
 }

I will also note that when you do encounter a problem that requires nested loops, you should not use the same name for the iterator in the inner and outer loops-it makes it impossible to refer to the outer loop's iterator value from inside the inner loop. And if there is no need to refer to the outer loop's iterator from inside the inner loop, then there is a good chance that you really don't need nested loops in the first place. In any case, it is potentially confusing to human reader's of the code. (It is not confusing to Stata, but the unambiguous interpretation of it that Stata uses also may not be what you want.)

As an aside, I would question operationalizing "the proportion of females in my dataset is approx. equal to the actual proportion of females." Unless what you are trying to do is verify that the sampling scheme used to accrue the people in your data set produced results consistent with a simple random sample, this is not a very helpful way to do it. For one thing, if the sample sizes differ across cityparts, then this makes "approx. equal" have a different meaning in the larger and smaller parts. It will be a stringent test of near equality in the larger parts and it will be very lax indeed in the tiny parts. The most effective way to operationalize approximate equality will depend on the particular reason you want to even ask the question, so I can't recommend what is best for your circumstance. I'm just pointing out that there are few circumstances where a t-test will be the best way.

[Daniel Schaefer]

I agree with #2. I don't think you need the inner loop. Clyde is also correct that you should not use the same local name for the inner and outer loop. I think it might actually work here since the outer index isn't referenced in the inner loop, but in any case it is bad practice. Here is what I came up with:

levelsof citypart, local(citypartlevels)
foreach l of local citypartlevels{
    sum sex if citypart== `l'
    local N_1 = r(N)
    local av_1 = r(mean)
    local sd_1 = r(sd)
    sum propfem if propfem == `l'
    local meanpropfem = r(mean)
    ttesti `N_1' `av_1' `sd_1' `meanpropfem'
    }
 }

Though, shouldn't this be a prtest rather than a ttest? You want the z rather than t distribution for this hypothesis, right?

[Mike Lacy]

I'm seeing this question as pointing toward a traditional one-sample chi-squared goodness of fit test. If the data were conceived as involving one observation for each city part, there would be an observed frequency of females in the sample, and an expected number of females based on the population proportion of females and total sample size within each city part. So, this would be a categorical variable with 99 values, with observed and expected frequencies for each value. There has been a user-written program for this, -csgof-, but it seems not to be available now.

So, what if the data set was collapsed to 99 observations, with an observed and expected frequency of females for each one, and the chi-square component [(Obs-Expected)^2]/Expected
calculated for each one? The test statistic would be the sum of these components, compared against the chi-squared critical value for df = 98.

Now, that being said, I wouldn't find an hypothesis test very interesting. I think I'd rather calculate some descriptive measure for the collapsed data set, say the percent difference in observed vs. expected proportion of females for each city part, this latter suggestion resonating with Clyde's point that the practical importance of observed vs. expected differences would depend on the size of the proportions being compared.

[George Ford]

set obs 99
g id = _n
g propfem = runiform(40,60)/100
expand 100
bys id: g pid = _n
sort id pid
g female = rbinomial(1,0.5)

collapse (mean) female propfem (sd) sdfem = female (count) N = female, by(id)

forv i = 1/99 {
    foreach var in female sdfem N propfem {
        local `var' = `var'[`i']
    }
    qui ttesti `N' `female' `sdfem' `propfem'
    local star = (cond(r(p)<.01, "***", cond(r(p)<.05, "**", cond(r(p)<.10, "*", ""))))
    di "ID = `i' :  t-stat = " %5.3f r(t) "`star'"
}

[Jennifer Hauschildt]

Thank you all for your responses and the tip about nested loops! They helped me a lot

I concluded on this final code

levelsof cityparts, local(citypartslevels) 
foreach l of local citypartslevels{
    sum sex if cityparts==`l' [aweight=weight]
    local N_1 = r(N)
    local av_1 = r(mean)
    local sd_1 = r(sd)
    sum propfem if cityparts==`l'
    local meanpropfem = r(mean)
    ttesti `N_1' `av_1' `sd_1' `meanpropfem'
    }

The reason I want to compare this is that we received a weight variable for the entire city and we want to test whether this is also applicable to the city parts or if we have to create our own weight variable for the city parts as for example the proportion of females might vary between city parts.

Daniel Schaefer

Though, shouldn't this be a prtest rather than a ttest? You want the z rather than t distribution for this hypothesis, right?

Could you please elaborate why you think a ttest is not appropriate here?

[Daniel Schaefer]

The mean of a binary variable is a proportion, so the mean of sex in a given city part should just be the proportion of females in that city part. The sampling distribution of proportions does not follow a t distribution, it is best approximated by a z distribution, so you need a z-test, not a t-test.

I wouldn't actually expect the results would be all that different, particularly if you have a large sample size, I just think the z-test should be a bit more efficient here.

[Clyde Schechter]

Let me reiterate what I said in #3, now that O.P. has explained why she is doing this. I am not an expert in survey data analysis, though I do have a little bit of experience with it. I question whether the statistically significant difference approach being used here is an appropriate way to determine whether or not different weights need to be calculated for the city parts. I don't know what would be the appropriate method, but it would surprise me if this is it. I think advice from somebody with a good understanding of the underlying issues is needed here. If that isn't available locally, perhaps starting a new thread, with a title that clearly identifies the issue at hand, would be the best way to get it. (It is unlikely that survey experts who haven't already joined this thread will do so at this point.)

That said, assuming that O.P.'s approach turns out to be correct, let me comment on the z vs t issue. First, as Daniel Schaefer points out, with a large sample size the difference between a z- and t-test is negligible. For practical purposes, a sample size greater than 60 is large enough to go with the t-test. But if some of the N_1's are smaller than that, here is how you could get a z-test. Instead of the -ttesti- command, do this:

sum propfem if cityparts==`l'
local meanpropfem = r(mean)
glm sex if cityparts == `l' [aweight = weight]
lincom _cons = `meanpropfem'

[Daniel Schaefer]

Clyde Schechter why the code in #9 rather than the -prtest- (or prtesti) command? Something to do with the weights?

[Clyde Schechter]

Right, -prtest- does not support weights.

[George Ford]

Surveys usually describe how a weight is computed. You could look that over for a possible reweighting solution.

I suppose you could collapse the data and then regress the weight on the Xs to see if female has much of an influence, and possibly use that to create a new weight. This is an uniformed guess.

[George Ford]

And check whether the weight even matters that much. Sometimes it doesn't.

[Jennifer Hauschildt]

Right, -prtest- does not support weights.

But if I just use prtesti instead the weights should not be a problem, no?

 levelsof cityparts, local(citypartslevels)
 foreach l of local citypartslevels{
    sum sex if cityparts==`l' [aweight=weight]
    local N_1 = r(N)
    local av_1 = r(mean)
    sum propfem if cityparts==`l'
    local meanpropfem = r(mean)
    prtesti `N_1' `av_1' `meanpropfem'
    }

[George Ford]

Isn't weight a constant within cityparts==`l'