Forloops and variable generation

[Taylor Vail]

I have a series of variables fam1 - fam14 that represent sex and age of all family members. They were entered as H45 or M35 (where H=male and M=woman, and their age after the letter). I need to separate the age from sex in order to run any analyses


I've figured out that I can go line by line, and make male1 - male14 to represent the sex of each family member and age1 - age14 to represent their age with the following commands:

gen male1 = regexs(2) if regexm(fam1, "(([A-Z]+)*([0-90 - 90-9]+))")
gen age1 = regexs(3) if regexm(fam1, "(([A-Z]+)*([0-90-90 - 9]+))")

However, I know this is rather labor intensive and feel like I should be able to do this with a loop, but can' seem to figure it out.

Any guidance would be greatly appreciated.

Thanks,
Taylor

[David Benson]

Hi Taylor, welcome to Statalist!

A forvalues loop is what you want and I would use Stata's substr() function rather than regular expressions.

Doing it with regex

forvalues i=1/14  {
gen male`i' = regexs(2) if regexm(fam`i', "(([A-Z]+)*([0-90 - 90-9]+))")
gen age`i'   = regexs(3) if regexm(fam`i', "(([A-Z]+)*([0-90-90 - 9]+))")
 }

Doing it with substr() and destring

forvalues i=1/14  {
gen male`i' =  (substr(fam`i', 1, 1)=="H")
destring male`i', gen(age`i') ignore("HM")
 }

[William Lisowski]

Welcome to Statalist, Taylor.

You can use a forvalues loop to streamline the code you have from 14 pairs of commands to a single pair.

forvalues i=1/14 {
  gen male`i' = regexs(2) if regexm(fam`i', "(([A-Z]+)*([0-90 - 90-9]+))")
  gen age`i' = regexs(3) if regexm(fam`i', "(([A-Z]+)*([0-90-90 - 9]+))")
}

Added in edit: this post crossed with #2, in which David's elegant second approach is the one I would recommend.

[Taylor Vail]

Hi David,

It seems like your code works to turn male1 into a numeric where 1 = males and 0 =females but it seems like there's a step missing as none of the age variables are generated and all subsequent male`i' variables are all 0 after male1 - any thoughts, seems like maybe you can't do both of these within the same loop?

In running the loop I kept getting this feedback for each male`i':

male1 already numeric; no generate
male2 already numeric; no generate

etc.

Thank you for your guidance!

[David Benson]

Sorry, I should have written it as:

 forvalues i=1/14  {
gen male`i' = (substr(fam`i', 1, 1)=="H")
destring fam`i', gen(age`i') ignore("HM")
}

Also, note that you'll need to drop the male1-male14 variables before re-running this code (or you'll get an error like "variable male1 already defined.") Or, just run the loop over the destring command.

[Taylor Vail]

Thank you for your help! I was able to successfully get the ages extracted, however I'm still not having much success with the male/female portion of the loop.

Using the code you provided in #5 I'm able to generate the proper amount of males=1 and females=0, for male1, however, for all following male2 - male14 they all equal 0 which isn't correct. Also, it seems to be generating the variable without reference to how many observations are in the corresponding fam`i'

Apologies for the additional queries - thank you for your help.

~Taylor

[David Benson]

Hi Taylor,

Not sure why it works for male1 but not male2-male14, but a couple of thoughts (or things you might try):

1. I wondered if your data was initially in a foreign language (I wondered that because it was coded H=male and M=woman), and so maybe it wasn't really an H but a foreign letter / character with an umlaut or tilde or something.
2. substr() is case sensitive and so "H" is treated differently than "h".
3. Sometimes the letter "H" or "M" isn't first (ie it was coded as "45H" rather than "H45", or more likely, there is an extra space at the beginning of observation " H45" rather than "H45"

Of all of these, I suspect it is #3, because you said creating the age variables worked fine and the problems #1 and #2 mean they wouldn't.

forvalues i=1/14 {
replace fam`i' = trim(fam`i')
replace fam`i' = itrim(fam`i')
replace fam`i' =  strupper(fam`i')
}

You could also check whether there are spaces (or funny characters) in your values by doing the following (note that charlist (SSC) is a user-created command that can be installed using ssc install charlist.

gen sex2 = substr(fam2, 1, 1)  // This should always end up being "H" or "M".
​​​​​​​* If it is anything else or blank, then you know whether the problem is #1-#3 above.
* I'm just creating one here for simplicity.  Obviously, you would want to do this for all of them, or run them through a loop. 

ssc install charlist  // just to install the package if you haven't already

forvalues i=2/14 {
charlist fam`i'
}

Regarding

Also, it seems to be generating the variable without reference to how many observations are in the corresponding fam`i'

You should update the code so that instances where fam`i'=="" don't get set to ==0 or ==1.

forvalues i = 1/14 {
gen male`i' = 0 if fam`i' !=""
replace male`i' = strpos(fam`i', "H") > 0
}

Finally, if you can't get it to work, I often use the LEFT() and RIGHT() functions in Excel for stuff like this.

[David Benson]

If none of those solve the problem, you'll probably need to post a sample of your data using -dataex-

[David Benson]

Sorry, just realized I left something out in that last piece of code. It should be:

forvalues i = 1/14 {
gen male`i' = 0 if fam`i' !=""
replace male`i' = 1 if strpos(fam`i', "H") > 0  
 }

[Taylor Vail]

Thank you so much for your help, it now works! It must have been that the data were entered in with extra spaces at the beginning or end of the observation - I greatly appreciate all of the support.
~Taylor