Nested foreach Loop sequence & outreg2??

[Kasim Ortiz]

Hi:

I'm having difficulty in getting the code I've developed to run successfully. Basically I'm trying to run a loop of regressions (Log Poisson) with mi commands and svy commands. I keep obtaining either one of these error codes [required r(100); or invalid syntax r(198)]. Below is the fragment of code:

foreach dv of varlist smokenow exercise obese hivtest colonoscopy sad emotional trbsleep {
foreach level in `sexid2' {
foreach var of varlist race sex nbtenure hispanic edlevel marstat immigrant nohi srvy_yr age_p {
mi estimate, or post : svy linearized : logit `dv' ib`level' `var', eform
outreg2 using "C:\Users\ortiz_000\Documents\Data\NHIS\Neighb orho odCohesionAll2.xls", append sideway bdec(2) cdec(2) label symbol(***, **, *) alpha (.001, .01, .05) stats(coef ci) eform cti("Model 1b, Smoker")
}
}
}


Also, it would be really helpful if I could get this to work with direct export to Word but Excel works only for me. Lastly, when the results are input into Excel in a series of regression, the results are not sequentially in the same rows. For example if I have 4 regression models, the 2nd model will be placed in the column directly adjacent to the 1st model but yet start in the immediate row following the previous model. If outreg2 is more cumbersome, I'd be extremely interested to see if anyone has a means for achieving this using estout.............

[Nick Cox]

You posted a slightly different version of this in http://www.statalist.org/forums/foru...oping-sequence on 6 April, bumped it on 12 April, alluded to it (without a cross reference) in http://www.statalist.org/forums/foru...n-another-loop on 13 April,and now here we are.

Please note general advice on posting code and explaining problems exactly within http://www.statalist.org/forums/help#stata ; on why questions don't get answered at http://www.statalist.org/forums/help#noanswer ; on bumping and reposting at http://www.statalist.org/forums/help#adviceextras #1

You're ignoring much of that advice despite the prompt to read the FAQ Advice that appears every time you post. That is, I guess, contributing to the lack of response. People can't read your code easily; they certainly can't test your code.

Turning to your code:

1. Setting

set trace on
set tracedepth 1

and showing the results of that trace would allow a clearer idea of what is happening. Here the depth of the trace is set at 1; you may need to bump that up.

2.

foreach level in `sexid2' {

That line refers to a local macro sexid2 we can't see. You need to show us the definition. Also, to my eye in your original in #1 there is no space between the end of the macro reference and the brace. That could be problematic. As copied here, it looks all right.

3.

mi estimate, or post : svy linearized : logit `dv' ib`level' `var', eform

There could be problems there with spaces too.

4.

outreg2 using "C:\Users\ortiz_000\Documents\Data\NHIS\Neighb orho odCohesionAll2.xls"

Spaces appear in that filename. Different spaces appear in your post on 6 April. I don't know whether that's problematic. It does, again, raise the question of whether what you are presenting is exactly what you type.

5.

You are not making clear how you run this code. It's essential to run it as a code block that each line of code that includes a local macro reference is aware of the definition of the macro.

6

outreg2 is a user-written program from SSC, as you are asked to explain. I never use it and don't answer questions on it. so I have essentially no idea on whether your syntax makes sense or should do what you want. It's a popular program but questions on it are often ignored. Its author is not a member here. Few of the most active people here seem to use it or to answer questions on it. That's no help, but part of your puzzlement is why your question is not being answered, and that's part of the story.

7.

Whether your data are suitable for what you are trying is another question. I have never seen logit regressions called "log Poisson".

The most important advice here is to show exact code using CODE delimiters as requested in the FAQ Advice and to use a code trace and to inspect the results minutely and if the problem is not then evident to show us the results. You're evidently trying different code and getting different errors but not typing the exact code together with the exact corresponding error message is too close to "I tried several things to get my appliance to work, but none was successful".

[Nick Cox]

I can add to an already long list. #1 alternates between telling us that you are getting error messages and telling us (I think) that you are getting some readable output, just not what you want. Those kinds of stories may be consistent, but they don't seem clearly consistent.

[Kasim Ortiz]

First, thanks so much for your very detailed advice. I now have a much better sense of how to use Statalist. Truly I am a learner by failure clearly! So here is my attempt to provide greater clarity to my question with much more detailed information.

Here the code that I've used:

Setting trace:

set trace on
set tracedepth 5

Establishing local macros:

local covariates race sex nbtenure hispanic edlevel marstat immigrant nohi srvy_yr age_p
local dv smokenow exercise obese hivtest colonscopy sad emotional trbslee 
local sexualorientation sexid2

Regression looping:

foreach `dv' of varlist smokenow exercise obese hivtest colonoscopy sad emotional trbsleep {
foreach level in `sexualorientation' {
foreach `covariates' of varlist race sex nbtenure hispanic edlevel marstat immigrant nohi srvy_yr age_p {
mi estimate, or post : svy linearized : logit `dv' ib`level' `var', eform
outreg2 using "C:\Users\ortiz_000\Documents\Data\NHIS\NeighborhoodCohesionAll2.xls", append sideway bdec(2) cdec(2) label symbol(***, **, *) alpha (.001, .01, .05) stats(coef ci) eform cti("Model 1b, Smoker")
}
}
}

Here is the error that I am receiving:

- foreach `dv' of varlist smokenow exercise obese hivtest colonoscopy sad emotional trbsleep {
= foreach smokenow exercise obese hivtest colonscopy sad emotional trbslee of varlist smokenow exercise obese hivtest colonoscopy sad emot
> ional trbsleep {
invalid syntax
r(198);

When I referenced being interested in running log-poisson models, I was referring to utilizing a GLM for the binomial family command:

svy linearized, : glm smokenow i.sexualorient i.age i.raceeth i.sex i.education i.employ i.income i.region ib2.currentsmoker, family(poisson) link(log) eform

I am utilizing Stata 14.2 on a PC (Windows 10).

[Nick Cox]

You have changed the code and introduced new errors, but the first error message is telling you where the first error is. The help for foreach -- e.g. http://www.stata.com/help.cgi?foreach -- explains that the typical call starts in one of these ways

foreach lname in

foreach lname of

lname is a single name that you provide. It can't be the name of a macro containing a list.

This may be closer to what you want, but I guess that ib needs some following punctuation. It's a matter of better readability, but that's still important whenever your code is buggy, to indent loops and not to repeat what you have already defined.

local X race sex nbtenure hispanic edlevel marstat immigrant nohi srvy_yr age_p
local Y smokenow exercise obese hivtest colonscopy sad emotional trbslee
local file C:\Users\ortiz_000\Documents\Data\NHIS\NeighborhoodCohesionAll2.xls

foreach y of local Y  {
    foreach level in sexid2 {
        foreach x of local X {
            mi estimate, or post : svy linearized : //
            logit `dv' ib`level' `var', eform

            outreg2 using "`file'", append sideway bdec(2) cdec(2) label ///
            symbol(***, **, *) alpha (.001, .01, .05) stats(coef ci)     ///
            eform cti("Model 1b, Smoker")
        }
    }
}

Note that you don't need a loop over a single item, but I've left that in place.

[Kasim Ortiz]

Hi Nick:

Thank so much for your input. This has been extremely helpful. So I modified the code to the following:

local X race sex nbtenure hispanic edlevel marstat immigrant nohi srvy_yr age_p
local Y smokenow exercise obese hivtest colonscopy sad emotional trbslee
local file C:\Users\ortiz_000\Documents\Data\NHIS\NeighborhoodCohesionAll2.xls

foreach y of local Y  {
    foreach level in sexid2 {
        foreach x of local X {
            mi estimate, or post : svy linearized : logit `Y' ib.`level' `X'

            outreg2 using "`file'", append sideway bdec(2) cdec(2) label symbol(***, **, *) alpha (.001, .01, .05) stats(coef ci) eform cti("Model 1b, Smoker")
        }
    }
}

Here is the error that is now stopping Stata from executing the above fragment code:

- di as err "an error occurred when {bf:mi} {bf:estimate} executed " an error occurred when mi estimate executed - di as err
"{bf:`cmdname'}" _c = di as err "{bf:svy:logit}" _c svy:logit - if ("`m'"!="") { = if ("1"!="") { - di as err `"`msg'"' = di as err `" on
{it:m}=1"' on m=1 - } - di as err "{p_end}"

r(111);

[William Lisowski]

Nick slipped, and your correction to the command was incorrect. I think you want

mi estimate, or post : svy linearized : logit `y' ib.`level' `x'

to substitute a single dependent variable and single independent variable rather than the full list of variables. Perhaps you also want the eform option that Nick specified and you omitted.

[Kasim Ortiz]

Thanks to both, William & Nick. I've been able to figure out what was going on. First, I had the path name incorrect for where the Table was located for the automatic generation. Second, I needed to specify each independent variable to change in each model but all of the covariates of X to stay the same. Below is the fragment of code that has worked:

local X race sex nbtenure hispanic edlevel marstat immigrant nohi srvy_yr age_p
local Y smokenow exercise obese hivtest colonscopy sad emotional trbslee
local file C:\Users\Owner\Desktop\P\Documents\Data\NHIS\NeighborhoodCohesionAll2.xls
foreach y of local Y  {
    foreach level in sexid2 {
        foreach x of local X {
            mi estimate, eform post : svy linearized : logit `y' i.`level' `X', or

            outreg2 using "`file'", append sideway bdec(2) cdec(2) label symbol(***, **, *) alpha (.001, .01, .05) stats(coef ci) eform cti("Model 1b, Smoker")
        }
    }
}

[Nick Cox]

You say you're happy but the innermost loop of your code as shown just fits the same models again and again

[Kasim Ortiz]

That's what happens when you walk away while the program is running and because it didn't produce any errors you think everything is fine! It took a good hour to run it had the same model repeatedly. I'm presuming your fix will correct this:

local X race sex nbtenure hispanic edlevel marstat immigrant nohi srvy_yr age_p
local Y smokenow exercise obese hivtest colonscopy sad emotional trbslee
local file C:\Users\Owner\Desktop\P\Documents\Data\NHIS\NeighborhoodCohesionAll2.xls
foreach y of local Y  {
    foreach level in sexid2 {
        foreach x of local X {
            mi estimate, eform post : svy linearized : logit `Y' i.`level' `X', or

            outreg2 using "`file'", append sideway bdec(2) cdec(2) label symbol(***, **, *) alpha (.001, .01, .05) stats(coef ci) eform cti("Model 1b, Smoker")
        }
    }
}

[William Lisowski]

I think this is what Nick had in mind in post #9.

local X race sex nbtenure hispanic edlevel marstat immigrant nohi srvy_yr age_p
local Y smokenow exercise obese hivtest colonscopy sad emotional trbslee
local file C:\Users\Owner\Desktop\P\Documents\Data\NHIS\NeighborhoodCohesionAll2.xls
foreach y of local Y  {
    foreach level in sexid2 {
        // no loop over X

        mi estimate, eform post : svy linearized : logit `y' i.`level' `X', or

        outreg2 using "`file'", append sideway bdec(2) cdec(2) label symbol(***, **, *) alpha (.001, .01, .05) stats(coef ci) eform cti("Model 1b, Smoker")

    }
}

[Chris Larkin]

A small addition, if you want to export the output direct to Word (instead of Excel) change the suffix of you local file to .rtf. On the resulting file, right-click and open in Word.

That line should read

local file C:\Users\Owner\Desktop\P\Documents\Data\NHIS\NeighborhoodCohesionAll2.rtf