2 sequential loops, second loop only runs those instances that converged in the first

[Michael Weinerman]

Hello all,

I'm running several instances of a maximum likelihood routines in a first set of loops. In the second loop I'd only like to run the subset of instances that actually converged, rather than the full set that was run in the first loop. It seem like the solution would be based on setting a set of conditional locals, but I've tried a few attempts at this with no success. Below is some stylized code to show what I'm trying to accomplish. Ideally, this would log and then call every treatment-outcome combinations that converged, but it's sufficient to just log each treatment for which any outcome converged. I'm running this code over several groups, so any reduction in computer run time over the second set of loops will save me a lot of time. I'm sure my issue is with how I'm using embedded quotes.Thanks in advance for any guidance!

foreach treatment in aaaa bbbb cccc dddd eeee {
        foreach outcome in nnnn mmmm oooo pppp {
        
        [estimation code]

        local zzzz [...generate local/ add to local based on each treatment (ideally, each treatment-outcome combination) that converged using e(converged)]
}
}

foreach treatment of local zzzz {
        foreach outcome in nnnn mmmm oooo pppp {
        
        [estimation code]
}
}

[Clyde Schechter]

If your estimation code relies on Stata's -ml maximize-, or uses one of Stata's regression commands, e(converged) is returned in e(). So you can do it something like this:

local zzzz
foreach treatment in aaaa bbbb cccc dddd eeee {
    local `treatment'_wwww
    foreach outcome in nnnn mmmm oooo pppp {
    
        [estimation code culminating in an estimation command or -ml maximize-]
        if e(converged) {
            local zzzz `zzzz' `treatment'
            local `treatment'_wwww ``treatment'_wwww' `outcome'
        }
    }
}

local zzzz: list uniq zzzz // REMOVE DUPLICATES

foreach treatment of local zzzz {
    foreach outcome of local `treatment'_wwww {
    
        [estimation code] // PRESUMABLY INVOLVING `TREATMENT' AND `OUTCOME'

    }
}

That said, I think this approach has a couple of problems.

First, for those combinations of treatment and outcome that do not converge, they are going to spin their wheels for a very long time before they give up. You probably should try to prevent that by adding an -iterate(#)- option to the estimation command itself, so that if it doesn't converge by # iterations, it quits, with # being some number large enough to give a high probability that convergence, if possible at all, will occur within that many iterations, but small enough that you are not waiting "forever" to find out that a model will fail to converge. I don't know quite what kind of models you are trying to fit here, but for the usual type of standard regressions, in my experience, it is rare to obtain convergence if it hasn't occurred within 50 iterations. Perhaps for SEM one should be a bit more patient, say 100 or 200.

Second, you have provided pseudocode, and perhaps a great deal of detail has been obscured as a result, but it looks like your final loop over the treatments that had convergence with some outcome just repeats what was done with them in the first loop, although perhaps your intention is that a different kind of model will be fit to the same combinations in this loop.

Note, for obvious reasons this code is untested. But it should set you on the right path.

[Michael Weinerman]

Thanks for the response, Clyde. I adapted your code and I'm hopeful this will do the trick!

Yes, I already have the iterate option specified (at 50), but I didn't want to bog down this post with all the estimation code since I'm not troubleshooting that. And yes, a fair amount of intent is obscured here. Sorry for that. Yes, the two sets of loops have different models, but I only need to run the second one if the first one converged.