Program code not exiting outer loop

[Girish Venkataraman]

Hello all:

I have this program to do single tabulations of frequency variables (as fractions, numbers and percentages) to use in putdocx. I was fiddling around trying to make the program better and found the code is now not exiting the inner loop and coming to do the math on the second variable that program is calling. The ereturn is only getting me the last variable in the program call. I can see both variables evaluating correctly in set trace but missing in the ereturn. Loop brackets and incrementing macro seem to be right. Have a reproducible example using the nhanes2 dataset.

*! d1tab.ado v1.1 GV 29nov2023-editing
*! d1tab.ado v1.0 GV 7nov2023


webuse nhanes2, clear

    // Program d1tab below Single tabulations fractions and percentages.
*--------------------------------------------------------------------

cap program drop d1tab
    program d1tab, eclass
    version 18.0
    syntax varlist(numeric min=1) [if] [, TEX(string asis)] [TEX2(string asis)] [TEX3(string asis)]
    marksample touse
set trace on
* Define all factor variables local fs
    foreach v of local varlist {
                        qui levelsof `v' if(`touse'), local(mylev)
                            if `r(r)' > 10 {
                                di as error "More than 10 levels; Check if continuous"
                            }
                        local maxlev = `r(r)'+1
                        qui estpost tab `v' if(`touse')
                        qui matmap e(pct) P`v', map(round(@, 0.1))
                        qui matmap e(b)   B`v', map(round(@, 0.1))
                        local `v'_tot      `=B`v'[1,`maxlev']'
                        eret local `v'_tot = "``v'_tot'"
                        local j 0
        foreach lev of local mylev{
                        local ++j
                        local `v'_`lev'n             `=B`v'[1,`j']'
                            eret local `v'_`lev'n         `=B`v'[1,`j']'    
                        local `v'_`lev'pct        :     di %3.1f   `=P`v'[1,`j']'
                            eret local `v'_`lev'p =      string(`=P`v'[1,`j']', "%3.1f") + "%"
                            eret local `v'_`lev'pn         ``v'_`lev'pct'% (n = ``v'_`lev'n')
                            eret local `v'_`lev'np         ``v'_`lev'n' (``v'_`lev'pct'%)
                            eret local `v'_`lev'fn        "``v'_`lev'n'/``v'_tot'"
                            eret local `v'_`lev'fp        "``v'_`lev'n'/``v'_tot' (``v'_`lev'pct'%)"
                            eret local `v'_`lev'pf        "``v'_`lev'pct'% (``v'_`lev'n'/``v'_tot')"

        }
        cap matrix drop _all
    }
    * Display header text
*---------------------    
    di as text "{hline 65}"
    di as text "  Tabulated variables {c |}" _column(20) as result " `varlist'"
    di as text "{c +}"_dup(64) "-"
    eret li
end


d1tab diabetes sex 
di " There are `e(diabetes_1pn)' comprising `e(diabetes_1fn)' individuals with `e(sex_2n)' females with poor health."

The return

macros:
            e(sex_2pf) : "52.5% (5434/10349)"
            e(sex_2fp) : "5434/10349 (52.5%)"
            e(sex_2fn) : "5434/10349"
            e(sex_2np) : "5434 (52.5%)"
            e(sex_2pn) : "52.5% (n = 5434)"
             e(sex_2p) : "52.5%"
             e(sex_2n) : "5434"
            e(sex_1pf) : "47.5% (4915/10349)"
            e(sex_1fp) : "4915/10349 (47.5%)"
            e(sex_1fn) : "4915/10349"
            e(sex_1np) : "4915 (47.5%)"
            e(sex_1pn) : "47.5% (n = 4915)"
             e(sex_1p) : "47.5%"
             e(sex_1n) : "4915"
            e(sex_tot) : "10349"
                e(cmd) : "estpost"
             e(subcmd) : "tabulate"
             e(depvar) : "sex"
         e(properties) : "b"

[Clyde Schechter]

My best guess about what's going wrong is that when you go through the outer loop the second time, when you hit the -estpost- command, it wipes out the current contents of e(), so you lose everything that you -ereturn-ed the first time through.

I'm curious why you are writing a program like this. While it does create a variety of ways to express the output of -tabulate-, if your goal ultimately is to export all of this to a Word document, do you really need this? What need will this satisfy that you couldn't get from -dtable- followed by -collect export-?

[Girish Venkataraman]

My best guess about what's going wrong is that when you go through the outer loop the second time, when you hit the -estpost- command, it wipes out the current contents of e(), so you lose everything that you -ereturn-ed the first time through.

You are right, Clyde Schechter. I had a backup copy of this with the program defined as an rclass program (which used to work well and still does) and the rclass version has no problems incrementally adding more variables with each loop into the return. However, the rclass program would not execute the last return list statement when coded into the program. Which I really wanted, since I want all the return li items to show up in the results window the moment I run the program in the results/analysis do file with having to type return list to figure out what the macro is called (to copy/paste into the putdocx statement). eret list in contrast, seemed to work well with e-returning the macro list immediately in the results window.

I'm curious why you are writing a program like this. While it does create a variety of ways to express the output of -tabulate-, if your goal ultimately is to export all of this to a Word document, do you really need this? What need will this satisfy that you couldn't get from -dtable- followed by -collect export-?

The thought did cross my mind to keep it simple. Perhaps I will investigate the dtable option, although I will admit I find the collect system quite unintuitive despite having read through the entire help for it.

As to why I am taking all this trouble for seemingly simple work, my use case involves data collection, analysis and writing all by myself. As a result, I like writing the analysis/results section in the manuscript entirely inline in a do file where I want the least amount of 'code clutter' between each putdocx statement so I can focus on the text portions better. I tried Ben Jann''s excellent estpost and other user-written programs besides regular Stata commands. However, the code clutter was still too much for me. Which is why I am going through this extensive trouble of making programs that would let me get what I want with 1-2 lines of code for all standard analyses. Doing this via include files got the code clutter down significantly after which I decided to wrap most of them into programs for easier use.

Here is a small sample that uses a several wrappers (for medians, death rates, median OS, cox HR and others) I wrote that keeps code to the minimum. My previous version took over 100 lines of code.

**# Results 1. Cohort summary

$head2
putdocx text ("Baseline outcome data")
*-------------------------------------
$para11
    dmed tos, unit(months)
putdocx text ("The median duration of follow up from diagnosis of TP53mutt myeloid neoplasm to study exit (censoring or death) was `r(tos_mr)'")

    d1tab dead 
putdocx text (" with `r(dead_1n)' deaths in `r(dead_tot)' patients")

qui count
    local n = `r(N)'

*OS death rate
*--------------
    mrate
putdocx text (" and a `r(drate)'") 

// Median & OS for entire cohort 24 months
*-----------------------------------------
    medos, time(24) unit(months) textos( )
putdocx text (" `r(oscall)'")


**# 2. Univariable OS24 : Demo/clinical
*=============================================
$head2
putdocx text ("Uni-variable analyses of OS24")
*=============================================

coxer age70
$para11
putdocx text ("Age at diagnosis > 70 years (`r(callhr)'),")

What the code gives:

 Baseline outcome data The median duration of follow up from diagnosis of TP53mutt myeloid neoplasm to study exit (censoring or death) was 7.1 months (range: 0.2–78.4 months) with 178 deaths in 204 patients and a death rate of 83.5 per 1000 patient-years with a 24-months survival of 17.0% (95% CI = [11.9%–22.8%]).  Uni-variable analyses of OS24 Age at diagnosis > 70 years (HR = 1.6 [1.1-2.1]; P = 0.005), antecedent myeloid neoplasm (HR = 2.2 [1.5-3.2]; P < 0.001) (See Fig. 3), and complex karyotype (HR = 1.5 [1.0-2.4]; P = 0.06) all predicted adverse outcome.

P.S.: Excuse my use of global macros to set styles for headings and paragraphs.

[Clyde Schechter]

Thanks for the explanation. I will just say that while I agree that -collect- as a whole is mind-bogglingly complicated, and I have absorbed only a small fraction of it, -collect export- is simplicity itself. Take a look at it. I think you'll find it both easy to use and useful.

[Girish Venkataraman]

Now that you mention it, I will certainly revisit the -collect- system again. The last time I read through it was 8-9 months ago at which time my knowledge base of Stata functions and programs was still quite rudimentary.