Summary stats table - r(130) expression too long

[Coen vd Kraats]

Hi all,

I am trying to compile a table with summary statistics using the template (code below) by Johannes Schmieder (see: https://sites.google.com/site/johannesschmieder/stata).

I have 2 categorical variables (marital status and educational attainment) for which I want to display the percentage of individuals in each category in the table. Therefore I created binary variables for each of the categories (variable names: educ_no educ_primary educ_sec_low educ_sec_up educ_post_sec educ_tert_first educ_tert_sec mar_married mar_partner mar_separ mar_divor mar_widow mar_never).

However, when I include more than 9 binary variables I get r(130) - "expression too long". The problem seems to be in the line where I define the local "binary": it can only have up to 9 arguments, as soon as I include 1 more I get the error. Is there a way in which I can define more than 9 variables as binary? I need this because I don't want the table to display the standard deviation for binary variables. Or should I look for another solution?

Thanks!
Coen

*Program for summary table
capture program drop sumtable_share
program define sumtable_share
    syntax , [file(str)]
    local size "\footnotesize"
    
    capture confirm file $out_files/`file'
    if _rc == 0{
        rm $out_files/`file'
    }
    
    use $working_data/SHAREsummary_test.dta, clear

    
    local binary "educ_no","educ_primary","educ_sec_low","educ_sec_up", ///
    "educ_post_sec","educ_tert_first","educ_tert_sec", "d_early_ret", "d_normal_ret", ///
    "mar_married", "mar_partner", "mar_separ", "mar_divor", "mar_widow", "mar_never"
    local outcomes reurod
    local controls ragey hittot radla riadlza  d_early_ret d_normal_ret ///
    mar_married mar_partner mar_separ mar_divor mar_widow mar_never
    local education educ_no educ_primary educ_sec_low ///
    educ_sec_up educ_post_sec educ_tert_first educ_tert_sec
    
    g full  = 1
    g smpl1 = full & ragender==1 & ragey>=50 & ragey<=70 
    g smpl2 = full & ragender==1 & ragey>=50 & ragey<=70 & country_area==1
    g smpl3 = full & ragender==1 & ragey>=50 & ragey<=70 & country_area==2
    g smpl4 = full & ragender==1 & ragey>=50 & ragey<=70 & country_area==3
    g smpl5 = full & ragender==2 & ragey>=50 & ragey<=70 
    g smpl6 = full & ragender==2 & ragey>=50 & ragey<=70 & country_area==1
    g smpl7 = full & ragender==2 & ragey>=50 & ragey<=70 & country_area==2
    g smpl8 = full & ragender==2 & ragey>=50 & ragey<=70 & country_area==3
    
    local conditions "smpl1" "smpl2" "smpl3" "smpl4" "smpl5" "smpl6" "smpl7" "smpl8"
    
*    local colhead0 &   (1)    &   (2)    &   (3)       &   (4) 
    local colhead1 & All & North & West & South & All & North & West & South
    local colhead2 & Men    & Men        & Men  & Men & Women & Women & Women & Women

    local cols : word count "`conditions'"
    local cols = `cols'+1

    writeln $out_files/`file' "\begin{table}[htbp]\centering "
    writeln $out_files/`file' "\caption{Summary Table of Share Data \label{tab:sumShare}}"
    writeln $out_files/`file' "\def\sym#1{\ifmmode^{#1}\else\(^{#1}\)\fi}"
    writeln $out_files/`file' "\begin{threeparttable} "
    writeln $out_files/`file' "\begin{tabular}{l *{`=`cols'-1'}{c}} "        
    writeln $out_files/`file' "\toprule  "
*    writeln $out_files/`file' " `colhead0'    \\"    
    writeln $out_files/`file' " `colhead1'    \\"    
    writeln $out_files/`file' " `colhead2'    \\"    
    local paneltitles "Outcome Demographics Education"
    local panels outcomes controls education
    foreach panel in `panels'  {
        local ppos: list posof "`panel'" in local(panels)
        local paneltit: word `ppos' of `paneltitles'
        writeln $out_files/`file' "\midrule"        
        writeln $out_files/`file' "\multicolumn{`cols'}{l}{\textbf{`size'{`paneltit'}}} \\ "
        foreach v in ``panel'' {
            local vl : variable label `v'
            local linemean `vl' 
            local linesd
            foreach c in "`conditions'" {
                if "`c'"=="full" & (("`panel'"=="dwscontrols") | ("`=substr("`v'",1,2)'"=="lj")) {
                    local linemean  `linemean' & 
                    local linesd    `linesd'   & 
                    continue
                }
                qui sum `v' if `c'
                sigdigits a2 `=r(mean)'
                local mean: disp `fmt' `=r(mean)'
                local linemean  `linemean' & `mean'
                sigdigits a2 `=r(sd)'
                local sd: disp `fmt' `=r(sd)'
                local linesd    `linesd'   &  `=cond(`sd'!=.,"[`=ltrim("`sd'")']","`sd'")' //  ,`=r(N)'
            }
            writeln $out_files/`file' "`linemean' \\"
            if !inlist("`v'","`binary'") writeln $out_files/`file' "`linesd'   \\"
        }    
    }
    writeln $out_files/`file' "\midrule  "    
    local j 1
    local countline 
    foreach c in "`conditions'" {
        count if `c'
        local count`j' = r(N)
        local countline `countline' & `count`j++''
    }
    writeln $out_files/`file' "Number of observations `countline' \\ "    

    writeln $out_files/`file' "\bottomrule "
    writeln $out_files/`file' "\end{tabular}"
    writeln $out_files/`file' "\textit{Source}: SHARE waves 1, 2, 4, 5 and 6"
    writeln $out_files/`file' "\end{threeparttable} "
    writeln $out_files/`file' "\end{table}"

end // program sumShare

[Clyde Schechter]

This is a lot of code to read. I suggest you -set tracedepth 1- and -set trace on- and run it again. This way when Stata hits the expression that is too long you will be able to see what the expression is and where in the code the problem arose. Once you see that there is a good chance that the source of the problem will be obvious. If not, post back with that output.

[William Lisowski]

In looking over the code, I noticed a problem I recently described in a different topic. Close readers may recognize material plagarized from my post there.

Here is code that reproduces your problem.

local binary "educ_no","educ_primary","educ_sec_low","educ_sec_up", ///
"educ_post_sec","educ_tert_first","educ_tert_sec", "d_early_ret", "d_normal_ret", ///
"mar_married", "mar_partner", "mar_separ", "mar_divor", "mar_widow", "mar_never"
macro list
display !inlist("gnxl","`binary'")

when when run from the Do-file editor results in

. local binary "educ_no","educ_primary","educ_sec_low","educ_sec_up", ///
> "educ_post_sec","educ_tert_first","educ_tert_sec", "d_early_ret", "d_normal_ret", ///
> "mar_married", "mar_partner", "mar_separ", "mar_divor", "mar_widow", "mar_never"

. macro list
     [stuff omitted]
_binary:        educ_no","educ_primary","educ_sec_low","educ_sec_up",
                "educ_post_sec","educ_tert_first","educ_tert_sec", "d_early_ret",
                "d_normal_ret", "mar_married", "mar_partner", "mar_separ", "mar_divor",
                "mar_widow", "mar_never

. display !inlist("gnxl","`binary'")
expression too long
r(130);

Note that macro list tells us that your local macro binary lacks the first and final quotation marks, and this is at the root of the problem.

This is sort of a known but subtle trap that I find difficult to retain the logic for clearly in my mind, much less explain coherently. All I can say is that the syntax for the local command (look at help local) accepts exactly one string, enclosed at least in double quotes, or optionally in compound double quotes, which are required in this case, since the string itself contains double quotes. Please don't ask me to explain that more coherently, it hurts my head, since I know that generally you can omit the quotation marks with no ill effect, even though the syntax suggests they are required.

So here's a revised version, which, since the line continuation syntax "///" is not allowed in the middle of a quoted string, uses the #delimit command to work around that.

#delimit ;
local binary `" "educ_no","educ_primary","educ_sec_low","educ_sec_up",
"educ_post_sec","educ_tert_first","educ_tert_sec", "d_early_ret", "d_normal_ret",
"mar_married", "mar_partner", "mar_separ", "mar_divor", "mar_widow", "mar_never" "';
#delimit cr
macro list
display !inlist("gnxl","`binary'")

which when run results in

. #delimit ;
delimiter now ;
. local binary `" "educ_no","educ_primary","educ_sec_low","educ_sec_up",
> "educ_post_sec","educ_tert_first","educ_tert_sec", "d_early_ret", "d_normal_ret",
> "mar_married", "mar_partner", "mar_separ", "mar_divor", "mar_widow", "mar_never" "';

. #delimit cr
delimiter now cr
. macro list
     [stuff omitted]
_binary:         "educ_no","educ_primary","educ_sec_low","educ_sec_up",
                "educ_post_sec","educ_tert_first","educ_tert_sec", "d_early_ret",
                "d_normal_ret", "mar_married", "mar_partner", "mar_separ", "mar_divor",
                "mar_widow", "mar_never"

. display !inlist("gnxl","`binary'")
1

[Coen vd Kraats]

Thanks a lot for your comments Clyde and William, very helpful!

I managed to get the code running with your revised code, William. However, something went wrong in writing the output to the latex table as the new code no longer suppresses the standard deviations for binary variables. I found the main problem to be in the line that gives the command for writing the standard deviations to the latex table:

if !inlist("`v'","`binary'") writeln $out_files/`file' "`linesd'   \\"

With the new code the !inlist condition is no longer interpreted correctly and standard deviations are displayed for all variables. I figured that perhaps I should change the line that writes the standard deviations to:

if !inlist("`v'",`binary') writeln $out_files/`file' "`linesd'   \\"

However, when I do this I get the r(130) again in the same location. I guess the reason is that inlist can only have up to 10 string arguments? I figured out I could solve this by creating two locals that label binary variables, e.g.:

    local binary "educ_no","educ_primary","educ_sec_low","educ_sec_up", ///
    "educ_post_sec","educ_tert_first","educ_tert_sec", "d_early_ret", "d_normal_ret"
    local binarytwo "mar_married","mar_partner","mar_separ","mar_divor","mar_widow","mar_never"

Then I just add an extra inlist condition to the line that prints standard deviations:

if !inlist("`v'","`binary'") & !inlist("`v'","`binarytwo'") writeln $out_files/`file' "`linesd'   \\"

This works fine but perhaps there is a more elegant solution that doesn't require me to include two definitions for binary variables and two inlist conditions?

[William Lisowski]

Yes, I see the problem now. The failure of the local command to include the opening and closing quotation marks was offset by the quotation marks around `binary' included in your original command. The syntax without the quotation marks then resurrects the original error message.

The problem is using inlist. There is macro syntax that provides the same functionality as inlist, without the limitations, quotation marks, and commas. Here is some sample code.

#delimit ;
local binary educ_no educ_primary educ_sec_low educ_sec_up
educ_post_sec educ_tert_first educ_tert_sec  d_early_ret
d_normal_ret  mar_married  mar_partner  mar_separ  mar_divor
mar_widow  mar_never;
#delimit cr
foreach v in mar_widow gnxl {
local match : list v in binary
if `match'==0 {
    display "`v' is not in the list"
}
if `match'==1 {
    display "`v' is in the list"
}
}

which when run produces

mar_widow is in the list
gnxl is not in the list

This is documented in the help files. I can never remember the direct name, so I start with help macro then click on "extended_fcn" where it appears in the first few lines, and on the new page, scroll down to "Macro extended functions for manipulating lists" and click on "macrolist_directive". The direct route is help macrolists.

[Coen vd Kraats]

Great, thank again William, very helpful!