Looping over multiple outcome variables and multiple control lists

[Scott Rick]

Hi. I am trying to run the flexpaneldid command across multiple outcomes and covariate locals. However my loops below keep failing - it runs for varList1 for the first outcome (confirmed), but then throws up the error "variable recovered not found". I'd greatly appreciate any help in helping me figure this out and not having to run multiple individual commands.

*outcome variables
local outcomes confirmed recovered deceased

* lists of covariates 
local varList1 "pop_perkm2_2019"
local varList2 "pop_perkm2_2019 per60_2011"
local varList3 "pop_perkm2_2019 per60_2011 exphealth_percap1516"
local varList4 "pop_perkm2_2019 per60_2011 hospital_beds_total"


    forvalues j = 1/4 {
        
        flexpaneldid_preprocessing, id(state) treatment(stepone) time(weeknum) ///
    matchvars(tested `varList`j'') matchtimerel(-1) prepdataset(flex_pre_`j') ///
    replace
    
    use "${outdir}\flex.dta", clear
    
    foreach v of local outcomes {
    
    flexpaneldid `v', id(state) treatment(stepone) time(weeknum)  ///
    prepdataset(flex_pre_`j'.dta) statmatching(con(tested `varList`j'')) ///
    outcometimerelstart(1) outcomedev(-1) test

    
        }

    }


Sample data:


input str50 state byte(stepone weeknum) float(confirmed recovered deceased tested pop_perkm2_2019 per60_2011)
"Andaman and Nicobar Islands" 0  3  .0025188916             0             0          0 48.12704  6.7
"Andaman and Nicobar Islands" 0  4   .024469234             0             0          0 48.12704  6.7
"Andaman and Nicobar Islands" 0  5   .026268443    .007196833             0          0 48.12704  6.7
"Andaman and Nicobar Islands" 0  6    .02806765      .0259086             0   .5048579 48.12704  6.7
"Andaman and Nicobar Islands" 0  7    .04713926     .02770781             0   .9640158 48.12704  6.7
"Andaman and Nicobar Islands" 0  8    .08312342     .03418496             0  2.3756747 48.12704  6.7
"Andaman and Nicobar Islands" 0  9    .08312342     .07916517             0          0 48.12704  6.7
"Andaman and Nicobar Islands" 0 10    .08312342     .08312342             0          0 48.12704  6.7
"Andaman and Nicobar Islands" 0 11    .08312342     .08312342             0  12.649874 48.12704  6.7
"Andaman and Nicobar Islands" 0 12    .08312342     .08312342             0   18.72976 48.12704  6.7
"Andaman and Nicobar Islands" 0 13    .08312342     .08312342             0  20.354803 48.12704  6.7
"Andaman and Nicobar Islands" 0 14    .08780137     .08312342             0  24.617487 48.12704  6.7
"Andaman and Nicobar Islands" 0 15    .10759266      .0845628             0   30.02123 48.12704  6.7
"Andaman and Nicobar Islands" 0 16    .13637999     .09859662             0  34.847786 48.12704  6.7
"Andaman and Nicobar Islands" 0 17     .2400144     .11982728             0   39.56459 48.12704  6.7
"Andaman and Nicobar Islands" 0 18     .3573228      .1925153             0    43.8136 48.12704  6.7
"Andaman and Nicobar Islands" 0 19     .4303707      .3004678             0   47.48615 48.12704  6.7
"Andaman and Nicobar Islands" 0 20    .55055773      .3983447             0   52.03814 48.12704  6.7
"Andaman and Nicobar Islands" 0 21     .9913638      .4893847      .0043181   57.70313 48.12704  6.7
"Andaman and Nicobar Islands" 0 22     2.338971      .7607053    .030946383   64.29327 48.12704  6.7
"Andaman and Nicobar Islands" 0 23     4.445124     1.9399065     .05325657   69.38575 48.12704  6.7
"Andaman and Nicobar Islands" 0 24     6.372796      3.602375      .0734077  72.835556 48.12704  6.7
"Andaman and Nicobar Islands" 0 25     7.401943      5.493343     .09427851   78.20691 48.12704  6.7
"Andaman and Nicobar Islands" 0 26     7.967974      6.804605     .11694854   88.05541 48.12704  6.7
"Andaman and Nicobar Islands" 0 27     8.549838      7.644117      .1270241   103.2965 48.12704  6.7
"Andaman and Nicobar Islands" 0 28     9.005398       8.34041     .13026269   118.5135 48.12704  6.7
"Andaman and Nicobar Islands" 0 29     9.310543      8.765743     .13098237  132.69522 48.12704  6.7
"Andaman and Nicobar Islands" 0 30     9.619647     9.0446205     .13314141   146.4822 48.12704  6.7
"Andaman and Nicobar Islands" 0 31     9.869018      9.267722     .13602015  161.50414 48.12704  6.7
"Andaman and Nicobar Islands" 0 32     10.16049      9.537244     .13889888  178.42174 48.12704  6.7
"Andaman and Nicobar Islands" 0 33    10.441525      9.825477     .14213745  194.62253 48.12704  6.7
"Andaman and Nicobar Islands" 0 34    10.762144     10.124505      .1468154  212.95827 48.12704  6.7
"Andaman and Nicobar Islands" 0 35    11.014033     10.445844     .15077366  232.24937 48.12704  6.7
"Andaman and Nicobar Islands" 0 36    11.281036     10.713926      .1518532   254.9417 48.12704  6.7
"Andaman and Nicobar Islands" 0 37    11.517812     10.988845      .1536524  276.54156 48.12704  6.7
"Andaman and Nicobar Islands" 0 38     11.74631      11.24973      .1536524  302.97482 48.12704  6.7
"Andaman and Nicobar Islands" 0 39     11.88341      11.50126      .1536524   330.2638 48.12704  6.7
"Andaman and Nicobar Islands" 0 40    12.028788     11.686218      .1536524   359.4645 48.12704  6.7
"Andaman and Nicobar Islands" 0 41    12.190716     11.811443      .1536524   391.5775 48.12704  6.7
"Andaman and Nicobar Islands" 0 42    12.323137     11.967255      .1554516   425.1036 48.12704  6.7
"Andaman and Nicobar Islands" 0 43    12.426413     12.118388      .1561713   451.5088 48.12704  6.7
"Andaman and Nicobar Islands" 0 44    12.471394      12.22598      .1561713   471.9406 48.12704  6.7
"Andaman and Nicobar Islands" 0 45    12.512415     12.304786      .1561713   495.7629 48.12704  6.7
"Andaman and Nicobar Islands" 0 46    12.564232      12.34113      .1561713   520.6128 48.12704  6.7
"Andaman and Nicobar Islands" 0 47    12.578626      12.38647      .1561713  544.35156 48.12704  6.7
"Andaman and Nicobar Islands" 0 48    3.5940986     3.5465994     .04462036   160.1342 48.12704  6.7
"Andhra Pradesh"              0  1 5.471253e-06             0             0          0 320.4234 10.1
"Andhra Pradesh"              0  2 .00003009189             0             0          0 320.4234 10.1
"Andhra Pradesh"              0  3 .00016413757             0             0          0 320.4234 10.1
"Andhra Pradesh"              0  4  .0014526175 .000024620636 2.7356264e-06 .014375716 320.4234 10.1
"Andhra Pradesh"              0  5   .005900746  .00011763193 .000068390655 .017436882 320.4234 10.1
"Andhra Pradesh"              0  6   .009243681   .0003419533  .00019970073  .23323677 320.4234 10.1
"Andhra Pradesh"              0  7   .014745026    .001917674   .0004240221   .7079199 320.4234 10.1
"Andhra Pradesh"              0  8   .023928523    .005214103   .0005991022  1.5572937 320.4234 10.1
"Andhra Pradesh"              0  9    .03275092    .012017606   .0006784353   2.542428 320.4234 10.1
"Andhra Pradesh"              0 10     .0401207     .02038589   .0008836073   3.673837 320.4234 10.1
"Andhra Pradesh"              0 11    .04784063     .03031621   .0009930324  4.3049293 320.4234 10.1
"Andhra Pradesh"              0 12    .05759861     .03819208   .0010997218   6.190895 320.4234 10.1
"Andhra Pradesh"              0 13    .07340232     .04643726   .0012638593    7.53984 320.4234 10.1
"Andhra Pradesh"              0 14    .09609161     .05391099    .001471767    9.22762 320.4234 10.1
"Andhra Pradesh"              0 15    .13052768     .06761648    .001690617  11.162204 320.4234 10.1
"Andhra Pradesh"              0 16    .18955427     .08865618   .0023198111  13.809762 320.4234 10.1
"Andhra Pradesh"              0 17     .2794497     .12601389    .003528958   17.07841 320.4234 10.1
"Andhra Pradesh"              0 18    .40790105     .19571765    .004853001  20.204346 320.4234 10.1
"Andhra Pradesh"              0 19     .6419585       .333213   .0079004895  22.918196 320.4234 10.1
"Andhra Pradesh"              0 20     1.162458     .55826473    .014559004   26.82829 320.4234 10.1
"Andhra Pradesh"              0 21    2.1597633     1.0149994     .02217772  33.748672 320.4234 10.1
"Andhra Pradesh"              0 22     3.397927     1.8642883    .030907106     41.749 320.4234 10.1
"Andhra Pradesh"              0 23     4.695284      2.959302     .04224628     49.658 320.4234 10.1
"Andhra Pradesh"              0 24     5.883935     4.1729164     .05406418   56.80329 320.4234 10.1
"Andhra Pradesh"              0 25     7.142318      5.322104    .066166595  64.338425 320.4234 10.1
"Andhra Pradesh"              0 26     8.525875      6.513978     .07742643   72.46796 320.4234 10.1
"Andhra Pradesh"              0 27     9.908241      7.944073      .0873376   81.21592 320.4234 10.1
"Andhra Pradesh"              0 28    11.181856      9.336517      .0965594   90.74339 320.4234 10.1
"Andhra Pradesh"              0 29    12.246616     10.736067      .1045064  100.27867 320.4234 10.1
"Andhra Pradesh"              0 30     13.16779     11.890487     .11077372  109.85808 320.4234 10.1
"Andhra Pradesh"              0 31    13.962908     12.869414     .11589754  119.09721 320.4234 10.1
"Andhra Pradesh"              0 32    14.617015      13.67466     .12043048   128.3992 320.4234 10.1
"Andhra Pradesh"              0 33    15.124684      14.34849     .12404697  137.95345 320.4234 10.1
"Andhra Pradesh"              0 34    15.551532     14.886323     .12682912  147.65498 320.4234 10.1
"Andhra Pradesh"              0 35      15.9089     15.348885     .12889725   158.3183 320.4234 10.1
"Andhra Pradesh"              0 36    16.203884     15.671088     .13048391    168.499 320.4234 10.1
"Andhra Pradesh"              0 37    16.397144     15.933694     .13191737  177.63554 320.4234 10.1
"Andhra Pradesh"              0 38    16.541574     16.154377     .13316208  186.30775 320.4234 10.1
"Andhra Pradesh"              0 39     16.63732     16.351946     .13402928  193.78273 320.4234 10.1
"Andhra Pradesh"              0 40     16.71569     16.474571      .1348062   201.4241 320.4234 10.1
"Andhra Pradesh"              0 41    16.782236     16.557583      .1352439   209.5833 320.4234 10.1
"Andhra Pradesh"              0 42    16.839167     16.626638     .13561869  217.58424 320.4234 10.1
"Andhra Pradesh"              0 43    16.882893     16.681576     .13597158   224.4178 320.4234 10.1
"Andhra Pradesh"              0 44    16.920576     16.727335     .13635455   231.0391 320.4234 10.1
"Andhra Pradesh"              0 45    16.951689     16.768778      .1365953  237.42604 320.4234 10.1
"Andhra Pradesh"              0 46    16.971249     16.801928     .13675943  242.32657 320.4234 10.1
"Andhra Pradesh"              0 47    16.989803     16.826271     .13691536  247.83926 320.4234 10.1
"Andhra Pradesh"              0 48     4.857258        4.8111     .03913587   71.76597 320.4234 10.1
"Arunachal Pradesh"           0  4  .0001899696             0             0          0 17.95971  4.6
"Arunachal Pradesh"           0  5  .0006648936             0             0          0 17.95971  4.6
"Arunachal Pradesh"           0  6  .0006648936   .0002849544             0   .0806421 17.95971  4.6
"Arunachal Pradesh"           0  7  .0006648936   .0006648936             0    .218845 17.95971  4.6
"Arunachal Pradesh"           0  8  .0006648936   .0006648936             0   .3649316 17.95971  4.6
"Arunachal Pradesh"           0  9  .0006648936   .0006648936             0    .587576 17.95971  4.6

[Daniel Schaefer]

Hi Scott,

This is a little difficult to debug because these user-written commands don't produce the same error with your example data as with your full dataset. That said, I don't see an obvious syntax issue here. If we just keep the structure of the loop without worrying about the content, we see that the local `v' points to an existing variable as expected.

local outcomes confirmed recovered deceased
forvalues j = 1/4 {
    foreach v of local outcomes {
        confirm variable `v'
    }
}

My guess is that variables which aren't used are dropped from the dataset during the course of the -flexpaneldid- command. You might start by running this code until you get your error, then just check to make sure "recovered" is still part of the dataset. It's definitely possible the variable is dropped somewhere else, or not included in flex.dta for some reason, but I would start with the idea that -flexpaneldid- is the culprit and work from there. You can also add confirm variable v just to make sure you're getting a meaningful error, and the variable is definitely no longer part of the dataset.

forvalues j = 1/4 {    
    flexpaneldid_preprocessing, id(state) treatment(stepone) time(weeknum) ///
    matchvars(tested `varList`j'') matchtimerel(-1) prepdataset(flex_pre_`j') ///
    replace
    
    use "${outdir}\flex.dta", clear
    
    foreach v of local outcomes {
        confirm variable `v'
        flexpaneldid `v', id(state) treatment(stepone) time(weeknum)  ///
        prepdataset(flex_pre_`j'.dta) statmatching(con(tested `varList`j'')) ///
        outcometimerelstart(1) outcomedev(-1) test
    }
}

you can also move that confirm variable `v' line around to figure out exactly where in the runtime of the loop the variable is dropped. If my hypothesis is correct, you should be able to fix this by rereading the preprocessed dataset from the file system on each iteration of the inner loop like so:

forvalues j = 1/4 {    
    flexpaneldid_preprocessing, id(state) treatment(stepone) time(weeknum) ///
    matchvars(tested `varList`j'') matchtimerel(-1) prepdataset(flex_pre_`j') ///
    replace
    
    foreach v of local outcomes {
        use "${outdir}\flex.dta", clear
        
        flexpaneldid `v', id(state) treatment(stepone) time(weeknum)  ///
        prepdataset(flex_pre_`j'.dta) statmatching(con(tested `varList`j'')) ///
        outcometimerelstart(1) outcomedev(-1) test
    }
}

But of course, none of this is tested.

[Daniel Schaefer]

I guess you might want to differentiate the error raised by confirm variable `v' from the one you usually get so you know which line produces the error. Otherwise, you will just get the same error when you hit the confirm variable `v' line. You can do this either with set trace on, or by explicitly defining the message like so:

capture confirm variable `v'
if _rc{
    di "[CONFIRM VARIABLE ERROR] `v' does not exist."
}

[Scott Rick]

Daniel Schaefer: Thanks a lot for this! Your guess is right - as shown below through your code, variables which aren't used are dropped from the dataset during the course of the -flexpaneldid- command. So when it runs for the first outcome variable, confirmed, it drops the other two outcome variables. Is there any way that you see around this, other than running the specifications individually for each outcome?

[CONFIRM VARIABLE ERROR] recovered does not exist.
variable recovered not found
r(111);

[Daniel Schaefer]

I'm not exactly sure what you mean by "running the specifications individually for each outcome." I don't think you have to preprocess the data each and every time you want to run -flexpaneldid-. I think it should be enough to reload the preprocessed data from the file system, as I do in the last version of the code in #2. I don't know of a convenient way to stop -flexpaneldid- from deleting the other outcomes in the first place.