Loop does not work

[Federico Nutarelli]

Hi Statalist,

I have the following problem. I am running the nested loop below in which basically I have 3 nested loops: the principal for dummy_1 dummy_2 dummy_3 dummy_4; the second for aggregation: product, firm and atc2 and the third one for the max of the variable ritiri. The code looks like below:

local recall "dummy_1 dummy_2 dummy_3 dummy_4" // dummy_1 = withdrawns, dummy_2 = recalls, dummy_3 = recalled_class_one, dummy_4 = recalled_class_two

foreach rec of local recall {.   // PRINCIPAL LOOP

    local aggr "product firm atc2"
    tokenize `aggr'
    local regressioni "reg_`1'_`rec' reg_`2'_`rec' reg_`3'_`rec'"
    foreach agg of local aggr {.  // SECOND LOOP
        use "/Users/federiconutarelli/Desktop/PhD/Lavoro_Riccaboni/DATi_USA/Pharma_project_2019 /abnormal_values/DB_`agg'_`rec'_aggregation_nomiss.dta", clear
        estimates restore reg_`agg'_`rec'  // serve per attivare le regressioni
        predict av_t, res
        replace av_t = round(av_t, .111)
        predict stdres, stdr
        predict stdyhat, stdp
        predict stdyj, stdf
        *gen id_unico = _n
        gen var_avt = stdyj - stdyhat 
        replace var_avt = stdyhat
        
        
        save "/Users/federiconutarelli/Desktop/PhD/Lavoro_Riccaboni/DATi_USA/Pharma_project_2019 /abnormal_values/demeaned_DBs/demeaned_`agg'_`rec'.dta", replace
        merge 1:1 _n using "/Users/federiconutarelli/Desktop/PhD/Lavoro_Riccaboni/DATi_USA/Pharma_project_2019 /abnormal_values/DB_`agg'_`rec'_aggregation_nomiss.dta", keepusing(*)
        save "/Users/federiconutarelli/Desktop/PhD/Lavoro_Riccaboni/DATi_USA/Pharma_project_2019 /abnormal_values/demeaned_DBs/demeaned_`agg'_`rec'.dta", replace

        su ritiri, meanonly   
        quietly forval j = 1/`r(max)'{   // THIRD LOOP --> THIS DOES NOT WORK ON ALL DATABASES
            gen av_t`j' = av_t
        }
        
        local avt
        local varavt
        local rescal
        
        su ritiri, meanonly
        quietly forval k = 1/`r(max)' { 
            local avt `avt' av_t`k' 
            local varavt `varavt' var_avt`k'
            local rescal `rescal' rescaled`k'
        }
        
        stack `avt' `varavt' `rescal', into(mean_avt var_avt rescaled) clear    
        
        collapse (mean) mean_avt var_avt, by(rescaled) 
        gen lower = mean_avt - 1.96*(sqrt(var_avt)/11)
        gen higher = mean_avt + 1.96*(sqrt(var_avt)/11)
        drop if rescaled ==.

        *Grafico:
        twoway (connected lower higher mean_avt rescaled,lpattern(solid)), yline(0,lcolor(black)) xline(0,lwidth(thin) lpattern(shortdash)) note("Recall Year = 0") ytitle(abnormal value) xtitle(Year) xlabel(-8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10 11) name(pre_post_gr_sales_`agg'_`rec', replace)
    } 
}

So the problem is that the third loop works only for the last database (specifically DB_atc2_dummy_4_aggregation) but not for the other ones. This seems to me like a classical loop problem but I cannot see what is going wrong? I would like the variables av_t`j' to be created for all the databases (from the first which according to the loop is DB_product_dummy_1 to the last, DB_atc2_dummy_4_aggregation) and not just for the last one.


Can you please help me figuring out what is going wrong?

Thank you,


Federico

[William Lisowski]

Why do you conclude that the variables from the third loop are not being created for the earlier datasets?

After creating these variables you do not again save the dataset which was in memory when the loop ran.

So at the start of the next iteration of your second loop, where you read in the next dataset, the previous dataset with the newly created variables will be cleared from memory without having first been saved to disk.

[Federico Nutarelli]

Hi William,

thank you. So I just need to move the "third part above the save"?

[William Lisowski]

I suppose so, but I am totally mystified by most of your code, so I cannot know whether you "just need to" do what you say, or there are other problems.

In general, the plan on Statalist is to try the advice you are given and either confirm that it solved your problem, or report that it did not, with a fuller explanation of what you expected and what actually resulted. My question above

Why do you conclude that the variables from the third loop are not being created for the earlier datasets?

is the sort of information you should be giving. In particular, I'm not sure why you think the variables were created in the final dataset. In no case did you save the data in memory after creating the av_t`j' variables.