Iteratively intersecting global and local lists

[Raahil Madhok]

In my analysis, I am using district-month panel data. I want to do the following many times: (1) select a bootstrap subsample from the data; (2) run a constrained regression that forces some coefficients to zero; (3) save the non-zero variables in a list. After all the iterations, I want a master list that contains the intersection of the selected variables over all the runs. The way I am currently envisioning the task is to: initiate an empty global, run the above steps in a for loop, in the loop compare the selected variables with the global and update it to hold the intersection.

I've tested it on the auto dataset with 3 iterations but the global list does not update. It seems the issue is when intersecting selected_vars with master_list (the part with the & operator). Is there a cleaner way to approach this? I tried making master_list a local but that produces an empty list as well. Thanks

sysuse auto.dta, clear
    
    
gl master_list ""
forvalues i = 1/3 {
    
    ** Subsample
    set seed 2468
    bsample
        
    ** Lasso
    set seed 1357 // For cross-validation
    lassoregress price mpg rep78 headroom trunk weight length turn displacement gear_ratio foreign
        
    ** Save selected variables
    local selected_vars "`e(varlist_nonzero)'"
        
    ** Update Master with intersection
    gl master_list: list master_list & selected_vars
        
    local dropped: list master_list - selected_vars
    local added: list selected_vars - master_list
        
    di "Added variables: " " `added' "
    di "Dropped variables: " " `dropped' "
    di "Support: " " $master_list "
 }

di " $master_list "

[Clyde Schechter]

There are a few problems with this code.

You cannot mix and match global and local macros in the macro list command.

In any case, there is no reason for you to use a global macro to hold the master list, and, global macros being inherently an unsafe programming practice, you should use a local macro for it in any case.

I will also point out that by resetting the side inside the loop, you will produce the same results each time through the loop. You should just -set seed- once, before you enter the loop, and then leave it alone.

Finally, you initialize master_list to be empty, and then you never actually add anything to it, so, of course, it stays empty. So you need to initialize it to something, or put some things in it inside the loop. Since I'm not entirely sure what you're trying to get here, I won't attempt to give you specifics here.

Correction: It is not true that you can't mix local and global macros with the macrolist commands. You can. But what is true is that everything after the colon is assumed to be a local macro by default. If you want to refer to a global macro after the colon you have to code that as -global(macroname)-. Also you must give only the name of the macro (whether it is local or global)-- no $ or `' dereferencing.