Can I automate regression margins for a loop?

[Dominik Fitze]

Hi guys,

So I'm basically making regression models for a loop of all variables and one fix variable, such as

foreach var of varlist _all { regress DependentVar IndependentVar1`var' }

I then manually look for R values above 0.9 and calculate margins for those, such as:

regress DependentVar IndependentVar1 IndependentVar2 //R=97

margins, at(IndependentVar1=(Value in Year X) IndependentVar2=(Value in Year X))

I was wondering whether I could automatize this such that it automatically enters the values for IndependentVar1 and IndependentVar2? All cases are identified by a year. Right now, this is done by hand which is quite time consuming given that the dataset has about 70 variables, and we try to do it for about 7 different parties.

Thank you for your help!

The background is that we are trying to find election predictions with fundamentals for a university class.

[Carole J. Wilson]

This contains 2 loops:
1) loop over variables you are using as your 2nd independent variable
2) loop over years if the r^2 is > value

I'm using the auto dataset and created a new variable "year" that is the same as foreign. mpg is the "fixed" independent variable here. You'll want to change the r^2 threshold to 0.9.

sysuse auto
clonevar year=foreign
local unfixed rep78 headroom trunk weight    //2nd independent variables
foreach var of varlist `unfixed' {    //loop 1
levelsof year, local(y)
reg price mpg `var'
if `e(r2)'> 0.20 {        //enter if r^2 is >.20 (change to .9)
    foreach Y of local y {    //loop 2
        qui sum mpg if year==`Y', meanonly    
        local yx1= `r(mean)'        //value of mpg for year Y
        qui sum `var' if year==`Y', meanonly 
        local yx2= `r(mean)'        //value of var for year Y
        margins, at(mpg=`yx1' `var'=`yx2')
        }
    }
}

[Dominik Fitze]

Dear Carole

Thank you so much for your answer!

When I run this code (adapted for my variables), I get the error message "option arbeitslos_kant not allowed" for my code

local unfixed arbeitslos_kant arbeitslos_ch inflation  sorgen_asyl    //some of my variables
foreach var of varlist `unfixed' {    //loop 1
levelsof year, `unfixed'
reg parteistaerke_sp kantonsparlament_sp `unfixed' if year<2015
if `e(r2)'> 0.90 {        //enter if r^2 is >.20 (change to .9)
    foreach Y of local y {    //loop 2
        qui sum kantonsparlament_sp if year==`2015', meanonly    
        local yx1= `r(mean)'        //value of mpg for year Y
        qui sum `var' if year==`Y', meanonly 
        local yx2= `r(mean)'        //value of var for year Y
        margins, at(kantonsparlament_sp=`yx1' `var'=`yx2')
        }
    }
}

Would you happen to know what my problem is here? I must admit I'm at a bit of a loss.

[Carole J. Wilson]

You've got some things wrong here:

local unfixed arbeitslos_kant arbeitslos_ch inflation  sorgen_asyl    //some of my variables
foreach var of varlist `unfixed' {    //loop 1
levelsof year, local(y)  //change to- levelsof year if year<2015, local(y) -if you want the margins commands only for values <2015
reg parteistaerke_sp kantonsparlament_sp `var' if year<2015
if `e(r2)'> 0.90 {      
    foreach Y of local y {    //loop 2
        qui sum kantonsparlament_sp if year==`Y', meanonly    
        local yx1= `r(mean)'        //value of mpg for year Y
        qui sum `var' if year==`Y', meanonly
        local yx2= `r(mean)'        //value of var for year Y
        margins, at(kantonsparlament_sp=`yx1' `var'=`yx2')
        }
    }
}

[Dominik Fitze]

thank you so much, I got it to work now!

For the record, I'm trying to calculate the regression for years 1991-2011 and then do the margins with the values of 2015 as a retrognosis to test how accurate the regression models are.

And it works! Thank you so much!

[Dominik Fitze]

If I may ask another question, maybe you know whether it's possible to have it such that both independent variables are set by the loop? could I just define two varlists with the same variables and replace kantonsparlament_sp with the second varlist?

[Carole J. Wilson]

Yes, but that will require a parallel loop. Let me play with it for a second and I'll post some code. You want the following? Do they need to be ordered this way or does it matter?

reg Y A B
reg Y A C
reg Y A D
reg Y A E
...
reg Y B A
reg Y B C
reg Y B D
reg Y B E
...
reg Y C A
reg Y C B
...

[Carole J. Wilson]

if you are only interested in the margins for a single year (2015), that will make the code a little lighter.

[Dominik Fitze]

Dear Carole,

Thank you so much, again! Yes, I am interested in a given Y and then two independent variables like you describe. The order doesn't matter.

Structurally, for the margins, the idea is that we calculate the regression with years 1991-2011 and calculate the margins for 2015 (if R2 is high). Then we calculate whether the margin for a given model is close to the actual value (this can probably not be automated, because it sometimes is a judgment call).

I can share my dataset if you want.

Eventually, I'll run the code for 7 different Ys (major parties) with 6 different datasets (biggest Swiss cantons), but this need not be automated at all, given that we'll want to check which models are best for which Y

[Carole J. Wilson]

Actually, a parallel loop isn't needed here. I think this will give you what you want:

local list1 arbeitslos_kant arbeitslos_ch inflation  sorgen_asyl
local list2 arbeitslos_kant arbeitslos_ch inflation  sorgen_asyl
    
foreach var1 of local list1 {
    foreach var2 of local list2 {
        if "`var1'"!="`var2'" {
            noi di "`var1' `var2'"
            reg parteistaerke_sp `var1' `var2' if inrange(year, 1991, 2011)
            if `e(r2)'> 0.90 {      
                qui sum `var1' if year==2015, meanonly    
                local yx1= `r(mean)'      
                qui sum `var2' if year==2015, meanonly
                local yx2= `r(mean)'
                margins, at(`var1'=`yx1' `var2'=`yx2')
                noi di _dup(80) "-"
                }
            }
        }
    }

[Dominik Fitze]

Dear Carole J. wilson,

Thank you so much, this is awesome and it seems to work perfectly! You saved me and my group at least 15 hours of work and about 1000 lines of code, thank you so much for helping a political science student out!

(It's a shame we were never shown this kind of code in our stats classes)