Postfile only posting to the first row of new dataset when used in a loop

[Chris Larkin]

Hello Statalisters,

I'm having a small issue with postfile that i'm unable to work around. I'm using postfile to store some values from a matrix of regression results at the end of a simulation, but it's only printing to the first row in the new dataset. As such I should end up with hundreds of rows, but i'm only ending up with one. Does anyone have any thoughts on why this is?

I've highlighted in red below the bit that's not working exactly as i'd like.

**Setting up a log to veiw output
capture log close _all
log using "stratification_sims$S_DATE.txt", append text name("First Attempt")


local sample 100 300 500 700 900 1100 1300 1500 1700 1900 2100 2300 2500 2700 2900 //set up local for different sample sizes
local strata = ""

forval iter =1/2{ //number of times the simulation is run
    foreach size of local sample{
        preserve
            qui set obs `size' //set different sample sizes
            qui gen id = _n //generate a unique id
            local cats "`"2"' `"2"' `"2"' `"3"' `"4"' `"5"'" //set up local for category values of stratification var
            local num : list sizeof local(cats)
            quietly forval i = 1/`num'{
                local catind :word `i' of `cats'
                gen obs`i' = mod(_n,`catind') //generate i number of stratification vars based on values of local cats
                if `i'<=2                  local fcn mod(_n,2) //generating two binary vars
                else if `i' == 3           local fcn mod(_n,4) //generating a categorical var    
                else if `i' == 4         local fcn rbeta(6, 1) //generating a negatively-skewed var
                else if `i' == 5        local fcn rbeta(1, 4) //generating a positively-skewed var
                else if `i' == 6           local fcn rnormal() //generating a normally-distributed var
                qui gen unobs`i' = `fcn'
            }
            
                qui ds obs*, skip(1)
                local stratlist "`r(varlist)'" //store all stratification vars in a local
                local strata
                forval stratnum = 1/`num'{
                    local strat: word `stratnum' of `stratlist'
                    local strata `"`strata' `strat'"' //looping through stratification vars and adding a new one on each iteration
                    di in red "the below randomisation is stratified on:" `"`strata'"'
                    
                    qui egen strata=group(`strata') //gen a variable that makes unique groups based on strat vars
                    set seed 31540 //setting a seed for replicability
                    qui gen randomnum = runiform() //generating a random number
                    qui bysort strata: egen order=rank(randomnum) //generating a rank order var based on the random number
                    qui bysort strata: gen treat = (order <= _N/2) //assigning condition based on rank
                    
                    qui ds unobs*, skip(1)
                    local unobs "`r(varlist)'"
                    local variables `strata' `unobs' //putting observables and unobservables in one local
                    foreach var of local variables{ //balance test
                        qui: reg `var' treat, r //regressing covariate on treatment  and testing for imbalance
                        matrix A = r(table) //convert regression results to matrix
                        mat li A
                        di `"`var'"' //this is the value I want in the below data
                        postfile buffer pvaltre pvalcon rhs using "sim_results.dta", replace
                        post buffer (A[4,1]) (A[4,2]) (`=`"`var'"'') //putting p-values and IV in buffer
                    postclose buffer
                    
                    }
                        des, s
                        qui drop strata randomnum order treat
                }
        restore
    }
}

Also, you will see from the above code that i'm trying to make three variables in the new dataset (pvaltre, pvalcon, and rhs). I've tried making the rhs variable string equal to the value of `var' on that iteration of the - foreach var of local variables - loop. It is, however, returning as a number in the new dta file - see the data below.

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(pvaltre pvalcon rhs)
.9405507 .4036193 1.2672622
end

[Clyde Schechter]

And if you look carefully, you'll see that the one observation you're getting in the postfile is the one for the last iteration.

This is because you have the -postfile- command inside the most deeply nested of your loops. So each time through, Stata dutifully replaces the file sim_results.dta, just as you tell it to, trashing any data that might have been put there.

Move your -postfile- command to a position before the start of the outermost loop. (Leave the -post- command right where it is.)

By the way, perhaps you have it but just haven't shown it yet, but you also need a -postclose- command after you exit the outermost loop to flush any observations that are still bufferred into the file itself. Then you can -use sim_results- for anything you want to do with it.

[Chris Larkin]

Thank you - that's done the trick.

I had the postfile command at the top previously but I had the post close command up inside the most deeply nested loop so it returned an error. I've now moved it down to just after - restore - before the final curly brace.

It might be a bit off topic, but do you have any thoughts on why the value of `var' is not being posted to the variable rhs, and instead i it is filled with numbers? Happy to create a new post if you think that's more appropriate

[William Lisowski]

If I understand correctly, you want the postfile variable rhs to contain a string with the variable name from the foreach var loop.

First, help postfile suggests that you can (and should!) specify that rhs is a string variable in the postfile command.

Then, you will want to change your post command to pass in a string value.

post buffer (A[4,1]) (A[4,2]) ("`var'")

Added in edit: if you followed Clyde's advice and put the postfile command before the start of the outermost loop (forval iter) then you will want the postclose command after the end of the outermost loop - after the final curly brace.

[Chris Larkin]

Thank you William. I should stop coding so late at night. I had previously specified str5 when I was posting the variables, not at the beginning in the postfile command. It's working great now!