Save mi estimate results as dataset

[paulvonhippel]

I'd like to run mi estimate over different subsets of data, and save the resulting estimates to another Stata dataset for further processing. I tried this:

mi estimate, saving("Data\Chile_1000_CIs.dta",replace): mean math_incomplete, over(school_num)

but the saved output is in .ster format, which I hadn't heard of before. I'd like to put the output in .dta format. Any suggestions?

[Bjarte Aagnes]

sysuse auto, clear 
reg length weight
est save res, replace
clear all

est use res.ster
ereturn list
_coef_table 
matrix res = r(table)'
svmat  res , names(col)

[Rich Goldstein]

it is not clear what you want to do with the saved data; if you want to, e.g., compute predicted values, then the .ster is what you want; see

help mi predict

if you want to do something else, please clarify

[paulvonhippel]

I don't want predicted values. I want to save the upper and lower bounds of the confidence interval for each school.

[paulvonhippel]

Here's a solution using a postfile, but I think it's a little clunky. It's 8 lines instead of 1, and it's slow because it repeatedly subsets the data with if. Also, it hard-codes the fact that the confidence limits are in the 5th and 6th row of rtable(). This is bad programming style since it's not transparent and the contents of rtable() could change in future versions.

Improvements welcome!

postfile CIs ll ul using "Data\Chile_1000_CIs.dta", replace
forvalues s = 1/1000 {
 mi estimate: mean math_incomplete if school_num==`s'
 local ll = r(table)[5,1]
 local ul = r(table)[6,1]
 post CIs (`ll') (`ul')
}
postclose CIs

[Bjarte Aagnes]

Ref #1 did you try reading the .ster file using estimates use ?

[Bjarte Aagnes]

I leave an example:

cls // make example .ster file
sysuse auto , clear
mean weight if foreign
estimates save estimates , replace
mean weight if !foreign
estimates save estimates , append
clear all

********************************************************************************

local groups 2

forvalues i = 1/`groups' {
    
    estimates use estimates , number(`i')
    
    * do something
    
    estimates describe
    local cmd  = r(cmdline)
    
    _coef_table
    matrix res = r(table)'
    
    clear
    svmat  res , names(col)
    gen cmd = "`cmd'" , before(b)
    tempfile res
    save "`res'"
        
    local results = "`results'" + " " + "`res'"
}

clear
append using `results'
list

. list

     +-------------------------------------------------------------------------------------------------------------------+
     |                     cmd          b         se          t     pvalue         ll         ul   df       crit   eform |
     |-------------------------------------------------------------------------------------------------------------------|
  1. |  mean weight if foreign   2315.909   92.31665   25.08658   3.87e-17   2123.926   2507.892   21   2.079614       0 |
  2. | mean weight if !foreign   3317.115    96.4296   34.39935   5.84e-37   3123.525   3510.706   51   2.007584       0 |
     +-------------------------------------------------------------------------------------------------------------------+

[Bjarte Aagnes]

#5

Also, it hard-codes the fact that the confidence limits are in the 5th and 6th row of rtable()

local ll = r(table)[5,1]
local ul = r(table)[6,1]
post CIs (`ll') (`ul')

can be changed to

post CIs ( r(table)["ll", 1] ) ( r(table)["ul", 1] )

---

But, reading the ster file, possibly combined with -frame post- should be faster.

[daniel klein]

While I like Bjarte's idea of reading the ster-file, I would like to add a word of caution (as almost every time you are trying to something with MI data).

mi estimate stores/saves multiple results into one ster-file. The combined results are stored as the M+1 set of results, where M is the number of imputations. Thus, Paul would need to

mi estimate , saving("Data\Chile_1000.ster",replace) : ...
estimates describe using "Data\Chile_1000.ster"
local M1 = r(nestresults)
estimates use "Data\Chile_1000.ster" , number(`M1')

to get the desired results into memory. Pauls should not use _coef_table, either. Instead, he should follow up with

mi estimate

which will replay the latest (loaded) results. This will leave behind the correct r(table) results.

Pauls should not be too worried about r(table) changing the layout in the future. Many commands in official Stata probably rely on r(table) right now. Changing this would probably mean a lot of work for StataCorp. Moreover, given recent plans to document r(table) (but see the cautery tales that follow this announcement), changes would probably be undone by version control.

[Bjarte Aagnes]

With the added neccesarry details of the mi results, a MWE might be:

webuse mheart1s20 , clear

tempfile ster
mi estimate , saving("`ster'", replace) : mean bmi, over(female)
clear all

tempname res
estimates describe using "`ster'"
estimates use "`ster'", number(`r(nestresults)')
estimates // to populate r(table) with correct content
matrix `res' = (r(table)["ll",.] \ r(table)["ul",.])'
svmat `res' , names(col)
list

. estimates // to populate r(table) with correct content
-------------------------------------------------------------------------------------------------------------------------------------------------
active results
-------------------------------------------------------------------------------------------------------------------------------------------------

Multiple-imputation estimates     Imputations     =         20
Mean estimation                   Number of obs   =        154
                                  Average RVI     =     0.1602
                                  Largest FMI     =     0.1578
                                  Complete DF     =        153
DF adjustment:   Small sample     DF:     min     =     110.32
                                          avg     =     115.22
Within VCE type:     Analytic             max     =     120.13

--------------------------------------------------------------
             |       Mean   Std. Err.     [95% Conf. Interval]
-------------+------------------------------------------------
c.bmi@female |
          0  |   25.33867   .4093295       24.5275    26.14983
          1  |   24.95759   .7383236      23.49578     26.4194
--------------------------------------------------------------


. list

     +---------------------+
     |       ll         ul |
     |---------------------|
  1. |  24.5275   26.14984 |
  2. | 23.49578    26.4194 |
     +---------------------+

[daniel klein]

What is wrong with the last digit of the first upper limit? There is a difference between the results table and the listed data. Is this a display issue or would you need to

svmat double ...

[Bjarte Aagnes]

Thanks, svmat does not respect

set type double

The corrected MWE is:

webuse mheart1s20 , clear

tempfile ster
mi estimate , saving("`ster'", replace) : mean bmi, over(female)
clear all

tempname res
estimates describe using "`ster'"
estimates use "`ster'", number(`r(nestresults)')
estimates // to populate r(table) with correct content
matrix `res' = (r(table)["ll",.] \ r(table)["ul",.])'
svmat double `res' , names(col)
list

. list

     +-----------------------+
     |        ll          ul |
     |-----------------------|
  1. | 24.527497   26.149835 |
  2. | 23.495776   26.419403 |
     +-----------------------+