Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • marksample not filtering casewise

    Hello all: I have this wrapper to ereturn all relevant OS and median survival from the ssc program -stpm2- I have a single main variable I am modeling with all adjustment variables in a separate list in `adjust' below. I have a missing value in one of the adjust variables mvaf25. The programs stalls and ask me to exclude with an -if- statement to exclude the missing case, despite the -marksample- line which I thought would pick all main and adjustment variables with complete data. I have included parts of the program which I think are relevant. Sorry this is not a reproducible example. I have a parallel program for Cox HR and CI etc. with the same exact code structure with marksample has no issues there. Perhaps the issue is related to stpm2-generated new variables? The program runs fine with all non-missing variables or if manually delete casewise missing ones.


    Code:
    cap program drop fpm50
    program define fpm50, eclass
        version 18.0
                ereturn clear
                qui count
                ereturn local tot = `r(N)'    
        syntax varlist(numeric max=1) [if]     ///
        [, ADJust(namelist)]                 ///
        [DOF(integer 3)]                     ///
        [TEXT0(string asis)]                 ///
        [TEXT1(string asis)]                 ///
        [TEXT2(string asis)]                 ///
        [unit(string)]
    *-------------------------------------------
    
            marksample touse
            di "`unit'"
        
        // Housekeeping
        *--------------        
            cap drop _rcs* _d_* tt means*            // Drops any variable created by prior stpm2
            qui stdescribe                            // Grab exit time
                local t = floor(`r(tr_max)')         // Makes it an integer
            local vlab: variable label `varlist'    // Need for calls
            local endash = ustrunescape("\u2013")    // Need for calls
    
      .(..spacing codes for return locals)
        
        // Restrict the program to take binary variables
        *-----------------------------------------------
            qui levelsof `varlist' if(`touse'), local(lev)
            scalar nt = `r(r)'
            cap assert nt>2
                if !_rc {
                    disp "There are `=nt' levels"
                    }        
    
    // Run the FPM model
    *--------------------        
    set trace on
            qui stpm2 `varlist' `adjust' if(`touse'), df(`dof') eform scale(h)
            qui mat li r(table)
                tempname a
                mat `a' = r(table)
    set trace off
        //    Calculate Somer's D explained variance
        *------------------------------------------
            qui str2d: `e(cmdline)'                // Repeats the stpm2 and runs the Somer's D
            local somd         = int(r(r2)*100)
            local somlow     = int(r(r2ll)*100)
            local somhigh     = int(r(r2ul)*100)
        
            // Gather HR, 95% and pvalue
            *--------------------------
            local b = string(`a'[1,1], "%09.1fc")
            local l = string(`a'[5,1], "%09.1fc")
            local u = string(`a'[6,1], "%09.1fc")
            local p = `a'[4,1]
            local n = e(N)    
                
            // Fix p-value formats
            *---------------------
                pvformat `p'
                local pvalue = scalar(p)
                local pfpm = "`pvalue'"
                
            range tt 1 `t' `t'
            foreach i of local lev {
            qui predict means`i' if `touse', meansurv ci timevar(tt) at(`varlist' `i')
                     local os`i' = string(round(means`i'[`t'] * 100, 0.1), "%9.1f") + "%"
                     scalar fos`i' = round(means`i'[`t'], 0.01)
            }
            
            //    Median survival frame
        tempname med
    frame create `med'
    cap frame drop __00*
    frame copy default `med'
        frame `med'{
            standsurv, atvars(med0 med1) at1(`varlist' 0) at2(`varlist' 1) centile(50) timevar(tt) ci
    
            * Parametric median survival and CIs
                *---------------------------------------
                 *** code for this
        
        // Create e-display text for the HR
        *---------------------------------
        ereturn clear
             ereturn needed locals...(here)
        
        //      Drop created vars
        *---------------------
        cap drop _rcs* _d_* tt means*
        cls
    end
    Code:
     fpm50 ep7, adjust(age70 mvaf25) unit(mos.) dof(4)
    Error

    Code:
    There are missing values for mvaf25
    You can restrict using an if statement
    r(198);
    Sample ereturned locals are below
    Code:
    ---------------------------------------------------------------------------
                  Flexible Parametric model for OS, HR and Somer's D
                  --------------------------------------------------
    Variable main      :    ep7 (EPI7 signature)
    Adjusted for       :    age70 time0
    Explained Variance :    7%
    ---------------------------------------------------------------------------
    
    scalars:
                   e(fos0) =  .21
                   e(fos1) =  .07
    
    macros:
                    e(os1) : "6.8%"
                    e(os0) : "21.4%"
               e(callos01) : "21.4% vs. 6.8%; Pfpm < .001"
               e(callos10) : "6.8% vs. 21.4%; Pfpm < .001"
                   e(somd) : "7% (95% CI: 2–14%)"
                      e(n) : "204"
                      e(u) : "2.5"
                      e(l) : "1.3"
                      e(b) : "1.8"
                   e(pfpm) : "< .001"
                e(callhrp) : "HR = 1.8 [1.3–2.5]; Pfpm < .001; N = 204"
                 e(callhr) : "HR = 1.8 [1.3–2.5]; Pfpm < .001"
              e(callmed01) : "9.1 mos. vs. 5.4 mos."
              e(callmed10) : "5.4 mos. vs. 9.1 mos."
    ---------------------------------------------------------------------------
    Last edited by Girish Venkataraman; 05 Dec 2023, 16:27. Reason: Added sample ereturned locals in case so folks can see the purpose behind it.

  • #2
    So you want marksample to also look at the variables in the option adjust(). marksample does not do that automatically. In fact, the way you set this up, Stata does not know yet that the things you specified in the adjust() option are variable names. So I would in the syntax command add ADJust(varlist) instead of ADJust(namelist). This will not solve your problem, but it does provide an early check of the user's input. To solve you problem you need to tell Stata to also look at the variables in the adjust option, and you do so by adding the command markout, like so:

    Code:
    marksample touse
    markout `touse' `adjust'
    ---------------------------------
    Maarten L. Buis
    University of Konstanz
    Department of history and sociology
    box 40
    78457 Konstanz
    Germany
    http://www.maartenbuis.nl
    ---------------------------------

    Comment


    • #3
      Beautiful. Glad to know about markout as the missing piece, Maarten Buis. Will see the help and read a bit more while adjusting my code. Up until I saw your reply, I had patched things with a foreach loop to keep if !missing(`adjust'), but I did not like doing that.

      Comment


      • #4
        Maarten Buis,

        I tried your suggestion and the markout seemed to be working fine too, but when it hits the main stpm2 command, it comes back with the missing value error again. I have included the stan2 dataset here for reproducibility. Presuming that you have stpm2 installed. I made observation 103 to be missing to reproduce the error.


        Code:
        webuse stan2, clear
        stset stime, failure(died) exit(time 100)
        replace wait = . in 103
        gen wt = wait>30 if !missing(wait)
        
        
        cap program drop fpm50
        program define fpm50, eclass sortpreserve
            version 18.0
                    ereturn clear
                    qui count
            syntax varlist(numeric max=1) [if]     ///
            [, ADJust(varlist)]                 ///
            [DOF(integer 3)]                     ///
            [MYOPTS * ]                            ///
            [TEXT0(string asis)]                 ///
            [TEXT1(string asis)]                 ///
            [TEXT2(string asis)]                 ///
            [unit(string)]
        *-------------------------------------------
        
                marksample touse
                set trace on
                count if `touse'
                markout `touse' `adjust'
                count if `touse'
                set trace off
                if r(N) == 0 error 2000
                di "`unit'"
        
        /*
                foreach v of local adjust {
                    keep if !missing(`v')
                }
                
        */
            
            // Housekeeping
            *--------------        
                cap drop _rcs* _d_* tt means*            // Drops any variable created by prior stpm2
                qui stdescribe                            // Grab exit time
                    local t = floor(`r(tr_max)')         // Makes it an integer
                local vlab: variable label `varlist'    // Need for calls
                local endash = ustrunescape("\u2013")    // Need for calls
        
            * Add space b/w number and text if a text exists
            *-----------------------------------------------
        
                if     "`text0'" != ""    {
                    local s0 = " "
                    local s1 = " "
                }
                
                else if "`text1'" != ""{
                    local s0 = " "
                    local s1 = " "
                }
                
                else if "`text2'" != ""{
                    local s2 = " "
                }
                
                else {
                    local s0 = ""
                    local s1 = ""
                    local s2 = ""
                }
            
            // Restrict the program to take binary variables
            *-----------------------------------------------
                qui levelsof `varlist' if(`touse'), local(lev)
                scalar nt = `r(r)'
                cap assert nt>2
                    if !_rc {
                        disp "There are `=nt' levels"
                        }        
        
        // Run the FPM model
        *--------------------    
                qui stpm2 `varlist' `adjust' if(`touse'), df(`dof') `myopts' eform scale(h)
        
                mat li r(table)
                    tempname a
                    mat `a' = r(table)
        
            //    Calculate Somer's D explained variance
            *------------------------------------------
                qui str2d: `e(cmdline)'                // Repeats the stpm2 and runs the Somer's D
                local somd         = int(r(r2)*100)
                local somlow     = int(r(r2ll)*100)
                local somhigh     = int(r(r2ul)*100)
            
                // Gather HR, 95% and pvalue
                *--------------------------
                local b = string(`a'[1,1], "%09.1fc")
                local l = string(`a'[5,1], "%09.1fc")
                local u = string(`a'[6,1], "%09.1fc")
                local p = `a'[4,1]
                local n = e(N)    
                    
                // Fix p-value formats
                *---------------------
                    pvformat `p'
                    local pvalue = scalar(p)
                    local pfpm = "`pvalue'"
                    
                range tt 1 `t' `t'
                foreach i of local lev {
                qui predict means`i' if `touse', meansurv ci timevar(tt) at(`varlist' `i') 
                         local os`i' = string(round(means`i'[`t'] * 100, 0.1), "%9.1f") + "%"
                         scalar fos`i' = round(means`i'[`t'], 0.01)
                }
                
                //    Median survival frame
                *-------------------------
                tempname med
                frame create `med' 
                cap frame drop __00*
                frame copy default `med'
                frame `med'{
                    standsurv, atvars(med0 med1) at1(`varlist' 0) at2(`varlist' 1) centile(50) timevar(tt) ci
        
                    * Parametric median survival and CIs
                        *---------------------------------------
                         
        
                        * Get rid of decimal zeros with code (for e.g. make 3.0 as 3 and leave 3.1 as 3.1)
                        *---------------------------------------
                            foreach x of varlist med0-med1_uci {    
                                if mod(`x'[1],floor(`x'[1]))<.1{
                                     local `x' = floor(`x'[1])
                                }
                                else if mod(`x'[1],floor(`x'[1]))>=.1{
                                     local `x' : di %3.1f `x'[1]
                                }
                            }
        
                }
            
            // Create e-display text for medians and return scalars
            *------------------------------------------------------
            ereturn clear
                    ereturn local callmed10     "`med1' `unit'`s1'`text1' vs. `med0' `unit'`s0'`text0'"
                    ereturn local callmed01     "`med0' `unit'`s0'`text0' vs. `med1' `unit'`s1'`text1'"
                    ereturn local callmed1         "`med1' `unit' (95% CI: `med1_lci'`endash'`med1_uci' `unit')"
                    ereturn local callmed0         "`med0' `unit' (95% CI: `med0_lci'`endash'`med0_uci' `unit')"
                    
                    forval i = 0/1 {
                        ereturn scalar med`i'     = `med`i''
                        ereturn scalar med`i'_l = `med`i'_lci'
                        ereturn scalar med`i'_u = `med`i'_uci'
                    }
            
            // Create e-display text for the HR
            *---------------------------------        
            ereturn local callhr  =     "HR = `b' [`l'`endash'`u']; Pfpm `pfpm'"
            ereturn local callhrp =     "HR = `b' [`l'`endash'`u']; Pfpm `pfpm'; N = `n'"
            ereturn local pfpm             "`pfpm'"
            ereturn local b =             "`b'"    
            ereturn local l =             "`l'"
            ereturn local u =             "`u'"
            ereturn local n =             "`n'"
            ereturn local somd "`somd'% (95% CI: `somlow'`endash'`somhigh'%)"
            
            // ereturn needed OS local calls
            *-------------------------------
            ereturn local callos10     "`os1'`s1'`text1' vs. `os0'`s0'`text0'; Pfpm `pvalue'"
            ereturn local callos01     "`os0'`s0'`text0' vs. `os1'`s1'`text1'; Pfpm `pvalue'"
            
            // If there are more than 2 levels
            *---------------------------------
            if nt>2 {
                ereturn local callos012 "`os0'`s0'`text0' vs. `os1'`s1'`text2' vs. `os2'`s2'`text3'; Pfpm `pvalue'"
            }    
            
            ereturn local os0 = "`os0'"
            ereturn local os1 = "`os1'"
            ereturn scalar fos0 = fos0
            ereturn scalar fos1 = fos1
        
            if nt>2 {
                ret local os2 = "`os2'"
                ereturn scalar fos2 = fos2
            }
            
            //      Drop created vars
            *---------------------
            cap drop _rcs* _d_* tt means*
        
        
                * Display header text
        *-----------------------------------------------------------------------------
            di as text _dup(75) "-"
            di as text _col(15) "Flexible Parametric model for OS, HR and Somer's D"
            di _col(15) _dup(50) "-"
            di as text "Variable main      : "_column(25) as res "`varlist' (`vlab')"
            di as text "Adjusted for       : "_column(25) as res "`adjust'"
            di as text "Explained Variance : "_column(25) as res "`somd'%"
            di as text _dup(75) "-"
        ereturn li
            di as text _dup(75) "-"    
        *-----------------------------------------------------------------------------    
        end
          
          
        * Call the program
         fpm50 transplant, adjust(wt) unit(days)

        Comment

        Working...
        X