Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • r(301) "last estimates not found" after did_multiplegt

    Dear Statalists,

    I'm running the following codes on my Stata/SE15.0:

    ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
    // Generate a complete panel of 300 units observed in 15 periods
    clear all
    timer clear
    set seed 10
    global T = 15
    global I = 300

    set obs `=$I*$T'
    gen i = int((_n-1)/$T )+1 // unit id
    gen t = mod((_n-1),$T )+1 // calendar period
    tsset i t

    // Randomly generate treatment rollout years uniformly across Ei=10..16 (note that periods t>=16 would not be useful since all units are treated by then)
    gen Ei = ceil(runiform()*7)+$T -6 if t==1 // year when unit is first treated
    bys i (t): replace Ei = Ei[1]
    gen K = t-Ei // "relative time", i.e. the number periods since treated (could be missing if never-treated)
    gen D = K>=0 & Ei!=. // treatment indicator

    // Generate the outcome with parallel trends and heterogeneous treatment effects
    gen tau = cond(D==1, (t-12.5), 0) // heterogeneous treatment effects (in this case vary over calendar periods)
    gen eps = rnormal() // error term
    gen Y = i + 3*t + tau*D + eps // the outcome (FEs play no role since all methods control for them)

    did_multiplegt Y i t D, robust_dynamic dynamic(5) placebo(5) breps(100) cluster(i)

    matrix B = e(estimates)
    matrix B = B'

    matrix V = e(variances)
    matrix V = V'
    matrix V = diag(V)

    _coef_table, bmatrix(B) vmatrix(V)
    ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

    But I got the error "last estimates not found" after running the _coef_table line. Would appreciate any advice on addressing this issue, or on reformatting the output table of did_multiplegt into a standard coefficient table. Thanks a lot!

    Best,
    ​​​​​​​Yuting

  • #2
    Does the treatment begin in the very first time period?

    My apologies my laptop is upstairs so I can't test this

    Comment


    • #3
      I'm not sure how dangerous this is to do, ... but the issue appears to be in the source code (line 454) shown below. If we just comment out that line, you'll get the estimates. Again, that appears to be why the last estimates not found error is showing up, but I can't comment on whether or not modifying the code is dangerous or not (at the very least I would make a copy of the file and give it some alternate name like did_multiplegt_1 while preserving the original).

      Code:
      // Clearing ereturn
      
      ereturn clear

      Code:
        did_multiplegt Y i t D, robust_dynamic dynamic(5) placebo(5) breps(3) cluster(i)
      
      DID estimators of the instantaneous treatment effect, of dynamic treatment effects if the dynamic option is used, and of placebo tests of
      the parallel trends assumption if the placebo option is used. The estimators are robust to heterogeneous effects, and to dynamic effects
      if the robust_dynamic option is used.
      
                   |  Estimate         SE      LB CI      UB CI          N  Switchers 
      -------------+------------------------------------------------------------------
          Effect_0 |  .1408578   .1782057  -.2084254   .4901409       1211        250 
          Effect_1 |  .7141733    .219661   .2836379   1.144709        899        205 
          Effect_2 |  1.132017   .0689387   .9968972   1.267137        628        164 
          Effect_3 |  1.565301    .048699   1.469851   1.660751        398        117 
          Effect_4 |  1.899413   .0814378   1.739795   2.059031        215         70 
          Effect_5 |  2.773816   .4031253   1.983691   3.563942         83         33 
         Placebo_1 | -.0482671   .1531997  -.3485385   .2520043       1211        250 
         Placebo_2 |  .0779603    .084377  -.0874186   .2433392       1211        250 
         Placebo_3 | -.0718441   .1232574  -.3134286   .1697405       1211        250 
         Placebo_4 | -.0127561   .1800204  -.3655961   .3400838       1211        250 
         Placebo_5 | -.0682272   .1082057  -.2803104   .1438561       1211        250 
      
      When dynamic effects and first-difference placebos are requested, the command does
      not produce a graph, because placebos estimators are DIDs across consecutive time periods,
      while dynamic effects estimators are long-difference DIDs, so they are not really comparable.
      
      . 
      . matrix B = e(estimates)
      
      . matrix B = B'
      
      . 
      . matrix V = e(variances)
      
      . matrix V = V'
      
      . matrix V = diag(V)
      
      . 
      . _coef_table, bmatrix(B) vmatrix(V)
      ------------------------------------------------------------------------------
         diff_d_XX |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
      -------------+----------------------------------------------------------------
          Effect_0 |   .1408578   .1782057     0.79   0.432    -.2137156    .4954312
          Effect_1 |   .7141733    .219661     3.25   0.002     .2771169     1.15123
          Effect_2 |   1.132017   .0689387    16.42   0.000     .9948506    1.269183
          Effect_3 |   1.565301    .048699    32.14   0.000     1.468406    1.662197
          Effect_4 |   1.899413   .0814378    23.32   0.000     1.737377    2.061449
          Effect_5 |   2.773816   .4031253     6.88   0.000     1.971723    3.575909
         Placebo_1 |  -.0482671   .1531997    -0.32   0.754    -.3530864    .2565522
         Placebo_2 |   .0779603    .084377     0.92   0.358    -.0899234     .245844
         Placebo_3 |  -.0718441   .1232574    -0.58   0.562    -.3170877    .1733995
         Placebo_4 |  -.0127561   .1800204    -0.07   0.944    -.3709402     .345428
         Placebo_5 |  -.0682272   .1082057    -0.63   0.530    -.2835226    .1470683
      ------------------------------------------------------------------------------
      Note - I changed the bootstrap reps to 3 just to speed things up. did_multiplegt is available from SSC.
      Last edited by Justin Niakamal; 11 Aug 2022, 19:53.

      Comment


      • #4
        Originally posted by Jared Greathouse View Post
        Does the treatment begin in the very first time period?

        My apologies my laptop is upstairs so I can't test this
        No need to apologize! If I understand what you're asking correctly -- No, treatment starts in period 10 in the data. Thanks!

        Comment


        • #5
          Originally posted by Justin Niakamal View Post
          I'm not sure how dangerous this is to do, ... but the issue appears to be in the source code (line 454) shown below. If we just comment out that line, you'll get the estimates. Again, that appears to be why the last estimates not found error is showing up, but I can't comment on whether or not modifying the code is dangerous or not (at the very least I would make a copy of the file and give it some alternate name like did_multiplegt_1 while preserving the original).

          Code:
          // Clearing ereturn
          
          ereturn clear

          Code:
          did_multiplegt Y i t D, robust_dynamic dynamic(5) placebo(5) breps(3) cluster(i)
          
          DID estimators of the instantaneous treatment effect, of dynamic treatment effects if the dynamic option is used, and of placebo tests of
          the parallel trends assumption if the placebo option is used. The estimators are robust to heterogeneous effects, and to dynamic effects
          if the robust_dynamic option is used.
          
          | Estimate SE LB CI UB CI N Switchers
          -------------+------------------------------------------------------------------
          Effect_0 | .1408578 .1782057 -.2084254 .4901409 1211 250
          Effect_1 | .7141733 .219661 .2836379 1.144709 899 205
          Effect_2 | 1.132017 .0689387 .9968972 1.267137 628 164
          Effect_3 | 1.565301 .048699 1.469851 1.660751 398 117
          Effect_4 | 1.899413 .0814378 1.739795 2.059031 215 70
          Effect_5 | 2.773816 .4031253 1.983691 3.563942 83 33
          Placebo_1 | -.0482671 .1531997 -.3485385 .2520043 1211 250
          Placebo_2 | .0779603 .084377 -.0874186 .2433392 1211 250
          Placebo_3 | -.0718441 .1232574 -.3134286 .1697405 1211 250
          Placebo_4 | -.0127561 .1800204 -.3655961 .3400838 1211 250
          Placebo_5 | -.0682272 .1082057 -.2803104 .1438561 1211 250
          
          When dynamic effects and first-difference placebos are requested, the command does
          not produce a graph, because placebos estimators are DIDs across consecutive time periods,
          while dynamic effects estimators are long-difference DIDs, so they are not really comparable.
          
          .
          . matrix B = e(estimates)
          
          . matrix B = B'
          
          .
          . matrix V = e(variances)
          
          . matrix V = V'
          
          . matrix V = diag(V)
          
          .
          . _coef_table, bmatrix(B) vmatrix(V)
          ------------------------------------------------------------------------------
          diff_d_XX | Coef. Std. Err. t P>|t| [95% Conf. Interval]
          -------------+----------------------------------------------------------------
          Effect_0 | .1408578 .1782057 0.79 0.432 -.2137156 .4954312
          Effect_1 | .7141733 .219661 3.25 0.002 .2771169 1.15123
          Effect_2 | 1.132017 .0689387 16.42 0.000 .9948506 1.269183
          Effect_3 | 1.565301 .048699 32.14 0.000 1.468406 1.662197
          Effect_4 | 1.899413 .0814378 23.32 0.000 1.737377 2.061449
          Effect_5 | 2.773816 .4031253 6.88 0.000 1.971723 3.575909
          Placebo_1 | -.0482671 .1531997 -0.32 0.754 -.3530864 .2565522
          Placebo_2 | .0779603 .084377 0.92 0.358 -.0899234 .245844
          Placebo_3 | -.0718441 .1232574 -0.58 0.562 -.3170877 .1733995
          Placebo_4 | -.0127561 .1800204 -0.07 0.944 -.3709402 .345428
          Placebo_5 | -.0682272 .1082057 -0.63 0.530 -.2835226 .1470683
          ------------------------------------------------------------------------------
          Note - I changed the bootstrap reps to 3 just to speed things up. did_multiplegt is available from SSC.
          Thank you very much Justin. It worked (like magic)!

          If it's not too complicated to explain, may I ask why deleting this line helps? I am a little bit confused because
          (1) In did_multiplegt.ado, - ereturn clear - is located before all the -ereturn scalar ... - codes. I thought those codes "refill" what got cleared.
          (2) I reformatted the estimation result of did2s (from "https://raw.githubusercontent.com/kylebutts/did2s_stata/main/ado/") using similar codes as in the initial post. There's also the line - ereturn clear - in its ado file, but I didn't encouter the error.

          Thanks again!

          Comment


          • #6
            To better understand why that is, I recommend reading this blog post https://blog.stata.com/2015/11/10/pr...t-ado-command/
            and in particular, the "Storing results in e()" section. Regarding did2s, I am not at all familiar with that command.

            Comment


            • #7
              Originally posted by Justin Niakamal View Post
              To better understand why that is, I recommend reading this blog post https://blog.stata.com/2015/11/10/pr...t-ado-command/
              and in particular, the "Storing results in e()" section. Regarding did2s, I am not at all familiar with that command.
              Hi Justin, may you do me a favor? I followed the sample codes provided by Yuting Chen, using your suggestion (that is adding 'ereturn clear' option before the did_multiplegt command, but it did not work. I wonder if I might put the 'ereturn clear' option in the wrong place. Could you please kindly clarify? Thank you.

              Comment


              • #8
                Hi Dear Nguyen,

                I am interresting to obtain the table that you presente after did_multiplegt but I can not.
                Can you help me?

                Comment


                • #9
                  Originally posted by Minh Thu Nguyen View Post

                  Hi Justin, may you do me a favor? I followed the sample codes provided by Yuting Chen, using your suggestion (that is adding 'ereturn clear' option before the did_multiplegt command, but it did not work. I wonder if I might put the 'ereturn clear' option in the wrong place. Could you please kindly clarify? Thank you.

                  Hi Minh Thu Nguyen ,

                  Apologies I overlooked this. It looks like did_multiplegt has a save_results option. Germain, here's an example using some code courtesy of Asjad Naqvi

                  Code:
                  clear all
                  
                  local units = 30
                  local start = 1
                  local end     = 60
                  
                  local time = `end' - `start' + 1
                  local obsv = `units' * `time'
                  set obs `obsv'
                  
                  egen id       = seq(), b(`time')  
                  egen t        = seq(), f(`start') t(`end')    
                  
                  sort  id t
                  xtset id t
                  
                  
                  set seed 20211222
                  
                  gen Y                = 0        // outcome variable    
                  gen D                = 0        // intervention variable
                  gen cohort      = .      // treatment cohort
                  gen effect      = .        // treatment effect size
                  gen first_treat = .        // when the treatment happens for each cohort
                  gen rel_time    = .     // time - first_treat
                  
                  levelsof id, local(lvls)
                  foreach x of local lvls {
                      local chrt = runiformint(0,5)    
                      replace cohort = `chrt' if id==`x'
                  }
                  
                  
                  levelsof cohort , local(lvls)
                  foreach x of local lvls {
                      
                      local eff = runiformint(2,10)
                          replace effect = `eff' if cohort==`x'
                              
                      local timing = runiformint(`start',`end' + 20)    //
                      replace first_treat = `timing' if cohort==`x'
                      replace first_treat = . if first_treat > `end'
                          replace D = 1 if cohort==`x' & t>= `timing'
                  }
                  
                  replace rel_time = t - first_treat
                  replace Y = id + t + cond(D==1, effect * rel_time, 0) + rnormal()
                  
                  
                  did_multiplegt Y i t D, robust_dynamic dynamic(5) placebo(5) breps(3) cluster(i) save_results(results)
                  
                  . use results, clear 
                  
                  . list, noobs sep(0)
                  
                    +-------------------------------------------------------------------+
                    | time_t~t   treatme~t   se_tre~t   N_trea~t   t~uppe~I   t~lower~I |
                    |-------------------------------------------------------------------|
                    |       -6    .0679468   .1028779         78   .2695876    -.133694 |
                    |       -5    .2572878   .1811405         78   .6123232   -.0977476 |
                    |       -4    .0639963    .337783         78   .7260509   -.5980584 |
                    |       -3    .1944381   .3412453         78   .8632789   -.4744026 |
                    |       -2   -.1308918   .4874427         78   .8244959    -1.08628 |
                    |       -1           0          .          .          0           0 |
                    |        0   -.0608394   .4495481         78   .8202748   -.9419537 |
                    |        1     8.49767   .2876669         78   9.061497    7.933843 |
                    |        2    17.64773   .6142032         78   18.85157    16.44389 |
                    |        3     25.9377   .2335396         78   26.39543    25.47996 |
                    |        4    34.62362   .7952709         75   36.18235    33.06489 |
                    |        5    42.85682   .7264699         64    44.2807    41.43294 |
                    +-------------------------------------------------------------------+

                  Comment


                  • #10
                    Dear Justin Niakamal ,

                    Thank you for your adivce, but when I run the code above I obtain the same error ("last estimates not found") even if I write "ereturn clear". I know that the command provides the option "save_results", but I prefer the table that you have shown in your first example.
                    Therefore, I am wondering why I am having this error and I would like to ask you if you could clarify your answer.
                    Thank you in advance.

                    Comment

                    Working...
                    X