Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Bootstrapping difference

    Hello,

    I want to test whether the differential of two coefficients is significantly larger than the differential of other two coefficients. If I want to bootstrap this, do I need to bootstrap in the estimation of each coefficient and then perform the test or not bootstrap in the estimation of the coefficients and bootstrap in the test?

    Thanks

  • #2
    Charlotte:
    I would say to bootstrap the test.
    However, please consider the above as a tempative reply, due to the very limited details about your research goal.
    Kind regards,
    Carlo
    (Stata 18.0 SE)

    Comment


    • #3
      To be more specific.

      I run the same two regressions on different samples:
      reg y x z [pw=w_f] , vce(cl id) /*sample 1*/
      reg y x z [pw=w_f] , vce(cl id) /*sample 2*/

      Then I calculate the differential of the x coefficient in the first regression and the x coefficient in the second regression

      Then I run two other regressions on two other samples. And once again I calculate differential of the x coefficient in the latter two regressions.

      Finally I want to test whether the first differential is significantly larger than the second differential. How can I bootstrap this test?


      Comment


      • #4
        You would need to write your own program to feed to bootstrap:

        Code:
        // This presumes you have a variable in your data that marks each "sample" you want to analyze.
        cap prog drop myregdiff
        prog myregdiff, rclass
        //
           reg y x z [pw=w_f] if sample == 1, vce(cl id) /*sample 1*/
           local b = _b[x]
           reg y x z [pw=w_f]  if sample == 2, vce(cl id) /*sample 2*/
           local diff12 = `b' - _b[x]
        //
           reg y x z [pw=w_f] if sample == 3, vce(cl id) /*sample 3*/
           local b = _b[x]
           reg y x z [pw=w_f] if sample == 4, vce(cl id) /*sample 4*/
           local diff34 = `b' - _b[x]
        //
           return scalar diff12_34 = `diff12' - `diff34'
        end
        //
        bootstrap diff = r(diff12_34), ....... options ..... : myregdiff
        This will give you a bootstrap standard error and confidence interval on "diff." I would not regard this as a test, since the standard error is not estimated under conditions that enforce a null hypothesis.

        Regards, Mike

        Comment


        • #5
          Thank you Mike!! When running "
          bootstrap diff = r(diff12_34), ....... options ..... : myregdiff I recieve the following error: insufficient observations to compute bootstrap standard errors no results will be saved Do you have an idea why this could be?

          Comment


          • #6
            Charlotte:
            this may be due to the fact that there's no variance in some of the items included in -bootstrap-.
            Kind regards,
            Carlo
            (Stata 18.0 SE)

            Comment


            • #7
              Carlo: Thanks for your reply. I am following Mike's code. The bootstrap is performed to obtain a standard error and a confidence interval on a scalar. Could you be more specific with what you mean with "no variance in some of the items included"?

              Comment


              • #8
                Charlotte:
                I meant that -bootstrap- procedure finds hard times in calculatng SE possibly due to too many reopeated observations in your simulation.
                There are some interesting threads about the isue you're facing. The one I like the most is the following one: http://www.stata.com/statalist/archi.../msg01156.html.
                You can access all of them by googling with the following string: -insufficient observations to compute bootstrap standard errors stata-.
                Kind regards,
                Carlo
                (Stata 18.0 SE)

                Comment


                • #9
                  I don't have any brilliant ideas here. I'd look at the size of the subgroups. If one of them is small, and if the variance of one or another variable is low in that group in the full sample, I can imagine a bootstrap sample happening to not have much variance.
                  You might try putting in a -tabstat- request (e.g., -tabstat x1 x2 ..., by(sample) statistics(mean sd)- ) at the top of the program, and then run -bootstrap- with the noisily option. That will create a lot of messy output, but might show you what's going on.

                  Regards, Mike

                  Comment


                  • #10
                    Mike, my problem could be that I have a panel. I copy my code in case it clarifies my issue:


                    xtset id

                    cap prog drop myregdiff
                    prog myregdiff, rclass

                    reg y x $correl i.sub [pw=w_f] if y2==1, vce(cl sch)
                    local b = _b[x]
                    reg y x $correl i.sub [pw=w_f] if y2==0, vce(cl sch)
                    local diff12 = `b' - _b[x]
                    //
                    xtreg y x i.sub if y2==1, fe vce(cl sch)
                    local b = _b[x]
                    xtreg y x i.sub if y2==0,fe vce(cl sch)
                    local diff34 = `b' - _b[x]
                    //
                    return scalar diff12_34 = `diff12' - `diff34'
                    end
                    //
                    bootstrap diff = r(diff12_34): myregdiff

                    The error I receive is:
                    insufficient observations to compute bootstrap standard errors
                    no results will be saved
                    r(2000);

                    Comment


                    • #11
                      I haven't used -bootstrap- with an -xt- estimation command. While I know that it can be done (see http://www.stata.com/support/faqs/st...th-panel-data/), I don't know if it can be done in a way that also fits with your -reg- command that does *not* treat the data as a panel. I don't know enough to be helpful with that problem, sorry.

                      Regards, Mike

                      Comment


                      • #12
                        Mike: I suspect that my problem is having more than one sample. When I run the same code only for one of the samples, then it works perfectly fine. Do you know how I can fix this problem?

                        Comment


                        • #13
                          So basically the question is whether STATA allows bootstrap from more than one sample

                          Comment


                          • #14
                            Per my earlier comment, no, I don't have any detailed knowledge about bootstrapping with an -xt- command.

                            I would also say that I don't understand your observation that "question is whether [Stata] allows bootstrap from more than one sample," since the code I provided you earlier manifestly does that. I think the conceptual issue here is that -xt- corresponds to one kind of sampling model, and non-xt estimation commands to another, so I don't see how both sampling models could be accommodated within one bootstrapping experiment. Again, bootstrapping with -xt- commands is outside my experience and likely outside my expertise; I am only pointing to what I see as a possible problem, and I don't have a solution.

                            Regards, Mike

                            Comment


                            • #15
                              I have used very similar code to that shown by Charlotte at #10 and at times get the same error as at #10. In my experience it is always a small-sample problem; if I increase the sample (artificially or otherwise) Stata completes the bootstrap.
                              Regards
                              Laurence

                              Comment

                              Working...
                              X