why the bias corredted confidence interval is missing?

Fred Lee

Join Date: Nov 2017
Posts: 473

why the bias corredted confidence interval is missing?

24 Dec 2019, 12:54

Hi all,
why the confidence interval after bootstraping is missing?

Code:

* Example generated by -dataex-. To install: ssc install dataex
clear
input byte(gender age edu) float(id serial gpMediator gpIV DV gpDV Mediator)
0 49 2 1 1   .54861116     .15625   5         1  4.944445
0 49 2 1 2  -.17361116    -.09375   4         0 4.2222223
0 49 2 1 3   .54861116    -.09375   4         0  4.944445
0 49 2 1 4      -.0625    -.09375   4         0 4.3333335
0 49 2 1 5    .3263888    -.09375   4         0 4.7222223
0 49 2 1 6   -.3958335    -.09375   3        -1         4
0 49 2 1 7   -.3958335     .40625   4         0         4
0 49 2 1 8   -.3958335    -.09375   4         0         4
0 46 1 2 1      -.4375    1.46875   4     -.375  4.111111
0 46 1 2 2   -.3819447     .71875   4     -.375 4.1666665
0 46 1 2 3   .00694418    -.53125   4     -.375 4.5555553
0 46 1 2 4    .2291665    -.03125   4     -.375  4.777778
0 46 1 2 5  -.54861116   -1.03125   4     -.375         4
0 46 1 2 6    .2291665     .21875   5      .625  4.777778
0 46 1 2 7    .4513888    -.28125   5      .625         5
0 46 1 2 8    .4513888    -.53125   5      .625         5
0 41 3 3 1 -.013888836    -.09375   4     .0625         4
0 41 3 3 2 -.013888836     .15625   4     .0625         4
0 41 3 3 3 -.013888836    -.09375   4     .0625         4
0 41 3 3 4   .09722233    -.09375 3.5    -.4375  4.111111
0 41 3 3 5 -.013888836    -.09375   4     .0625         4
0 41 3 3 6 -.013888836     .15625   4     .0625         4
0 41 3 3 7 -.013888836    -.09375   4     .0625         4
0 41 3 3 8 -.013888836     .15625   4     .0625         4
0 46 3 4 1 -.007936478 .035714388   4 1.2857144  4.111111
0 46 3 4 2   .04761887  -.9642856   3  .2857144 4.1666665
0 46 3 4 3   -.0634923   .2857144   3  .2857144 4.0555553
0 46 3 4 4  -.11904764   .2857144   2 -.7142856         4
0 46 3 4 5   .10317469   .7857144   2 -.7142856 4.2222223
0 46 3 4 6     .325397 .035714388   3  .2857144 4.4444447
0 46 3 4 7           .          .   .         .         .
0 46 3 4 8   -.2857144  -.4642856   2 -.7142856  3.833333
1 59 2 5 1     .583333     .03125 3.5     .8125 4.5555553
1 59 2 5 2   .02777767    -.21875   1   -1.6875         4
1 59 2 5 3    .4166665    -.21875   2    -.6875  4.388889
1 59 2 5 4        -.25     .03125 3.5     .8125  3.722222
1 59 2 5 5   -.3611112     .53125 3.5     .8125  3.611111
1 59 2 5 6  -.19444466     .28125 2.5    -.1875  3.777778
1 59 2 5 7        -.25    -.21875 2.5    -.1875  3.722222
1 59 2 5 8   .02777767    -.21875   3     .3125         4
0 40 5 6 1   -.0833335     .65625   4    -.0625  4.111111
0 40 5 6 2   .47222185     .40625 4.5     .4375 4.6666665
0 40 5 6 3   .02777767     .15625 4.5     .4375 4.2222223
0 40 5 6 4   -.0833335    -.34375   4    -.0625  4.111111
0 40 5 6 5   -.0833335    -.34375   4    -.0625  4.111111
0 40 5 6 6  -.13888931    -.34375 3.5    -.5625 4.0555553
0 40 5 6 7   -.0833335     .15625   4    -.0625  4.111111
0 40 5 6 8  -.02777815    -.34375   4    -.0625 4.1666665
0 41 4 7 1       .0625      -.125   4     -.125 4.1666665
0 41 4 7 2   .34027815      -.125   5      .875 4.4444447
0 41 4 7 3   .11805582      -.125   4     -.125 4.2222223
0 41 4 7 4   -.1041665      -.125   4     -.125         4
0 41 4 7 5   -.1041665       .375   4     -.125         4
0 41 4 7 6   -.1041665       .125   4     -.125         4
0 41 4 7 7   -.1041665       .125   4     -.125         4
0 41 4 7 8   -.1041665      -.125   4     -.125         4
1 36 4 8 1           .          .   .         .         .
1 36 4 8 2           .          .   .         .         .
1 36 4 8 3  -.11111116       .125 4.5       1.5 4.6666665
1 36 4 8 4  -.11111116      -.375 3.5        .5 4.6666665
1 36 4 8 5  -.05555534       .125   3         0 4.7222223
1 36 4 8 6   .22222233      -.375   2        -1         5
1 36 4 8 7     .166667       .375   3         0  4.944445
1 36 4 8 8  -.11111116       .125   2        -1 4.6666665
0 36 3 9 1    .4722221          0   4         0  3.888889
0 36 3 9 2   .58333325        .25   4         0         4
0 36 3 9 3   -2.027778       -.25   4         0  1.388889
0 36 3 9 4         .25        .25   4         0  3.666667
0 36 3 9 5         .25          0   4         0  3.666667
0 36 3 9 6    .4722221       -.25   4         0  3.888889
end
label values gender gender
label def gender 0 "男", modify
label def gender 1 "女", modify
label values edu edu
label def edu 1 "大专以下", modify
label def edu 2 "大专", modify
label def edu 3 "本科", modify
label def edu 4 "硕士", modify
label def edu 5 "博士", modify


xtset id serial

global ctrl gender age edu





****//examine medaition
set seed 10000
capture program drop Med
program define Med, rclass

mixed Mediator $ctrl l.gpMediator cl.gpIV || id: l.gpMediator  cl.gpIV , cov(exc)
    return scalar a=(_b[Mediator:L.gpIV] )
    
mixed DV $ctrl l.gpDV cl.gpIV c.gpMediator|| id:cl.gpIV c.gpMediator l.gpDV,  cov(exc)
    return scalar b=(_b[DV:gpMediator])    
end

bootstrap mediation=(r(a)*r(b)) , reps(5) reject(e(converged)!=1) cluster(id) idcluster(newid) group(serial): Med
estat bootstrap,percentile bc
program drop Med

Last edited by Fred Lee; 24 Dec 2019, 12:58.

Tags: None

Fred Lee

Join Date: Nov 2017

Posts: 473
#2

05 Jan 2020, 18:09

can anyone help me out of this? Thanks very much!
Comment

Leonardo Guizzetti

Join Date: Jul 2016
Posts: 2400

06 Jan 2020, 12:43

Hi Fred,

The issue is that you don't have enough replications to make a valid interval in your example, when 3 of 5 replications result in non-valid results (the models did not converge). In addition, your code, I believe needs to -xtset- on the new id variable in order to properly account for the cluster resampling.

The relevant changes (and some tidying) are below:

Code:

set seed 10000
capture program drop Med
program define Med, rclass

  xtset newid serial

  mixed Mediator $ctrl l.gpMediator cl.gpIV || id: l.gpMediator  cl.gpIV , cov(exc) iter(200)
  scalar med_conv = e(converged)
  scalar a = _b[Mediator:L.gpIV]
  return scalar a=a
    
  mixed DV $ctrl l.gpDV cl.gpIV c.gpMediator || id:cl.gpIV c.gpMediator l.gpDV,  cov(exc) iter(200)
  di "a"
  scalar dv_conv = e(converged)
  scalar b = _b[DV:gpMediator]
  return scalar b=b
 
  scalar all_conv = med_conv==1 & dv_conv==1
  return scalar all_conv = all_conv
 
  return scalar abprod = a * b
end

bootstrap mediation=r(abprod) , reps(20) reject(r(all_conv)!=1) cluster(id) idcluster(newid) group(serial): Med
estat bootstrap, all

Comment

Fred Lee

Join Date: Nov 2017

Posts: 473
#4

07 Jan 2020, 09:15

Originally posted by Leonardo Guizzetti View Post

Hi Fred,

The issue is that you don't have enough replications to make a valid interval in your example, when 3 of 5 replications result in non-valid results (the models did not converge). In addition, your code, I believe needs to -xtset- on the new id variable in order to properly account for the cluster resampling.

The relevant changes (and some tidying) are below:

Code:

set seed 10000 capture program drop Med program define Med, rclass xtset newid serial mixed Mediator $ctrl l.gpMediator cl.gpIV || id: l.gpMediator cl.gpIV , cov(exc) iter(200) scalar med_conv = e(converged) scalar a = _b[Mediator:L.gpIV] return scalar a=a mixed DV $ctrl l.gpDV cl.gpIV c.gpMediator || id:cl.gpIV c.gpMediator l.gpDV, cov(exc) iter(200) di "a" scalar dv_conv = e(converged) scalar b = _b[DV:gpMediator] return scalar b=b scalar all_conv = med_conv==1 & dv_conv==1 return scalar all_conv = all_conv return scalar abprod = a * b end bootstrap mediation=r(abprod) , reps(20) reject(r(all_conv)!=1) cluster(id) idcluster(newid) group(serial): Med estat bootstrap, all

Thanks a lot, Leonardo!
The pronblem in the sample data can be solved, however, the missing problem of confidence inerval in my original data still exists, using your code. Could you please explain a bit more the possible reason?
Thanks again!
Comment

Fred Lee

Join Date: Nov 2017
Posts: 473

07 Jan 2020, 09:19

Originally posted by Leonardo Guizzetti View Post

Code:

set seed 10000
capture program drop Med
program define Med, rclass

xtset newid serial

mixed Mediator $ctrl l.gpMediator cl.gpIV || id: l.gpMediator cl.gpIV , cov(exc) iter(200)
scalar med_conv = e(converged)
scalar a = _b[Mediator:L.gpIV]
return scalar a=a

mixed DV $ctrl l.gpDV cl.gpIV c.gpMediator || id:cl.gpIV c.gpMediator l.gpDV, cov(exc) iter(200)
di "a"
scalar dv_conv = e(converged)
scalar b = _b[DV:gpMediator]
return scalar b=b

scalar all_conv = med_conv==1 & dv_conv==1
return scalar all_conv = all_conv

return scalar abprod = a * b
end

bootstrap mediation=r(abprod) , reps(20) reject(r(all_conv)!=1) cluster(id) idcluster(newid) group(serial): Med
estat bootstrap, all

By the way, I can send you the original data through email if you would like to help me to have a look!
Thanks very much!

Comment

Fred Lee

Join Date: Nov 2017

Posts: 473
#6

07 Jan 2020, 09:21

here is the information for your possible use.
Comment

Leonardo Guizzetti

Join Date: Jul 2016
Posts: 2400

07 Jan 2020, 09:39

Hi Fred,

There is surely something wrong with your data. You're screenshot shows that despite 20,000 replicates, there is absolutely no variation among estimates, as confirmed by the bootstrap SE equal to exactly zero, and percentile confidence intervals also identical.

Here is the output I get from the code I wrote above using your data example. Note I only ran 20 bootstrap replicates. Notice that a relatively large number of replicates (6/20) were rejected for failure to converge, yet a bias-corrected confidence interval is still estimable. So there is nothing necessarily wrong with the process of doing the bootstrap, but with how your data appear.

Code:

. bootstrap mediation=r(abprod) , reps(20) reject(r(all_conv)!=1) cluster(id) idcluster(newid) group(serial): Med
(running Med on estimation sample)

Bootstrap replications (20)
----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5
..xx.....x..x..xx...

Bootstrap results                               Number of obs     =         57
                                                Replications      =         14

      command:  Med
    mediation:  r(abprod)

                                      (Replications based on 9 clusters in id)
------------------------------------------------------------------------------
             |   Observed   Bootstrap                         Normal-based
             |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
   mediation |   .8060528   22.19868     0.04   0.971    -42.70256    44.31466
------------------------------------------------------------------------------
Note: One or more parameters could not be estimated in 6 bootstrap replicates;
      standard-error estimates include only complete replications.

. estat bootstrap, all

Bootstrap results                               Number of obs     =         57
                                                Replications      =         14

      command:  Med
    mediation:  r(abprod)

                                      (Replications based on 9 clusters in id)
------------------------------------------------------------------------------
             |    Observed               Bootstrap
             |       Coef.       Bias    Std. Err.  [95% Conf. Interval]
-------------+----------------------------------------------------------------
   mediation |    .8060528   1.439076   22.198679   -42.70256   44.31466   (N)
             |                                      -35.51286   56.79079   (P)
             |                                      -35.51286   56.79079  (BC)
------------------------------------------------------------------------------
(N)    normal confidence interval
(P)    percentile confidence interval
(BC)   bias-corrected confidence interval
Note: One or more parameters could not be estimated in 6 bootstrap replicates;
      standard-error estimates include only complete replications.

Comment

Fred Lee

Join Date: Nov 2017

Posts: 473
#8

07 Jan 2020, 10:13

Thanks a lot, Leonardo!
How to find the problems of the data? Could you please tell me your email so that I can send you the original data?
Comment
Leonardo Guizzetti

Join Date: Jul 2016

Posts: 2400
#9

07 Jan 2020, 10:29

Sorry, there's one issue I overlooked. the clustering of -id- should be changed to -newid- within the Med progam in order to match the fact that a new cluster variable, -newid- is being created for the bootstrap command.

As for the problems with the data or model specification, I'm not sure how I can help you because I don't use these models. You may try to simplify the model specification if it makes sense to do so, or you can try using a smaller subset of clusters to see if you can get some reasonable looking results. I also will not accept emails containing attachments from public forums to be cautious.
Comment
Fred Lee

Join Date: Nov 2017

Posts: 473
#10

07 Jan 2020, 12:09

Originally posted by Leonardo Guizzetti View Post

Sorry, there's one issue I overlooked. the clustering of -id- should be changed to -newid- within the Med progam in order to match the fact that a new cluster variable, -newid- is being created for the bootstrap command.

As for the problems with the data or model specification, I'm not sure how I can help you because I don't use these models. You may try to simplify the model specification if it makes sense to do so, or you can try using a smaller subset of clusters to see if you can get some reasonable looking results. I also will not accept emails containing attachments from public forums to be cautious.

Alright, thanks again! I will try to check the data.
Comment

Announcement

why the bias corredted confidence interval is missing?

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment