dtable Command & Correction for Multiple Testing

[Kevin Blaine]

Hi all -- I know Jeff Pitblado (StataCorp) is the resident expert on dtable here on the forums, so I'm hoping he might chime in (he's been super helpful in the past!)

Is there anyway to use dtable AND account for multiple testing? I know pearson and regress are the primary workhorses for the testing, but I haven't yet found a way to incorporate a correction for multiple testing. I can always adjust the p-value thresholds, but I've had my physician researcher colleagues express confusion when previously employing that approach. I'd much rather adjust the p-values themselves to fall within the tradition "0.05/0.01/0.001" delineation of significance.

Do you have any insight?

Thanks,
Kevin

[Jeff Pitblado (StataCorp)]

dtable does not currently have any option/feature that adjusts p-values for multiple comparisons.

At the time we were developing dtable we were not aware of anyone doing this. Can you provide an example and point out where/how you want the p-values to be adjusted for multiple comparisons?

[Kevin Blaine]

Sorry for the delay in getting back to you (I've been out of the country).

See the example below:

* Example generated by -dataex-. For more info, type help dataex
clear
input int studyid byte gender float(age_cat race_ethnicity_sorted) byte(language religion payor) float(coi_overall_cat coi_edu_cat coi_social_cat coi_health_cat type_embryo grade_cat) byte num_sym float predispose_cat byte outpatient float delay
  1 2 0 3 1 1 1 40 80 20 20 3 . 0 0 0 0
  2 2 2 2 1 1 1 80 80 80 40 2 1 0 0 1 0
  3 2 0 2 1 1 1 80 80 80 60 3 . 0 0 1 0
  4 2 3 1 2 0 1 20 20 20 20 2 1 0 0 1 0
  5 2 1 3 1 1 2 80 80 80 40 2 1 0 0 0 0
  6 2 1 2 1 1 2 40 40 60 20 2 1 0 1 0 0
  7 2 1 2 1 0 1 60 80 60 40 2 1 0 0 1 0
  8 1 1 2 1 1 2 20 20 20 60 2 1 0 0 0 0
  9 2 3 2 1 0 1 60 60 40 60 2 1 1 1 1 0
 10 2 2 2 1 1 1 60 80 60 20 2 1 1 0 1 0
 11 1 2 3 1 1 1 60 60 60 20 2 1 0 1 1 0
 12 2 2 3 1 1 2 80 80 80 80 2 1 0 0 0 0
 13 1 1 1 2 0 1 60 60 60 40 2 2 1 0 0 0
 14 2 1 1 2 1 2 40 60 40 20 2 2 0 0 0 0
 15 2 0 2 1 1 2 80 80 80 60 2 1 0 0 1 0
 16 1 1 2 1 1 1 80 80 80 80 1 2 2 0 0 0
 17 2 1 2 1 1 2 80 80 80 60 2 2 0 0 0 0
 18 2 1 3 1 1 2 80 80 80 80 2 1 2 0 1 0
 19 1 0 3 1 1 1 60 80 60 40 1 2 1 0 0 0
 20 1 1 2 1 1 2 80 60 80 80 2 . 1 0 0 0
 21 2 3 1 2 1 2 40 20 40 60 2 1 2 0 0 0
 22 2 0 3 1 1 2 40 60 40 40 1 2 1 0 0 0
 23 2 2 1 1 1 2 80 80 80 40 3 2 2 0 0 0
 24 1 1 1 1 1 1 80 80 80 60 1 2 1 0 0 0
 25 2 1 1 1 0 1 20 60 20 20 1 2 3 0 0 0
 26 2 1 1 1 0 1 40 60 40 20 1 2 2 0 0 0
 27 2 3 1 1 1 2 80 80 80 60 3 1 1 0 0 0
 28 2 0 3 1 1 2 80 80 80 60 1 2 1 0 0 0
 29 2 3 2 1 0 2 40 40 60 20 2 1 3 0 0 0
 30 2 1 2 1 1 2 80 80 80 60 2 2 0 0 0 0
 31 1 2 2 2 1 2 20 40 40 20 2 2 2 0 0 0
 32 2 1 2 1 0 2 60 80 60 20 2 2 2 0 0 0
 33 1 1 3 1 1 1 80 80 60 40 2 2 3 0 0 0
 34 1 1 2 1 0 1 60 80 40 40 2 1 1 0 0 0
 35 2 1 2 1 0 2 60 60 60 20 3 2 1 0 0 0
 36 2 2 1 2 1 1 40 60 40 20 3 2 3 0 0 0
 37 1 2 1 1 1 2 20 40 40 20 2 2 1 0 0 0
 38 1 1 2 1 0 1 80 80 80 60 3 . 1 0 1 0
 39 2 1 2 1 1 1 80 80 60 60 1 2 2 0 0 0
 40 2 3 . 1 1 1 80 80 80 60 1 2 1 0 1 0
 41 1 1 1 1 1 2 20 20 20 20 3 2 1 0 0 0
 42 2 1 2 1 0 1 20  0 20 20 2 1 1 0 0 0
 43 1 1 1 1 0 1 40 60 40 20 1 2 3 0 0 0
 44 2 1 1 1 1 2 40 60 40  0 2 1 2 0 0 0
 45 2 3 1 1 1 2 40 40 60 20 1 2 3 0 0 0
 46 1 2 1 1 0 2 60 60 60 20 3 2 1 0 0 0
 47 2 1 1 2 0 1 40 60 40 20 1 2 2 0 0 0
 48 2 2 3 1 1 2 80 80 60 40 1 2 3 0 0 0
 49 1 1 1 1 1 1 40 60 40 20 1 2 1 0 0 0
 50 1 1 . 1 1 1  0 20  0 20 2 2 2 0 1 0
 51 1 1 3 1 0 1 60 60 60 20 2 2 1 0 0 0
 52 2 1 1 1 1 2 80 80 60 40 2 2 2 0 0 0
 53 1 2 1 1 1 2 60 80 60 40 2 2 2 0 0 0
 54 2 2 3 1 0 1 80 80 60 40 2 2 1 0 0 0
 55 2 1 2 1 1 1 80 80 80 80 3 . 1 1 1 0
 56 1 1 3 1 1 2 60 60 80 40 2 2 2 0 0 0
 57 1 1 3 1 1 2 80 80 80 40 2 2 1 0 0 0
 58 2 0 3 1 1 2 80 80 60 40 2 . 0 0 1 0
 59 2 1 1 1 0 1 80  . 60 60 1 2 1 0 0 0
 60 1 0 2 1 1 1 80 80 80 60 1 2 1 0 0 0
 61 1 1 2 1 0 1 80 80 80 60 2 2 2 0 0 0
 62 1 1 1 1 1 2 60 60 40 40 2 1 1 0 0 0
 63 2 1 1 2 0 1 80 80 80 40 2 2 1 0 0 0
 64 2 1 2 1 1 2 20 20 20 20 1 2 2 0 0 0
 65 1 1 1 2 1 1 20 40 40 20 2 1 3 0 0 0
 66 1 1 1 1 1 2 40 60 40 20 2 1 1 0 0 0
 67 1 1 1 1 1 2 40 20 40 20 1 2 2 0 0 0
 68 1 1 1 1 0 1 60 80 60 40 2 1 2 0 0 0
 69 2 3 2 1 0 2 60 60 60 60 2 1 1 0 0 0
 70 1 1 2 1 0 1 80 80 80 40 1 2 2 0 0 0
 71 1 2 2 1 1 2 80 80 80 80 2 1 1 0 0 0
 72 2 1 1 1 1 1 20 20 20  0 3 2 2 0 0 0
 73 1 1 1 2 1 1 60 60 60 40 1 2 3 0 0 0
 74 2 1 1 1 1 2 40 40 60 20 2 . 2 0 0 0
 75 2 2 3 1 1 1 20 60 20 20 2 . 1 0 1 0
 76 1 0 3 1 1 1 20 20 40  0 2 2 2 0 0 0
 77 2 1 3 1 1 1 40 80 40 20 2 . 1 0 0 0
 78 1 1 2 1 0 2 60 60 60 20 2 2 1 0 0 0
 79 2 2 3 1 0 1 60 60 60 20 2 2 2 0 0 0
 80 2 1 3 1 1 1 60 60 40 40 2 1 2 0 0 0
 81 1 2 1 2 0 1 20 20 20  0 1 2 1 0 0 0
 82 2 1 2 1 1 2 80 80 80 60 1 2 3 0 0 0
 83 1 1 1 1 1 2  0  0  0 20 2 1 2 0 0 0
 84 2 2 1 2 1 2 20  0 20  0 2 1 3 0 1 0
 85 2 2 1 2 1 1 40 20 40 20 1 2 3 0 0 0
 86 1 1 1 1 1 1 20  0 40 20 2 1 3 0 1 0
 87 2 3 2 1 0 2 80 80 80 60 3 1 1 0 1 0
 88 1 2 1 1 1 1 40 40 40 20 2 1 2 0 1 0
 89 1 1 1 1 0 1 80 80 80 40 3 2 2 0 1 0
 90 2 3 1 2 0 1 40 20 40 20 3 1 1 1 1 0
 91 2 3 2 1 0 2 60 80 60 20 3 . 2 0 1 0
 92 2 0 1 1 1 1 20 20 20  0 1 2 1 0 0 1
 93 2 1 1 1 1 1 40 40 60 20 1 2 3 0 0 1
 94 2 3 2 1 1 2 60 60 60 40 3 . 2 0 0 1
 95 2 2 2 1 0 1 80 80 80 40 1 2 2 0 0 1
 96 2 1 1 1 0 2 40 40 40 20 1 2 1 1 1 1
 97 2 1 1 1 1 1 80 80 80 40 1 2 1 0 0 1
 98 2 3 1 1 1 2 20 20 20  0 2 1 1 0 0 1
 99 2 1 2 1 1 1 40 20 40 20 2 1 2 0 0 1
100 1 3 1 2 1 1 80 80 80 80 3 1 2 0 0 1
end
label values gender gender
label def gender 1 "Female", modify
label def gender 2 "Male", modify
label values age_cat la_age
label def la_age 0 "Infant (<2)", modify
label def la_age 1 "Early Childhood (2-7)", modify
label def la_age 2 "Late Childhood (8-13)", modify
label def la_age 3 "Adolescence (14+)", modify
label values race_ethnicity_sorted race_ethnicity_sorted
label def race_ethnicity_sorted 1 "Hispanic", modify
label def race_ethnicity_sorted 2 "Non-hispanic White", modify
label def race_ethnicity_sorted 3 "Other", modify
label values language language
label def language 1 "English", modify
label def language 2 "Spanish", modify
label values religion la_yesno
label values outpatient la_yesno
label values predispose_cat la_yesno
label def la_yesno 0 "No", modify
label def la_yesno 1 "Yes", modify
label values payor payor
label def payor 1 "Private", modify
label def payor 2 "Public", modify
label values coi_overall_cat la_coi_cat
label values coi_edu_cat la_coi_cat
label values coi_social_cat la_coi_cat
label values coi_health_cat la_coi_cat
label def la_coi_cat 0 "Very Low", modify
label def la_coi_cat 20 "Low", modify
label def la_coi_cat 40 "Moderate", modify
label def la_coi_cat 60 "High", modify
label def la_coi_cat 80 "Very High", modify
label values type_embryo la_type_embryo
label def la_type_embryo 1 "Embryonal", modify
label def la_type_embryo 2 "Glioma", modify
label def la_type_embryo 3 "Other", modify
label values grade_cat la_grade_cat
label def la_grade_cat 1 "Low Grade", modify
label def la_grade_cat 2 "High Grade", modify
label values num_sym la_sym
label def la_sym 0 "0", modify
label def la_sym 1 "1", modify
label def la_sym 2 "2", modify
label def la_sym 3 "3+", modify
label values delay la_delay
label def la_delay 0 "Less than 30 Days", modify
label def la_delay 1 "More than 30 days", modify

And here is how I want to use dtable:

dtable     2.gender i.age_cat i.race_ethnicity_sorted i.language 1.religion i.payor i.(coi_overall_cat coi_edu_cat coi_social_cat coi_health_cat)    ///
                i.type_embryo i.grade_cat i.num_sym 1.predispose_cat 1.outpatient, ///
                by(delay, tests)    ///
                nformat(%9.1fc p50 p25 p75) ///
                title(Table 1. Sample Characteristics) titlestyle(font(,size(10)))    ///
                notestyle(font(,size(6)))    //    Removed Notes section to account for the new note below.
collect style cell, font(,size(8))        
collect style header gender religion predispose_cat outpatient survival, level(hide)
collect style putdocx, layout(autofitc) halign(center)
collect preview

Note that this is a subset of the data, and with the entire dataset there are multiple tests that are significant -- hence my desire to account for multiple testing.

Thanks!

- Kevin

[Kevin Blaine]

Hey there Jeff Pitblado (StataCorp). Is this helpful?

[Jeff Pitblado (StataCorp)]

Do you want to adjust the p-values across all the variables in the table?

[Kevin Blaine]

That would be the idea; I know Bonferonni is the most conservative, but that's probably what I'd be using most the time.

[Kevin Blaine]

Hi Jeff Pitblado (StataCorp) -- just following up on this discussion. Thanks!

[Jeff Pitblado (StataCorp)]

Hi Kevin. I noted your request, thanks for the example and answer to my recent question.