test "dose-response" with categorical variable

[Katrina Blindow]

Hi! I am performing cox regressions with a categorical indep. variable. In the survey, we have a likert-scale with 7 responses from "never" "once" ... to "daily". We coded 1) never 2) once 3) reoccurring. I want to know if (3) reoccurring exposure has a stronger association with the outcome than exposure (2) "once". My supervisor suggested treating the indep. variable as continuous (put it in stata without the i.) and claims that getting a stat. sign. p-value would prove that there is a trend. I find this hard to believe, it has been questioned by a reviewer and I can't find this described anywhere. I suggested instead to compare the two coefficients with a Wald test for stat. sign. of "not being the same". I would be greatefull for a) an explanation that supports my supervisors suggestion (possibly literature I can reference) b) support for the Wald test or c) an alternative to test this. Just to say it in advance: Including the variable with finer categories is off the table due to low power.

[Joseph Coveney]

I want to know if (3) reoccurring exposure has a stronger association with the outcome than exposure (2) "once". My supervisor suggested treating the indep. variable as continuous (put it in stata without the i.) and claims that getting a stat. sign. p-value would prove that there is a trend. I find this hard to believe . . . I suggested instead to compare the two coefficients with a Wald test for stat. sign. of "not being the same".

You can show the output of the code below to your supervisor in support of your skepticism. (I've attached the output as a SMCL log-file below.)

It shows that you can get a statistically significant p-value proving that there indeed is s trend, and yet the two categories are not at all different in their association with the outcome. It includes a graphical display (also attached ) of how this can obtain.

version 17.0

log using TrendbutNoDifference.smcl, nomsg name(lo)

clear *

// seedem
set seed 2027836111

quietly set obs 21

generate byte cat = mod(_n, 3) + 1
label define Frequency 1 Never 2 Once 3 Recurring
label values cat Frequency

generate double out = rnormal(2 * (cat > 1), 1)

*
* Begin here
*

// There is a statistically significant linear trend (slope is ca. 1)
regress out c.cat

// But the third category is no *more* associated with the outcome than the second
// (both "Once" and "Recurring" are ca. 2 away from "Never")
regress out i.cat
// And the test of their difference yields a test statistic of zero, with p-value ca. 1.0
test 3.cat = 2.cat

// Indeed, their estimates (see categories 2 and 3) are essentially identical
margins cat

/* To see why graphically that your supervisor's idea is erroneous: the continuous
   trend line (solid line) is overlaid on the plot with the individual categories'
   predictions.  You can get a linear trend; nevertheless, categories 2 and 3 are
   identical. */
marginsplot , noci plotopts(msize(medium) mcolor(black)) ///
    recast(scatter) title("") ///
        addplot(lfit out cat, lcolor(black)) ///
            xtitle(Frequency) xlabel(#3, valuelabel angle(45)) ///
            ytitle(Outcome) ylabel(0(0.5)1.5, format(%03.1f) ///
                angle(horizontal) nogrid) legend(off)

quietly graph export TrendbutNoDifference.png

quietly log close lo

exit

[Katrina Blindow]

Thank you, Joseph. This is great and illustrates exactly what I was suspecting.

You write: “And the test of their difference yields a test statistic of zero, with p-value ca. 1.0 test 3.cat = 2.cat” So, what is the appropriate test? Is it a wald test? I read that it shouldn’t be used when one has low power, also we have very different distributions in the three categories (twice as many “once” than reoccurring “). I wasn’t sure if the test would be reliable under these circumstances.

[Joseph Coveney]

So, what is the appropriate test? Is it a wald test?

I used a Wald test above for illustration of the point, but that wasn't meant to advocate any particular test method.

I read that it shouldn’t be used when one has low power . . . I wasn’t sure if the test would be reliable under these circumstances.

I'd be reluctant to do any NHST if I know that I have inadequate power.

[Katrina Blindow]

Great, thanks. Fully agree! And amazing to get such a quick response. 👍🏻