Comparing 2 mean changes per year for statistical difference?

[Buyadaa Oyunchimeg]

Dear Stata users,

I have calculated mean changes in eGFR in 2 groups using mixed model.

Group 1 mean changes in eGFR per year =-1.29
Group 2 mean changes in eGFR per year =-1.86


Now I am trying to compare mean changes in eGFR of groups 1 and 2 for statistical significance.
Is there a reasonably straightforward way to calculate a p-value for this, from the data provided above.


Thank you so much.
Oyun

[Clyde Schechter]

Is there a reasonably straightforward way to calculate a p-value for this, from the data provided above.

No, there is no way, reasonable or unreasonable, to do this from the information provided. Mean values without any indication of variability cannot be contrasted using p-values.

[Buyadaa Oyunchimeg]

Thank you so much for prompt reply prof.Schechter.

Group 1 mean changes in eGFR per year =-1.29, 95% CI (-1.416996 -1.202826); n=2722
Group 2 mean changes in eGFR per year =-1.86, 95% CI (-1.9471403 -1.4189395); n=432

In this case, could we run test to check if the 2 groups are significant different from each other?


Thank you
Oyun

[Clyde Schechter]

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(mean ci_lb ci_ub n group)
-1.29  -1.416996  -1.202826 2722 1
-1.86 -1.9471403 -1.4189395  432 2
end

gen std_err = (ci_ub - ci_lb)/(2*invnormal(.975))
local df = n[1] + n[2] - 2
local pooled_se = sqrt(((n[1]-1)*std_err[1]^2 + (n[2]-1)*std_err[2]^2)/`df')
local t_stat = (mean[1] - mean[2])/(`pooled_se'*sqrt(1/n[1] + 1/n[2]))
local p = 2*t(`df', -abs(`t_stat'))

display "t = " %3.2f `t_stat' ", df = " `df'  ", p = " %05.3f `p'

Note: Your sample is very large; the group differences would have to be extremely small in order to not be statistically significant. I strongly urge you to ignore statistical significance and focus on whether the difference between the groups is clinically meaningful. https://www.nature.com/articles/d41586-019-00857-9.

[Buyadaa Oyunchimeg]

Dear prof.Schechter,

Thank you so much for codes and very helpful advices.


Oyun

[Buyadaa Oyunchimeg]

Hello Clyde Schechter

Thank you so much for your help given to me and others.
Using this old example now I am trying to calculate the difference between means with 95% CIs.

Group 1 mean changes in eGFR per year =-1.29, 95% CI (-1.416996 -1.202826); n=2722
Group 2 mean changes in eGFR per year =-1.86, 95% CI (-1.9471403 -1.4189395); n=432

I would greatly appreciate it if you may help me with this.

Many thanks,
Oyun

[Buyadaa Oyunchimeg]

Hello Clyde Schechter

Thank you so much for your help given to me and others.
Using this old example now I am trying to calculate the difference between means with 95% CIs.

Group 1 mean changes in eGFR per year =-1.29, 95% CI (-1.416996 -1.202826); n=2722
Group 2 mean changes in eGFR per year =-1.86, 95% CI (-1.9471403 -1.4189395); n=432

I would greatly appreciate it if you may help me with this.

Many thanks,
Oyun

[Buyadaa Oyunchimeg]

Sorry for the multiple emails. It looks like something went wrong with my connection.

[Clyde Schechter]

I'm assuming that the two samples are independent of each other.

The first task is to get the standard error of each of the separate means. As the 95% CI is calculated as mean +- invnormal(0.975)*se, we just take the difference between the upper and lower limits, and divide by 2*invnormal(0.975). (Actually, depending on where these means and CI's come from, the width of the CI may be calculated from a t-distribution rather than the normal. But your N's are sufficiently large that those t-distributions would be indistinguishable from the normal for practical purposes.)

Then the standard error of a sum or difference is the square root of the weighted (by N-1) sum of the squares of the individual standard errors when the two samples are independent. Then you use the standard error of the difference to calculate the confidence interval using the same formula

Overall, the calculations are only a minor variation on what was shown in #4.

scalar mean1 = -1.29
scalar se1 = (-1.202826 - (-1.416996))/(2*invnormal(0.975))
scalar n1 = 2722

scalar mean2 = -1.86
scalar se2 = (-1.4189395 - (-1.9471403))/(2*invnormal(0.975))
scalar n2 = 432

scalar mean_diff = mean2 - mean1
scalar se_diff = sqrt(((n1-1)*se1^2 + (n2-1)*se2^2)/(n1+n2))
scalar ci_lb_95 = mean_diff - invnormal(0.975)*se_diff
scalar ci_ub_95 = mean_diff + invnormal(0.975)*se_diff

display as text "Difference = " as result %3.2f =mean_diff ///
    as text ", 95% CI (" as result %3.2f =ci_lb_95 ", " %3.2f =ci_ub_95 ")"

[Buyadaa Oyunchimeg]

It works perfectly! As always, thank you so much for your help.