Risk difference (non-inferiority) for a proportion adjusting for predictors

[Megan Moreton]

Hello!

I am trying to assess the non-inferiority of one group against another using confidence intervals. To compute the risk difference & CI of the proportion with success between the groups I have been using prtest:

prtest success, by(group)

I now want to extend the model adjusting for various covariates. As a basic model, I have tried used a logit model (followed by margins) or a glm with the identity link:

logit success i.group
margins, dydx(group)

glm success i.group, fam(bin) link(identity)

Both models give me the same result & 95%CI as the prtest command.

If I then add covariates, such as c.age, i.gender, etc - the RD & the 95%CI differ if I generate this immediately using the glm model or using logit followed by margins.

logit success i.group c.age i.gender
margins, dydx(group)

glm success i.group c.age i.gender, fam(bin) link(identity)

Apologies for my ignorance, but could someone help explain this difference to me? Is it something to do with the covariate values at which the margins are estimated?


Thank you for any help!

[Joseph Coveney]

How different are they? The two models that you're fitting don't have the same likelihood function and so won't have the exact same fit (in particular, age is linear in the latter but nonlinear in the former), but their predictions ought to be reasonably close, except if some or all of the predictions from your latter model lie outside the parameter space. (It's also possible that gross imbalances in the covariates between the two levels of the group variable magnify the discrepancy when using default covariate-adjustment settings for -margins-.)

You can see below (begin at the "Begin here" comment) that when the predictions lie toward the center of the parameter space (between, say, about 20% to 80%), -margins- gives you pretty close results.

.ÿ
.ÿversionÿ17.0

.ÿ
.ÿclearÿ*

.ÿ
.ÿsetÿseedÿ`=strreverse("1628939")'

.ÿ
.ÿquietlyÿsetÿobsÿ250

.ÿ
.ÿgenerateÿbyteÿgrpÿ=ÿmod(_n,ÿ2)

.ÿgenerateÿdoubleÿageÿ=ÿruniform(0,ÿ100)

.ÿisidÿage

.ÿbysortÿgrpÿ(age):ÿgenerateÿbyteÿsexÿ=ÿmod(_n,ÿ2)

.ÿ
.ÿgenerateÿdoubleÿxbÿ=ÿ0ÿ+ÿ(grpÿ-ÿ0.5)ÿ/ÿ1.25ÿ+ÿ(ageÿ-ÿ50)ÿ/ÿ250ÿ+ÿ(sexÿ-ÿ0.5)ÿ/ÿ1.25

.ÿ
.ÿgenerateÿbyteÿoutÿ=ÿrbinomial(1,ÿinvlogit(xb))

.ÿ
.ÿ*
.ÿ*ÿBeginÿhere
.ÿ*
.ÿlogitÿoutÿi.grpÿc.ageÿi.sex,ÿnolog

LogisticÿregressionÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿ=ÿÿÿÿ250
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿLRÿchi2(3)ÿÿÿÿ=ÿÿ26.10
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿProbÿ>ÿchi2ÿÿÿ=ÿ0.0000
Logÿlikelihoodÿ=ÿ-160.11119ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿPseudoÿR2ÿÿÿÿÿ=ÿ0.0754

------------------------------------------------------------------------------
ÿÿÿÿÿÿÿÿÿoutÿ|ÿCoefficientÿÿStd.ÿerr.ÿÿÿÿÿÿzÿÿÿÿP>|z|ÿÿÿÿÿ[95%ÿconf.ÿinterval]
-------------+----------------------------------------------------------------
ÿÿÿÿÿÿÿ1.grpÿ|ÿÿÿ.7850733ÿÿÿÿ.268984ÿÿÿÿÿ2.92ÿÿÿ0.004ÿÿÿÿÿ.2578744ÿÿÿÿ1.312272
ÿÿÿÿÿÿÿÿÿageÿ|ÿÿÿ.0103562ÿÿÿ.0048397ÿÿÿÿÿ2.14ÿÿÿ0.032ÿÿÿÿÿ.0008705ÿÿÿÿ.0198418
ÿÿÿÿÿÿÿ1.sexÿ|ÿÿÿ.8995928ÿÿÿ.2684202ÿÿÿÿÿ3.35ÿÿÿ0.001ÿÿÿÿÿ.3734988ÿÿÿÿ1.425687
ÿÿÿÿÿÿÿ_consÿ|ÿÿ-1.287319ÿÿÿÿ.333772ÿÿÿÿ-3.86ÿÿÿ0.000ÿÿÿÿÿÿ-1.9415ÿÿÿ-.6331376
------------------------------------------------------------------------------

.ÿmarginsÿ,ÿdydx(grp)

AverageÿmarginalÿeffectsÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿ=ÿ250
ModelÿVCE:ÿOIM

Expression:ÿPr(out),ÿpredict()
dy/dxÿwrt:ÿÿ1.grp

------------------------------------------------------------------------------
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|ÿÿÿÿÿÿÿÿÿÿÿÿDelta-method
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|ÿÿÿÿÿÿdy/dxÿÿÿstd.ÿerr.ÿÿÿÿÿÿzÿÿÿÿP>|z|ÿÿÿÿÿ[95%ÿconf.ÿinterval]
-------------+----------------------------------------------------------------
ÿÿÿÿÿÿÿ1.grpÿ|ÿÿÿ.1812926ÿÿÿ.0609568ÿÿÿÿÿ2.97ÿÿÿ0.003ÿÿÿÿÿ.0618194ÿÿÿÿ.3007657
------------------------------------------------------------------------------
Note:ÿdy/dxÿforÿfactorÿlevelsÿisÿtheÿdiscreteÿchangeÿfromÿtheÿbaseÿlevel.

.ÿpredictÿdoubleÿprl,ÿpr

.ÿ
.ÿglmÿoutÿi.grpÿc.ageÿi.sex,ÿfamily(binomial)ÿlink(identity)ÿnolog

GeneralizedÿlinearÿmodelsÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿÿÿ=ÿÿÿÿÿÿÿÿ250
Optimizationÿÿÿÿÿ:ÿMLÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿResidualÿdfÿÿÿÿÿ=ÿÿÿÿÿÿÿÿ246
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿScaleÿparameterÿ=ÿÿÿÿÿÿÿÿÿÿ1
Devianceÿÿÿÿÿÿÿÿÿ=ÿÿ319.2647994ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ(1/df)ÿDevianceÿ=ÿÿÿ1.297824
Pearsonÿÿÿÿÿÿÿÿÿÿ=ÿÿÿÿÿÿÿÿÿÿ250ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ(1/df)ÿPearsonÿÿ=ÿÿÿÿ1.01626

Varianceÿfunction:ÿV(u)ÿ=ÿu*(1-u)ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ[Bernoulli]
Linkÿfunctionÿÿÿÿ:ÿg(u)ÿ=ÿuÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ[Identity]

ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿAICÿÿÿÿÿÿÿÿÿÿÿÿÿ=ÿÿÿ1.309059
Logÿlikelihoodÿÿÿ=ÿ-159.6323997ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿBICÿÿÿÿÿÿÿÿÿÿÿÿÿ=ÿÿ-1039.015

------------------------------------------------------------------------------
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿOIM
ÿÿÿÿÿÿÿÿÿoutÿ|ÿCoefficientÿÿstd.ÿerr.ÿÿÿÿÿÿzÿÿÿÿP>|z|ÿÿÿÿÿ[95%ÿconf.ÿinterval]
-------------+----------------------------------------------------------------
ÿÿÿÿÿÿÿ1.grpÿ|ÿÿÿ.1822772ÿÿÿ.0607828ÿÿÿÿÿ3.00ÿÿÿ0.003ÿÿÿÿÿ.0631451ÿÿÿÿ.3014092
ÿÿÿÿÿÿÿÿÿageÿ|ÿÿÿ.0027051ÿÿÿ.0011377ÿÿÿÿÿ2.38ÿÿÿ0.017ÿÿÿÿÿ.0004752ÿÿÿÿ.0049351
ÿÿÿÿÿÿÿ1.sexÿ|ÿÿÿ.2160611ÿÿÿ.0597105ÿÿÿÿÿ3.62ÿÿÿ0.000ÿÿÿÿÿ.0990306ÿÿÿÿ.3330915
ÿÿÿÿÿÿÿ_consÿ|ÿÿÿ.1909086ÿÿÿ.0690474ÿÿÿÿÿ2.76ÿÿÿ0.006ÿÿÿÿÿ.0555781ÿÿÿÿ.3262392
------------------------------------------------------------------------------
Coefficientsÿareÿtheÿriskÿdifferences.

.ÿmarginsÿ,ÿdydx(grp)

AverageÿmarginalÿeffectsÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿ=ÿ250
ModelÿVCE:ÿOIM

Expression:ÿPredictedÿmeanÿout,ÿpredict()
dy/dxÿwrt:ÿÿ1.grp

------------------------------------------------------------------------------
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|ÿÿÿÿÿÿÿÿÿÿÿÿDelta-method
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|ÿÿÿÿÿÿdy/dxÿÿÿstd.ÿerr.ÿÿÿÿÿÿzÿÿÿÿP>|z|ÿÿÿÿÿ[95%ÿconf.ÿinterval]
-------------+----------------------------------------------------------------
ÿÿÿÿÿÿÿ1.grpÿ|ÿÿÿ.1822772ÿÿÿ.0607828ÿÿÿÿÿ3.00ÿÿÿ0.003ÿÿÿÿÿ.0631451ÿÿÿÿ.3014092
------------------------------------------------------------------------------
Note:ÿdy/dxÿforÿfactorÿlevelsÿisÿtheÿdiscreteÿchangeÿfromÿtheÿbaseÿlevel.

.ÿpredictÿdoubleÿpri,ÿmu

.ÿ
.ÿgenerateÿdoubleÿdelÿ=ÿpriÿ-ÿprl

.ÿsummarizeÿpr?ÿdel

ÿÿÿÿVariableÿ|ÿÿÿÿÿÿÿÿObsÿÿÿÿÿÿÿÿMeanÿÿÿÿStd.ÿdev.ÿÿÿÿÿÿÿMinÿÿÿÿÿÿÿÿMax
-------------+---------------------------------------------------------
ÿÿÿÿÿÿÿÿÿprlÿ|ÿÿÿÿÿÿÿÿ250ÿÿÿÿÿÿÿÿ.516ÿÿÿÿ.1586949ÿÿÿ.2187502ÿÿÿ.8068338
ÿÿÿÿÿÿÿÿÿpriÿ|ÿÿÿÿÿÿÿÿ250ÿÿÿÿ.5246681ÿÿÿÿ.1667785ÿÿÿ.1946581ÿÿÿ.8588739
ÿÿÿÿÿÿÿÿÿdelÿ|ÿÿÿÿÿÿÿÿ250ÿÿÿÿ.0086681ÿÿÿÿ.0127338ÿÿ-.0240921ÿÿÿ.0520401

.ÿ
.ÿexit

endÿofÿdo-file


.