ANCOVA - planned contrast tests code

[Jiahui Lu]

Hello.

I fit the following ANCOVA model and want to conduct some planned contrasts tests within factor paycondition:

anova wbiam i.paycondition c.sdbam i.paycondition#c.sdbam

paycondition has 3 groups. I used the following code to compare within the groups right after running the above anova code.
contrast r.paycondition
contrast ar.paycondition

However, I also found the following code to do planned contrasts, for the contrast comparing group1 and group2:
anovacontrast paycondition, values (1 -1 0)

The two types of code give me different p-values and test statistics. I'm not sure which way to rely on.

Thanks for your help.

[Joseph Coveney]

The two types of code give me different p-values and test statistics. I'm not sure which way to rely on.

Maybe neither?

Your ANOVA model includes an interaction term. I think that contrast as you show it just decomposes the main-effects term of your categorical predictor, that is, it doesn't take into account that the mean of the outcome at each category depends upon the values of the continuous predictor.

And it seems that the user-written command anovacontrast gives a one-degree-of-freedom contrast for a particular pair of categories of your categorical predictor. I'm not sure how it handles the presence of an interaction term in the model.

Given that you believe the paycondition × sdbam interaction is important enough to include in your ANOVA model, wouldn't it be better to examine the contrast at interesting (or at least representative) values of your continuous covariate? Maybe something like that illustrated below (begin at the "Begin here" comment; the code above just creates a toy dataset for use in illustration), where I perform contrasts of the categorical predictor at the minimum, mean and maximum observed values of the continuous covariate.

Perhaps as important as test statistics and p-values is a so-called profile plot after fitting such an ANOVA model. And (at the risk of clutter) the contrast p-values can be overlaid onto the profile plot for a combined view, also illustrated below.

.ÿ
.ÿversionÿ17.0

.ÿ
.ÿclearÿ*

.ÿ
.ÿ//ÿseedem
.ÿsetÿseedÿ88631271

.ÿ
.ÿquietlyÿsetÿobsÿ21

.ÿ
.ÿgenerateÿbyteÿpayÿ=ÿmod(_n,ÿ3)

.ÿgenerateÿdoubleÿsdbÿ=ÿruniform(-0.5,ÿ0.5)

.ÿgenerateÿdoubleÿxbÿ=ÿ!payÿ/ÿ1.5ÿ+ÿsdbÿ/ÿ30ÿ+ÿ!payÿ*ÿsdb

.ÿ
.ÿgenerateÿdoubleÿwbiÿ=ÿrnormal(xb,ÿsqrt(0.15))

.ÿ
.ÿanovaÿwbiÿpay##c.sdb

ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿ=ÿÿÿÿÿÿÿÿÿ21ÿÿÿÿR-squaredÿÿÿÿÿ=ÿÿ0.5990
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿRootÿMSEÿÿÿÿÿÿ=ÿÿÿÿ.278841ÿÿÿÿAdjÿR-squaredÿ=ÿÿ0.4653

ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿSourceÿ|ÿPartialÿSSÿÿÿÿÿÿÿÿÿdfÿÿÿÿÿÿÿÿÿMSÿÿÿÿÿÿÿÿFÿÿÿÿProb>F
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ-----------+----------------------------------------------------
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿModelÿ|ÿÿ1.7421663ÿÿÿÿÿÿÿÿÿÿ5ÿÿÿ.34843325ÿÿÿÿÿÿ4.48ÿÿ0.0107
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿpayÿ|ÿÿ.75145743ÿÿÿÿÿÿÿÿÿÿ2ÿÿÿ.37572872ÿÿÿÿÿÿ4.83ÿÿ0.0240
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿsdbÿ|ÿÿ.62134827ÿÿÿÿÿÿÿÿÿÿ1ÿÿÿ.62134827ÿÿÿÿÿÿ7.99ÿÿ0.0127
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿpay#sdbÿ|ÿÿ.63712668ÿÿÿÿÿÿÿÿÿÿ2ÿÿÿ.31856334ÿÿÿÿÿÿ4.10ÿÿ0.0380
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿResidualÿ|ÿÿ1.1662884ÿÿÿÿÿÿÿÿÿ15ÿÿÿ.07775256ÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ-----------+----------------------------------------------------
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿTotalÿ|ÿÿ2.9084547ÿÿÿÿÿÿÿÿÿ20ÿÿÿ.14542273ÿÿ

.ÿ
.ÿ*
.ÿ*ÿBeginÿhere
.ÿ*
.ÿ//ÿContrastsÿatÿminimum,ÿmeanÿandÿmaximumÿofÿcontinuousÿcovariate
.ÿsummarizeÿsdb,ÿmeanonly

.ÿlocalÿminÿ`r(min)'

.ÿlocalÿmaxÿ`r(max)'

.ÿlocalÿmeaÿ`r(mean)'

.ÿ
.ÿmarginsÿpay,ÿat(sdbÿ=(`min'ÿ`mea'ÿ`max'))ÿcontrast

ContrastsÿofÿadjustedÿpredictionsÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿNumberÿofÿobsÿ=ÿ21

Expression:ÿLinearÿprediction,ÿpredict()
1._at:ÿsdbÿ=ÿ-.4974601
2._at:ÿsdbÿ=ÿ-.1737856
3._at:ÿsdbÿ=ÿÿ.4550488

------------------------------------------------
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|ÿÿÿÿÿÿÿÿÿdfÿÿÿÿÿÿÿÿÿÿÿFÿÿÿÿÿÿÿÿP>F
-------------+----------------------------------
ÿÿÿÿÿpay@_atÿ|
ÿÿÿÿÿÿÿÿÿÿ1ÿÿ|ÿÿÿÿÿÿÿÿÿÿ2ÿÿÿÿÿÿÿÿ1.50ÿÿÿÿÿ0.2557
ÿÿÿÿÿÿÿÿÿÿ2ÿÿ|ÿÿÿÿÿÿÿÿÿÿ2ÿÿÿÿÿÿÿÿ3.32ÿÿÿÿÿ0.0641
ÿÿÿÿÿÿÿÿÿÿ3ÿÿ|ÿÿÿÿÿÿÿÿÿÿ2ÿÿÿÿÿÿÿÿ5.39ÿÿÿÿÿ0.0172
ÿÿÿÿÿÿJointÿÿ|ÿÿÿÿÿÿÿÿÿÿ4ÿÿÿÿÿÿÿÿ4.06ÿÿÿÿÿ0.0199
ÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿDenominatorÿ|ÿÿÿÿÿÿÿÿÿ15
------------------------------------------------

.ÿ
.ÿ//ÿProfileÿplotÿofÿpredictedÿoutcomesÿatÿsameÿvaluesÿofÿcontinuousÿcovariate
.ÿquietlyÿmarginsÿpay,ÿat(sdbÿ=(`min'ÿ`mea'ÿ`max'))

.ÿlocalÿoptsÿmcolor(black)ÿmfcolor(white)ÿ/*ÿmsize(small)ÿ*/ÿlcolor(black)

.ÿlocalÿsizeÿsize(medsmall)

.ÿmarginsplotÿ,ÿ///
>ÿÿÿÿÿÿÿÿÿnociÿtitle("")ÿ///
>ÿÿÿÿÿÿÿÿÿxdimension(sdb)ÿ///
>ÿÿÿÿÿÿÿÿÿplotdimension(pay,ÿelabels(1ÿ"Payÿ0"ÿ2ÿ"Payÿ1"ÿ3ÿ"Payÿ2"))ÿ///
>ÿÿÿÿÿÿÿÿÿplot1opts(msymbol(O)ÿ`opts')ÿ///
>ÿÿÿÿÿÿÿÿÿplot2opts(msymbol(T)ÿ`opts')ÿÿ///
>ÿÿÿÿÿÿÿÿÿplot3opts(msymbol(S)ÿ`opts')ÿÿ///
>ÿÿÿÿÿÿÿÿÿxtitle(SDB)ÿxlabel(ÿ,ÿformat(%03.1f))ÿ///
>ÿÿÿÿÿÿÿÿÿytitle(PredictedÿWBI)ÿylabel(ÿ,ÿformat(%03.1f)ÿangle(horizontal)ÿnogrid)ÿ///
>ÿÿÿÿÿÿÿÿÿtext(0.3ÿ-0.5ÿ"Pÿ=ÿ0.3",ÿ`size'ÿplacement(e))ÿ///
>ÿÿÿÿÿÿÿÿÿtext(-0.2ÿ`mea'ÿ"Pÿ=ÿ0.06",ÿ`size'ÿplacement(c))ÿ///
>ÿÿÿÿÿÿÿÿÿtext(0ÿ0.47ÿ"Pÿ=ÿ0.02",ÿ`size'ÿplacement(w))ÿ///
>ÿÿÿÿÿÿÿÿÿlegend(position(11)ÿring(0))

Variablesÿthatÿuniquelyÿidentifyÿmargins:ÿsdbÿpay

.ÿquietlyÿgraphÿexportÿProfile.png

.ÿ
.ÿexit

endÿofÿdo-file


.

[Jiahui Lu]

Hi Joseph, thank for taking your time on my question. Yes, I will take this in consideration when I switch to this project. Right now, I and my coauthor are working on the project related to my most recent post you just helped me with! I also think my recent post would help with my question in this post.