Wald Test to Compare Two Models

[Anna Cherry]

I am running a logistic regression with the outcome (ser) and a number of predictor variables,
with standard errors clustered (by provider_id). One of the predictors (years_cat) is a categorical
variable that I created from a continuous variable (tenure); in other words, years_cat is categories
of tenure. The odds ratios on the categories of years_cat are monotonically increasing and I would
like to test if the variable tenure is in fact linear (though I will be reporting the categorical
variable in my paper).

Usually, I would run the model with the categorical variable (cat1), store the estimates, and then
run the model with the linear variable (lin1), and store the estimates, and then use a likelihood ratio test to compare
the two (lrtest cat1 lin1). Since I have clustered standard errors, the likelihood ratio test is not
appropriate and I would like to use a Wald test. However, when I run the wald test, I am getting errors (see details below) that Stata cannot find the models that I've stored (despite confirming that they are in fact stored). How can I run the wald test to compare the equality of these two stored models?

*Run the model with categorical variable (years_cat), store.
logistic ser i.age_ra i.sex ib5.indication ib2.gender_doc ib1.specialty1 i.vol ib4.years_cat, cluster(provider_id)
    estimates store cat1

*Run the model with continuous variable (tenure), store.
logistic ser i.age_ra i.sex ib5.indication ib2.gender_doc ib1.specialty1 i.vol tenure, cluster(provider_id)
    estimates store lin1
    
*Confirm that stata has stored the estimates    
estimates table cat1 lin1
 
*Run wald test    
test lin1 cat1
    *returns error "r(111) lin1 not found"

*Run wald test with alternative syntax
test [lin1 cat1]
    *returns error "r(303) equation lin1 not found"

/*The same errors are reutrned if I put an = between lin1 and cat1. If I put cat1 before lin1,
the error tells me instead taht cat1 is not found.*/

*I've also tried using the suest command, but I get the same errors
logistic ser i.age_ra i.sex ib5.indication ib2.gender_doc ib1.specialty1 i.vol ib4.years_cat
    estimates store cat2

logistic ser i.age_ra i.sex ib5.indication ib2.gender_doc ib1.specialty1 tenure i.vol ib4.years_cat
    estimates store lin2

suest cat2 lin2, cluster(provider_id)

test lin2 cat2
    *Error: r(111) lin2 not found

[William Lisowski]

Cross-posted on Stack Overflow at https://stackoverflow.com/questions/...odels-in-stata

[Maarten Buis]

A Wald test is not a test forcomparing models, but one for constraints on a single model. The error message you got was not that test could not find the model lin1, but that test could not find the variable lin1 in currently active model.

If you want to use a Wald test you need to rephrase your linear model as a constrained on the categorical model. So in the example below 5.rep78 is two "steps" above the reference category. If the effect were linear, then it has to be equal to twice a single step from the reference category, so twice 4.rep78. If there were a 6.rep78, then would be three steps above the reference category, it should be equal to three times 4.rep78, etc.

sysuse auto, clear
recode rep78 (1/3=3)
logit foreign i.rep78
test 2*5.rep78=4.rep78

[William Lisowski]

The specific reason you get the errors described in post #1 is that you are trying to apply the syntax for the lrtest command when using the test command. That won't work, the two commands have a much different syntax, as befits the different objectives, per Martin's post.

[Anna Cherry]

Thanks very much, this clears up my confusion!

[Ali Balapour]

A Wald test is not a test forcomparing models, but one for constraints on a single model. The error message you got was not that test could not find the model lin1, but that test could not find the variable lin1 in currently active model.

If you want to use a Wald test you need to rephrase your linear model as a constrained on the categorical model. So in the example below 5.rep78 is two "steps" above the reference category. If the effect were linear, then it has to be equal to twice a single step from the reference category, so twice 4.rep78. If there were a 6.rep78, then would be three steps above the reference category, it should be equal to three times 4.rep78, etc.

sysuse auto, clear
recode rep78 (1/3=3)
logit foreign i.rep78
test 2*5.rep78=4.rep78

Hi,

I have a same issue but in a different context.

I have two samples of survey data and I am running similar SEM models on each sample. I want to compare the beta coefficients achieved from each model.
e.g.:

model 1 (Sample size 554):
sem ( AW ->x1-x3) (AR -> y1-y3) (AZ -> z1-z3) (AW AR -> AZ) Covstruct (_lexogenous, unstructured) standardized latent (AW AZ AR) cov(e.x1*e.y2 e.x3*e.y1) nocapslatent

model 2 (Sample size 480):
sem ( AW ->x1-x3) (AR -> y1-y3) (AZ -> z1-z3) (AW AR -> AZ) Covstruct (_lexogenous, unstructured) standardized latent (AW AZ AR) cov(e.z1*e.z3) nocapslatent

I want to compare coefficients of AW -> AZ and AR -> AZ for both models. Also, I cannot use group because models have different "cov()" commands and the goodness of for each will be low without adding the specified "cov()" for each model.

I do not want to compare models! I just want to compare the standardized or unstandardized beta coefficient for AW->AZ for model 1 and model 2. Is the beta coefficient for AW-AZ of model 1 statistically different from model 2?

Please advise.