Item Response Theory - including age in the model to scale the latent variable

[Helena Hutchins]

Hello,

I am refining a 1P IRT analysis (using svyset data) to predict age of attainment groupings for child development milestones. I received feedback that it would be useful to include age (in years) in my model (such as in a SEM model) to predict the latent variable. The utility of this, would be that it would allow the IRT model to estimate median ages (in years), rather than parameters on a latent scale with an arbitrary mean.

I have gotten as far as generating the latent trait variable with an SEM model (see code below), but how do I tell Stata to employ this latent trait in the IRT command and/or corresponding ICCs (code below)?

I have referenced the following Sheldrick and Perrin, 2013 paper for methods: Evidence-based milestones for surveillance of cognitive, language, and motor development - PubMed (nih.gov)

SEM model code:

 svy linearized, subpop(if include == 1) : sem (Development@1 -> age, ) (Development -> q1, ) (Development -> q2, ) (Development -> q3, ).... (Development --> q12), trace latent(Development ) nocapslatent

IRT model code (without incorporating age in the model):

 svy, subpop(include): irt 1pl q1 q2 q3... q12
estat report, sort(b) byparm
irtgraph icc

Any input would be much appreciated.

Thank you!
Helena

[Weiwen Ng]

By my reading, your SEM code specifies what I think is a Multiple Indicators and Multiple Causes (MIMIC) model. You are using q1-q12 to predict a latent variable called Development. You are simultaneously fitting a linear regression with Development as the dependent variable and age as the sole predictor. I'm not super familiar with the question if adding covariates improves the prediction of the trait - normally I'm interested in how variables are associated with the trait, but what your interlocutors say makes sense.

You can write an IRT version of this model (it's a generalized SEM, some would call this an explanatory IRT model). However, I don't quite understand how your interlocutors get this:

The utility of this, would be that it would allow the IRT model to estimate median ages (in years), rather than parameters on a latent scale with an arbitrary mean.

I'm not a super-duper IRT expert, but I'm reasonably competent in it, and as far as I know, the IRT model you plan on fitting is estimating child development. It's not estimating median ages. They're asking you to put age in as a predictor.

Also, I don't quite get what the hang up with the arbitrary mean is. You're assuming that development has a standard normal distribution. So, mean is 0 and the SD is 1. That's arbitrary, but so would be converting the predicted values to T-scores (mean 50 SD 10, which I see used in some healthcare applications).

At this point, I have to confess that I am not at all familiar with 1 parameter IRT models. I know the discrimination parameter is constrained to be equal across all items. Here is how I'd write an explanatory IRT 2PL model:

gsem (Development -> q1-q12, logit) (age -> Development), variance(e.Development@1)

Note that you're constraining the error term of Development to be equal to 1. Why you do that isn't intuitively obvious to me, but I cross-checked my results with an R package that fits explanatory IRT models with a Stata gsem model written this way, and I got equivalent results. In a plain IRT model written in gsem, you'd just constrain the variance of Development to be 1.

I'm not sure how the syntax works for a 1PL model. However, when you fit a model with the irt command, it is really calling gsem behind the scenes. You can fit an IRT model and then type

di e(cmdline)

to get the full command. There's probably a list of constraints that the command issued, and I think you will ultimately need to copy those.

Then, another issue. In the SEM syntax you wrote, you have this bit of code:

sem (Development@1 -> age, ) ...

I don't know exactly what this does, but I have a feeling you're constraining the slope on age to be equal to 1. If that's the case, was that what you meant to do? I can't see a reason to do that, but I don't regularly fit MIMIC models.

[Helena Hutchins]

Thank you very much for this reply, Weiwen. This is all very helpful! I will look more into your MIMIC model suggestion.

Another question: do you know whether the code above for the MIMIC models you suggested can also be used to generate ICC curves? I know ICC graphs can be easily generated with a postestimation command following IRT.

I'll respond to each of your comments in more detail with some additional questions here:

1. "I'm not a super-duper IRT expert, but I'm reasonably competent in it, and as far as I know, the IRT model you plan on fitting is estimating child development. It's not estimating median ages. They're asking you to put age in as a predictor.

Also, I don't quite get what the hang up with the arbitrary mean is. You're assuming that development has a standard normal distribution. So, mean is 0 and the SD is 1."

The Sheldrick and Perrin, 2013 paper (see link below) provides an example of scaling the latent trait by age for child development items, and interpreting estimates as median "developmental" age. You can see in their fig. 2 the x-axis for the ICC curves is presented as "developmental age in months" from 0 - 60 months. I believe this is what our reviewer is asking us to do, as opposed to presenting the x-axis and corresponding difficulty estimates ranging from -2 to 2.

2. Note that you're constraining the error term of Development to be equal to 1. Why you do that isn't intuitively obvious to me....I don't know exactly what this does, but I have a feeling you're constraining the slope on age to be equal to 1. If that's the case, was that what you meant to do? I can't see a reason to do that, but I don't regularly fit MIMIC models.

In Sheldrick and Perrin's methods supplement, they provide an example of the SEM model they used to generate the latent variable and explain the following:
"The loading on chronological age was constrained to 1 with an intercept of 0. Thus, the latent variable shares the same units and scale as the natural logarithm of chronological age and can be conceptualized as the natural logarithm of developmental age." Can you suggest revisions to the code in order to better achieve this?

Thank you again!

Cited article: Sheldrick and Perrin (2013) Evidence-based milestones for surveillance of cognitive, language, and motor development - PubMed (nih.gov)

[Weiwen Ng]

Thank you very much for this reply, Weiwen. This is all very helpful! I will look more into your MIMIC model suggestion.

Another question: do you know whether the code above for the MIMIC models you suggested can also be used to generate ICC curves? I know ICC graphs can be easily generated with a postestimation command following IRT.

I'll respond to each of your comments in more detail with some additional questions here:

1. "I'm not a super-duper IRT expert, but I'm reasonably competent in it, and as far as I know, the IRT model you plan on fitting is estimating child development. It's not estimating median ages. They're asking you to put age in as a predictor.

Also, I don't quite get what the hang up with the arbitrary mean is. You're assuming that development has a standard normal distribution. So, mean is 0 and the SD is 1."

The Sheldrick and Perrin, 2013 paper (see link below) provides an example of scaling the latent trait by age for child development items, and interpreting estimates as median "developmental" age. You can see in their fig. 2 the x-axis for the ICC curves is presented as "developmental age in months" from 0 - 60 months. I believe this is what our reviewer is asking us to do, as opposed to presenting the x-axis and corresponding difficulty estimates ranging from -2 to 2.

2. Note that you're constraining the error term of Development to be equal to 1. Why you do that isn't intuitively obvious to me....I don't know exactly what this does, but I have a feeling you're constraining the slope on age to be equal to 1. If that's the case, was that what you meant to do? I can't see a reason to do that, but I don't regularly fit MIMIC models.

In Sheldrick and Perrin's methods supplement, they provide an example of the SEM model they used to generate the latent variable and explain the following:
"The loading on chronological age was constrained to 1 with an intercept of 0. Thus, the latent variable shares the same units and scale as the natural logarithm of chronological age and can be conceptualized as the natural logarithm of developmental age." Can you suggest revisions to the code in order to better achieve this?

Thank you again!

Cited article: Sheldrick and Perrin (2013) Evidence-based milestones for surveillance of cognitive, language, and motor development - PubMed (nih.gov)

Thank you for providing a source. I maintain that in most instances, the objection that the latent variable's scale is arbitrary is spurious. But this appears to be an exception: in child development (which is totally not my field, so I didn't realize this), there's the concept of developmental age. If the latent variable is on the standard normal scale, 0 is average development. The problem in this case is that it's the average of all the kids in your sample, who presumably span quite an age range. So, you really would want to be able to express milestone attainment in terms of developmental age. This becomes apparent when we read through the appendices to the article.

And we can infer from the figure in the appendix that this is what they did: they're using the developmental questions as indicators of a latent variable called ln(Developmental age). And ln(age) is a covariate - basically, you simultaneously regress ln(age) on the latent variable. Only they fixed the loading to 1, so the latent variable has the same units as the observed variable. So, again, this now makes sense to me. I was getting cranky at this reviewer, and it turns out that this wasn't justified in this case.

The thing is, they used both Stata 12 and MPlus for this analysis. I believe that Stata 12 lacked the generalized SEM command, and if so, the analysis would have been done in MPlus. I believe that at least part of this can be replicated manually in Stata's gsem command. It requires you to know what you are doing behind the scenes - it's possible to manually generate an item or test characteristic curve, for example. Normally, you don't need to as it's prepackaged in the irt set of commands. Those really just plot probability (or whatever else you want) against Theta, but here, you want the x-axis to be age.

One difficulty here is that this paper log transformed age. You would have to manually transform and un-transform things. In the gsem syntax, you could specify the relationship of the latent variable to its predictor as a generalized linear model with, say, a log link, but I don't think that's the same as what they did here. In a GLM, you'd be saying that ln(mean developmental age) = f(x). Again, not quite the same thing - and also we lack a means to automatically transform the predictor. So, if you went down this route, it would probably be better not to take natural logs, and this could be an issue. I don't know the literature well enough to comment.

[Weiwen Ng]

Actually, examining the figure in their appendix, I am wrong. Calendar age is included in the IRT model as an indicator, but its loading was constrained to 1 and the intercept to 0 - if I read their technical appendix correctly. I left this unanswered because, unfortunately, there's a complication I can't really overcome.

Back to the IRT command. When you ask for an item characteristic curve, you're getting the probability of responding in each level of the specified indicator on the y axis, and the level of the latent trait (in SDs around the mean), i.e. in a 1PL model the probability of responding 0 or 1 as the latent trait goes from -4 to +4 (the default setting, can be changed). Behind the scenes, Stata is creating a dataset with theta taking on values of -4 to +4 with 300 total observations (again, the default setting). From the difficulty parameter(s) for each item and the slope (which you could have constrained to 1 if you were fitting a Rasch model), you know the probability of someone responding a 0 or a 1. It's really just the inverse logit of a{(theta_j - b_i}, where j indexes persons (or observations in the fictional dataset) and i indexes items/questions. It does this behind the scenes. If, for some reason, you are still on Stata 13, you could do this manually. After that, you or Stata can just plot the ICC: literally just twoway line theta p1 p2 or whatever the variables are named.

The original question, however, calls for you to fit a generalized structural equation model where one part is an IRT model and the other part is a linear model with the aforementioned constraints. You will need to manually define a series of constraints that tell the model that the discrimination parameters for all questions are equal (if you want a 1PL model) or 1 (if you want a Rasch model). The initial command should look like:

svy, subpop(include): gsem (Development -> q1-q12, logit) (Development -> age), variance(Development@1) latent(Development) constraints(...)

From there, I'm not sure exactly how to proceed. My interpretation is that if you left the parameters for the effect of age on Development unconstrained, the equivalent age should be intercept + loading * theta. If you constrained the loading to 0 and the intercept to 1, then it seems like the effect of age will be in the same units as theta - basically, you would be left with age ranging from -4 to 4. It seems like this would do nothing. I don't have access to real data, so I can't really try this out. Does anyone have a better idea?