'Not concave' message when running IRT 2PL model

[Mariana Simoes]

Hello,

I am new to IRT and I am playing around with a dataset that I collected during my pilot study for my doctoral studies using a math test to assess young children. The dataset is small (n=83), the test has 36 items and answers are coded as 0/1. I understand that my sample size is very small, so I am using CTT (p level and item-test correlations) as my primary item analysis for this dataset. However, I want to check if the IRT analysis will give me similar results. My goal is to decide which test items to keep and to drop for my main study later this year.

I am using Stata/IC 15.1. I am learning about Stata's IRT commands from the Stata's Reference Manual - Release 15.

I ran the 1PL model just fine. But when I try to run the 2PL, the 'fitting full model' keeps running, and I get the 'not concave' message over and over again, until I stop it. The same happens when I select just a few items and try to run the model. Here it is the command that I ran: irt 2pl $qrt

What are the possible reasons for not being able to run the 2PL model with my dataset?

Thank you,
Mariana

[Carlo Lazzaro]

Mariana:
welcome to this forum.
The coding of your variables might be the problem: if only 0/1 values are available, the maximum likelihood machinery probably chokes on them.

[Mariana Simoes]

Hi Carlo,

Many thanks for the swift reply!
My understanding is that IRT can be used with binary data, so I didn't think that coding answers with 0/1 would be a problem. Would you have any suggestions on how I can address this?

Many thanks,
Mariana

[Carlo Lazzaro]

Mariana:
add one variable at time and see when Stata starts choking on.

[Weiwen Ng]

If Stata keeps iterating, but the log likelihood doesn't change and the log keeps saying "not concave", that's often because the model is not identified. Not identified means there isn't enough information to iterate to a unique solution. You have 83 observations. With the 1PL model, you are estimating a difficulty parameter for each item. With the 2PL model, you are estimating one difficulty parameter plus one discrimination parameter per item.

I am not an expert on model identification in SEM. But I suspect you don't have enough observations to identify a 36-question, 2PL model, with 83 people.

[wbuchanan]

It almost definitely is an identification issue. However, I’d also suggest that you evaluate whether or not the unidimensionality assumption is reasonable in this case as well. If you believe the items are all measuring a single dimension, you could try using the difficult option. In either case, I would not put much faith in the final results given the sample size and number of items. If, however, the items measure multiple dimensions you may want to consider fitting a separate IRT model for each distinct dimension (assuming that no individual items measure multiple dimensions simultaneously).