3PL IRT (Item Response Theory) in Stata

[Freddy DD]

Hello Friends,
When we use Stata to run IRT models, it sometimes cannot generate the output for IRT models, saying "not concave". For example, when I used Stata to run 2PL model, it can generate the parameters. However, when I used Stata to run 3PL model, it cannot generate the parameters but keep saying "not concave". Do you know why?

[Weiwen Ng]

If you have output, can you post it? You can limit the number of iterations. For example, say by iteration 30 the model is iterating to about the same log likelihood value, but it keeps declaring not concave. You could type:

irt 3pl ..., iterate(30)

Don't treat those as final results, I just want to see what parameter values the model is at, particularly the guessing parameter. In particular, the pseudoguessing parameter seems to be (by my read of the 3PL documentation) interpreted in probability terms, as the minimum chance of responding correctly to any item. Probabilities near 0 and 1 can give logit models trouble with convergence. It may be that your pseudoguessing parameter is near 0.

I don't know exactly what to do about that. If it is, you could report that the parameter seemed to be near 0. You could constrain it to zero, but if you did that, then you have the 2PL model.

[Freddy DD]

If you have output, can you post it? You can limit the number of iterations. For example, say by iteration 30 the model is iterating to about the same log likelihood value, but it keeps declaring not concave. You could type:

irt 3pl ..., iterate(30)

Don't treat those as final results, I just want to see what parameter values the model is at, particularly the guessing parameter. In particular, the pseudoguessing parameter seems to be (by my read of the 3PL documentation) interpreted in probability terms, as the minimum chance of responding correctly to any item. Probabilities near 0 and 1 can give logit models trouble with convergence. It may be that your pseudoguessing parameter is near 0.

I don't know exactly what to do about that. If it is, you could report that the parameter seemed to be near 0. You could constrain it to zero, but if you did that, then you have the 2PL model.

Hi Weiwen,
Thank you so much for your great help! It did really help when I used the code you recommend to restrict the number of iterations!!!
By the way, I have one more question about Item Characteristic Curves (ICC). Usually, ICC is in an upward trend. However, I ran a data set and found that some items' ICCs are in a downward trend.
Please see the picture as follows:

q1 refers to Question1, and q2 refers to Question 2. The ICC of q1 is in a regular upward trend but the ICC of q2 is in a downward trend.
Could the reason be that the "wrong answer (coded as 0)" in q2 should be the actual, correct answer?

Thanks for the update of "dataex". However, I still do not figure it out... I typed help dataex at the command line, and the output looks like a handbook...
Again, thank you so much for your help!

Best,
Freddy

[Weiwen Ng]

Thank you so much for your great help! It did really help when I used the code you recommend to restrict the number of iterations!!!

Hang on! In anything that uses maximum likelihood estimation, which is most estimation commands, you absolutely must let the algorithm iterate to convergence (i.e. near zero change in the log likelihood) before you accept the results! I just wanted to see the pseudoguessing parameter from your non-final results, to see if it was the problematic parameter! I realize we don't normally end every sentence in an exclamation mark on this forum, but I absolutely cannot emphasize enough that what I asked you to do was not meant to help you get results, but to attempt a diagnosis! The results are not final!!

Anyway, downward trend in the ICC, i.e. the probability of a positive response declines as Theta (the latent trait) increases. Yes, reverse coding (intentional or not) would explain this. I am not certain if anything substantive about the results is affected by this, but you might as well code it such that 1 is correct, lest you scare people reviewing the results.

[Freddy DD]

Hang on! In anything that uses maximum likelihood estimation, which is most estimation commands, you absolutely must let the algorithm iterate to convergence (i.e. near zero change in the log likelihood) before you accept the results! I just wanted to see the pseudoguessing parameter from your non-final results, to see if it was the problematic parameter! I realize we don't normally end every sentence in an exclamation mark on this forum, but I absolutely cannot emphasize enough that what I asked you to do was not meant to help you get results, but to attempt a diagnosis! The results are not final!!

Anyway, downward trend in the ICC, i.e. the probability of a positive response declines as Theta (the latent trait) increases. Yes, reverse coding (intentional or not) would explain this. I am not certain if anything substantive about the results is affected by this, but you might as well code it such that 1 is correct, lest you scare people reviewing the results.

I understand what you mean! Thank you so much for your great help again Weiwen!!! You are so nice!!!

Sincerely,
Freddy