Very simple question from a complete novice

[Paul Bowen]

Dear Colleagues

I am very new to both IRT and STATA, so please pardon the ignorance.

When performing IRT analysis with STATA using a Graded Response Model on a Sleep Problems scale (3 items) (see below), the program says that the log likelihood is not concave and that a discontinuous region was encountered. This model could not be fitted to the data, or graphs produced.


However, if the following models are applied, a solution is found and the graphs look “normal”.

Partial Credit Model

Generalised Partial Credit Model

Rating Scale Model.


Why is the GRM not working, and which of the alternative models is most suitable?


The questions and response options are:

In the past 3 months, how often have you:

1. Had trouble falling or staying asleep (R).

2. Woke up before you wanted to (R).

3. Woke up feeling refreshed.


The response options are: all of the time; most of the time; some of the time; a little of the time; and none of the time. Higher scores = greater sleep problems.

Thank you.
Paul

[Clyde Schechter]

First, I am more of an IRT novice than you are, as I have never used it even once. But I've been around Stata for a while, and I think that this problem is more generic than specific to IRT. I could be wrong about that, but we'll see.

In general, when a model fails to converge and is stuck in an area where it says "not concave" it usually means that the model is not identifiable. This usually means, in turn, that there is some variable whose contribution appears to be infinitely large (positive or negative) or that there is some subset of the predictor variables that participate in some relationship such that their individual contributions cannot be separately estimated, or simply that the data themselves don't happen to have the information needed to disambiguate them. Here's how I would start the troubleshooting process. Review your output logs and note how many iterations transpired up to the point where non-convergence and non-concavity arose. Let's say, for the sake of concreteness, this was at the 50th iteration. Then re-run your model exactly as before, but add -iterate(52)- to the options. Stata will go 52 iterations and then halt and give interim output. You cannot use that interim output as results--they are not valid estimates. But you will be able to see if there are any variables where the effect estimates are obviously insanely large, or the standard errors are insanely large. Those are the variables causing the problem. If you remove some or all of them and re-run your model, without the -iterate()- option, you will likely achieve success. Sometimes there are other clues that are helpful such as missing values for standard errors or significance tests, or oddities in the number of observations in the estimation sample, etc.

If that does not prove fruitful, I recommend posting back. Before doing that, however, please read the FAQ, which is replete with excellent advice on how to write posts that provide the information needed to get helpful and timely responses. Among the things that could be improved about this current post are:

1. Showing the exact commands you ran and the exact output you got from Stata would have enabled a more specific response; descriptions of what you did are nice, but the devil is nearly always in the details, and no detail is too small to be important. So you stand a better chance of getting useful advice if you show everything that happened, exactly as it happened. (And I do mean exactly, as in don't retype anything: use copy/paste and don't edit it in any way).

2. It is often helpful to show a data example so people can experiment with it to see if a potential solution actually works. That might or might not be useful here.

3. Choose an informative title for your thread. (And I strongly recommend that if you post back for more help on this problem you start a new thread that does have an informative title.) Most people here pick what threads they read based on their interest in and knowledge about the topic. "Very simple question from a complete novice" is attractive to almost nobody. I, for one, seldom read posts with titles like that--I'm really not sure why I read this one today. So give your post a title that describes the general theme or topic of the question, not who you are or how you think about yourself. Also, there are many people who learn from this Forum not by posting but by searching it to see what others have asked and answered. If somebody in the future encounters a similar problem and tries searching on "not concave" or "IRT", etc., this thread will not be turn up, even though it might have led to exactly the solution needed.

When reading the FAQ, please pay special attention to #12 so that in future posts when you display data, code or output you will do it in the way that maximizes its usefulness: -dataex- for data examples, code delimiters for code or output, and, above all, not using screenshots (except to show graphs).

[Paul Bowen]

Thank you.