Generate scores and test model fit of Item Response Theory model

[Zora Ng]

Hi, I used the -irt- function in Stata 14 to fit a material deprivation index using some typically accepted indicators.

irt 1pl matdep1-matdep8
estimates store m1
irt 2pl matdep1-matdep8
estimates store m2
lrtest m1 m2


1. The 2 parameter model comes out significantly different to the first model. i can only presume that it fit better, not worse. Going through the new IRT instructions in Stata, I couldn't figure out how to test how well the model fit the data. Any advice there?
2. I also couldn't figure out how to generate an actual 'predict' style indicator score for the model's latent trait either, so just used -gsem- for the model (see below). I'm not sure it's the same kind of 2-parameter model as when using the -irt- function, or how to test that model fit against the data either. Can anyone help?

gsem (Matdep -> a_matdep1-a_matdep8), logit var(Matdep@1)
predict matdep*, latent

Thanks,

ZN

[wbuchanan]

For your second question see:

help irt 1pl postestimation##predict
help irt 2pl postestimation##predict
help irt 3pl postestimation##predict

For the first question, the likelihood ratio test is fine, but you also need to consider the assumptions that each type of model makes about the functional form of the response (e.g., guessing, varying difficulties, etc...). In cases where you are using the marginal maximum likelihood estimator, there is an assumption that theta is known and/or that a sum of the keyed responses is a sufficient statistic (in the case of Rasch modeling). When both theta and the item parameters are unknown, the Joint Maximum Likelihood Estimator can be used to estimate the person and item parameters simultaneously. If that is what you were interested in doing, see the example below:

. net inst raschjmle, replace from("http://www.paces-consulting.org/stata")

. webuse masc1, clear
(Data from De Boeck & Wilson (2004))


. raschjmle q*
  800
 Iteration                 Delta           Log-likelihood
--------------------------------------------------------------
         1     0.502591842208104       -3402.304331969046
         2     0.142412255554409       -3397.822027114892
         3     0.020979991419945       -3397.719031584525
         4     0.003561687956111       -3397.716620516149
         5     0.000591506681447       -3397.716599152711
--------------------------------------------------------------
                                                                                                              
=================================================================================================        
Item           Difficulty     Std. Error        WMS       Std. WMS        UMS       Std. UMS             
-------------------------------------------------------------------------------------------------        
q1                  -0.40           0.08         0.85        -4.32         0.84        -2.86                  
q2                   0.11           0.08         1.03         1.04         1.05         1.04                  
q3                  -1.36           0.10         0.93        -1.39         0.86        -1.39                  
q4                   0.49           0.08         0.99        -0.25         1.02         0.38                  
q5                   1.66           0.09         0.93        -1.54         1.02         0.28                  
q6                   0.82           0.08         0.93        -2.05         0.95        -0.82                  
q7                   1.37           0.09         1.10         2.42         1.17         1.99                  
q8                  -1.87           0.11         0.77        -3.81         0.85        -1.14                  
q9                  -0.81           0.09         1.04         1.04         1.13         1.66                  
=================================================================================================        



SCALE QUALITY STATISTICS                          
==================================================
Statistic                  Items     Persons  
--------------------------------------------------
Observed Variance         1.3031      1.4411
Observed Std. Dev.        1.1415      1.2005
Mean Square Error         0.0080      0.7097
Root MSE                  0.0894      0.8425
Adjusted Variance         1.2951      0.7314
Adjusted Std. Dev.        1.1380      0.8552
Separation Index         12.7235      1.0151
Number of Strata         17.2980      1.6868
Reliability               0.9939      0.5075
==================================================

            SCORE TABLE                
==================================
  Score        Theta      Std. Err
----------------------------------
    0.00        -3.94         1.89     
    1.00        -2.55         1.12     
    2.00        -1.59         0.89     
    3.00        -0.89         0.80     
    4.00        -0.28         0.77     
    5.00         0.31         0.76     
    6.00         0.91         0.79     
    7.00         1.59         0.87     
    8.00         2.53         1.11     
    9.00         3.89         1.89     
==================================

Hope this helps.

[wbuchanan]

I removed the package from my personal server but cannot update the previous post. You can use:

net inst raschjmle, from("http://wbuchanan.github.io/StatajMetrik/")

And it should install from the current location.

[fahad alharbi]

Is it only for Rach model, do you have any package that can work for GSM or partial credit? I tried to use it but I got NAN

[wbuchanan]

fahad alharbi
it only works with dichotomously keyed items at the moment. The underlying java library has way more capabilities, but I’ve not had much time to work on putting together similar wrappers for those cases. You could also use jMetrik as a stand alone program if that is what you are needing.

[fahad alharbi]

fahad alharbi
it only works with dichotomously keyed items at the moment. The underlying java library has way more capabilities, but I’ve not had much time to work on putting together similar wrappers for those cases. You could also use jMetrik as a stand alone program if that is what you are needing.

Thank you!
I want to get expected score for each person with its error in all of my data as two variables. One for expected score and another for SE. Do you have any way for that?
thanks

[wbuchanan]

Not with anything scored polytomously. You also need to be a bit more specific about which standard error you are referring to. I'm assuming you mean the conditional standard error of measurement. You can get it using jMetrik, I just haven't had time to put together the wrappers for the polytomous classes.