Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • IRT within loop

    Hi

    I am using Stata/SE 14.

    I have item bank of 260 questions (binary response), these items are spread across 50 bins/categories. There are 2000 candidates who gave their responses to these items already.

    Now I would like to generate say 100 exams each with 50 items (one item from each bin). Then I would like to run 2pl irt model for each exam and see if these 100 exams are parallel (by checking the test information curve for example)

    I assume a loop can be used to do this, but I am not sure how . I want the loop to randomly select 50 items from the bank , one item from each bin , then run irt, store estimates and curve, and repeat this process say 100 times. So then I can compare the estimates and curves generated.

    This is the syntax I have in mind for an existing example:


    webuse masc1
    (Data from De Boeck & Wilson (2004))

    generate id = _n
    quietly reshape long q, i(id) j(item)
    rename q y
    fvset base none id item

    gen bin=1 if item==1 | item==2
    replace bin=2 if item==3 | item==4
    replace bin=3 if item==5 | item==6
    replace bin=4 if item==7 | item==8 | item==9

    *****I believe I can start a loop from here, this is the part I am having difficulty with***

    Randomly select one item from each bin (end up with an exam of four items in total, one from each bin)

    drop bin
    reshape wide y , i(id) j(item)
    irt 2pl y*
    estat report, byparm sort(b)
    irtgraph tif
    estimates store exam1 (for all 100 exams)

    Repeat this process 100 times (generate 100 exams with four items, one from each bin-then run 2pl irt)
    estimates table _all

    End of loop

    Can someone kindly help me how to do such loop or if you have a better idea

    Thank you

    Kind regards
    Nahla Betelmal

  • #2
    Your idea of how to construct "bins" won't work. You are defining bins as though they were characteristics of observations, but they are instead characteristics of variables. (Does this variable belong to this bin?). What you need is a data structure to hold the list of items (variables) in each bin. In Stata, one natural structure would be 50 locals, each one containing the list of items (variable names) in each bin. Here's an illustration of how your problem might be solved for five exams with three bins:
    Code:
    local exams 5
    local nbin = 3
    local bin1 q1 q10 q4 q6 q8 q17 q200
    local bin2 q4 q15 q16 q19
    local bin3 q9 q11 q7 q43 q28
    //
    forval i = 1/`exams' {
       // Assemble a question list for this exam.
       local qlist ""
       forval b = 1/3 { // loop over bins
          local numq: word count `bin`b''  // size of bin
          local pos = ceil(runiform() * `numq') // random q in bin
          local nextq: word `pos' of `bin`b''
          local qlist "`qlist' `nextq'"
       }
       di "Items for exam`i': `qlist'"
       // irt `qlist'......
    }
    The tedious part here would be making the list of items in each bin. If that's a problem for you, you'll need to tell us exactly how you have those lists stored currently.

    Comment


    • #3
      It isn’t quite as simple as binning. In the attempt to generate parallel forms, you are essentially needing to conduct an equating study. In this sense the study design will end up dictating the appropriate methodological approach. For example, a typical approach is to define a set of anchoring items which have already been calibrated. With the item parameters for that subset of items know, you could constrain the parameter estimates for those items to calibrate the other items and construct a stable scale across forms. Without knowing more about your study design it will be difficult for others to provide clear guidance on the best way forward. The best I can suggest at the moment is to consult the Kolen and Brennan book on test linking and equating.

      Comment


      • #4
        Hi Mike

        Thank you very much for your help. It works beautifully!

        Code:
        webuse masc1
        (Data from De Boeck & Wilson (2004))
        generate id = _n
        local exams 5
        local nbin = 3
        local bin1 q1 q2 q3
        local bin2 q4 q5 q6
        local bin3 q7 q8 q9 
        
        
        forval i = 1/`exams' {
           // Assemble a question list for this exam.
           local qlist ""
           forval b = 1/3 { // loop over bins
              local numq: word count `bin`b''  // size of bin
              local pos = ceil(runiform() * `numq') // random q in bin
              local nextq: word `pos' of `bin`b''
              local qlist "`qlist' `nextq'"
           }
           di "Items for exam`i': `qlist'"
           irt 1pl `qlist'
           estat report, byparm sort(b)
        irtgraph tif
        }
        I am afraid that making the list of items in each bin will be a manual work unless there is another smart code. I have 50 bins and more than 260 items. This is an example of stored list:

        dataex Bin q

        Code:
        * Example generated by -dataex-. To install: ssc install dataex
        clear
        input byte Bin float q
         1 15218
         1 23543
         2 34968
         4 38020
         5 38494
         5 10934
         7 36193
         8   677
        11 10086
        11 16837
        12 17904
        10 13551
        10 25125
        15  8640
        16 23692
        14 26180
        17 19396
        18 18308
        19 29815
        21 31264
        20 32884
        17 30773
        23 23770
        24 10672
        25 21073
        26 21793
        25 18365
        26  1940
        29 37056
        31  3517
        32 33751
        32 32626
        33  7843
        33 30640
        35  9615
        36  4581
        39  5237
        36   134
        39  2940
        42 36928
        40 10702
        41 39908
        43 37136
        48 13356
        45 30889
        45 35524
        45  7206
        46 24872
        47 16948
        49 39975
        50 38149
         1 14457
         2 13312
         3 37126
         4 23483
         5 26197
         6   862
         6 22727
         8 27823
         9 29570
        10   948
         9  8213
        12 26286
        13 39001
        15 28313
        14 22752
        19  3029
        16 36887
        16 28653
        17 30217
        20 34948
        21 29322
        22  4111
        23 36994
        24 11276
        14 15998
        27 26519
        25 34303
        26  9338
        31 20003
        29  3043
        31   738
        32 35892
        33  5413
        33 33191
        35 15338
        36 27257
        36 38899
        39 38728
        39 38083
        38  5501
        40  5071
        41 10076
        38   634
        44    25
        45 27786
        44  1507
        45  8122
        46  6013
        48 12458
        end
        Many thanks again

        Kind regards
        Nahla

        Comment


        • #5
          This construct lists of questions in each bin, and puts them in locals named binq1 - binq50:

          Code:
          * Example generated by -dataex-. To install: ssc install dataex
          clear
          input byte Bin float q
          1 15218
          1 23543
          2 34968
          4 38020
          5 38494
          5 10934
          7 36193
          8 677
          11 10086
          11 16837
          12 17904
          10 13551
          10 25125
          15 8640
          16 23692
          14 26180
          17 19396
          18 18308
          19 29815
          21 31264
          20 32884
          17 30773
          23 23770
          24 10672
          25 21073
          26 21793
          25 18365
          26 1940
          29 37056
          31 3517
          32 33751
          32 32626
          33 7843
          33 30640
          35 9615
          36 4581
          39 5237
          36 134
          39 2940
          42 36928
          40 10702
          41 39908
          43 37136
          48 13356
          45 30889
          45 35524
          45 7206
          46 24872
          47 16948
          49 39975
          50 38149
          1 14457
          2 13312
          3 37126
          4 23483
          5 26197
          6 862
          6 22727
          8 27823
          9 29570
          10 948
          9 8213
          12 26286
          13 39001
          15 28313
          14 22752
          19 3029
          16 36887
          16 28653
          17 30217
          20 34948
          21 29322
          22 4111
          23 36994
          24 11276
          14 15998
          27 26519
          25 34303
          26 9338
          31 20003
          29 3043
          31 738
          32 35892
          33 5413
          33 33191
          35 15338
          36 27257
          36 38899
          39 38728
          39 38083
          38 5501
          40 5071
          41 10076
          38 634
          44 25
          45 27786
          44 1507
          45 8122
          46 6013
          48 12458
          end
          // Easier with question numbers as strings
          gen str qq = "q" + trim(string(q, "%10.0f"))
          forval b = 1/50 {
          quiet levelsof qq if (Bin == `b'), clean local(binq`b')
          if (trim("`binq`b''") == "") { //flag empty bins
          local binq`b' = "Empty"
          }
          }
          // Examine locals containing lists of questions in each bin.
          forval b = 1/50 {
          di "bin`b': `binq`b''"
          }

          Comment

          Working...
          X