Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Differential Item Functioning - Create graphs based on groups

    Hello, I created an IRT model (generalized partial credit model) based on 2 different groups (male/female). The items are 7-point Likert-type. My command was:

    irt gpcm var1 var2 var3, group(sex)

    The model runs fine and I get results that make sense. When I then run

    estat greport

    All parameters (thetas) other than the mean and variance are identical (see numbers for only var3 and overall mean / variance). How can it be that everything is identical but then mean / variance are different?

    And finally (and most importantly), I would like to generate category response functions, boundary characteristic curves etc. However, when I then run:

    irtgraph icc var1 var2 var3, group(sex)

    Stata says: "Nothing to draw."

    What am I doing wrong?

    Thanks!



    ..................male ................female
    Discrim 5.8792584 ........5.8792584

    Diff
    2 vs 1 -1.7224575..........-1.7224575
    3 vs 2 -1.0862329 ..........-1.0862329
    4 vs 3 -.72656654 ..........-.72656654
    5 vs 4 -.30533028.......... -.30533028
    6 vs 5 .06475272 .............06475272
    7 vs 6 1.0806044 ............1.0806044

    mean(Theta) 1.110e-16 ......-.46675985
    var(Theta) 1 ....................1.4448088

    Last edited by Casey Martin; 05 Mar 2021, 06:12.

  • #2
    Some good questions here.

    First, you did in fact tell Stata to fit a GPCM, that there are two groups (by sex), and that all the item parameters are identical. I know this might sound odd at first, but in this syntax, you do in fact need to tell Stata which specific items have their item parameters differ. If you do not, then the program does in fact estimate a separate mean and variance for the focal group - here, that's women. So, women have about a 0.467 standard deviation lower (insert the name of whatever latent trait is being estimated here) than men. After all, what is Stata supposed to assume otherwise, that all the difficulty and discrimination parameters differ?

    You'll need to investigate the irt, group() syntax to learn how to specify how to vary the difficulty and discrimination parameters if you want to use the likelihood ratio method to detect DIF. There's an example in this post here. My understanding is that you want to be fitting the no-DIF model that you just fit, save it with estimates store, then compare it via likelihood ratio test to models where, for one item at a time, the difficulty parameters are unconstrained (uniform DIF), and the difficulty and discrimination are unconstrained (non-uniform DIF). The Stata IRT manual by Raykov and Marcoulides outlined that procedure, although it was written under Stata 14 (I think), and it definitely predates the group option, so the example there used the gsem command.

    If you read later in the thread I linked, I outlined how you can use ordinal logistic regression instead to attempt to detect which items have DIF (and what type of DIF). It's the equivalent of diflogistic for ordinal items. I am not sure if the theoretical IRT literature regards this procedure as inferior to the likelihood ratio based procedure outlined above. For that matter, if you read the end of the thread I linked, in the procedure I outlined, you are using a potentially dirty anchor (anchor = the items that have parameters constrained equal between groups; I believe that ideally you want to have this anchor free of DIF, but I don't see how you can know this a priori in most situations), so basically no procedure is perfect.

    After you finalize a model with one or more items permitted to have DIF, irtgraph whatever will display the curves for each group without needing any other options.
    Last edited by Weiwen Ng; 05 Mar 2021, 08:23.
    Be aware that it can be very hard to answer a question without sample data. You can use the dataex command for this. Type help dataex at the command line.

    When presenting code or results, please use the code delimiters format them. Use the # button on the formatting toolbar, between the " (double quote) and <> buttons.

    Comment


    • #3
      Originally posted by Casey Martin View Post
      ...
      And finally (and most importantly), I would like to generate category response functions, boundary characteristic curves etc. However, when I then run:

      irtgraph icc var1 var2 var3, group(sex)

      Stata says: "Nothing to draw."

      What am I doing wrong?
      ...
      You may have uncovered a bug in graphing after a generalized partial credit model. I'm not familiar with the GPCM, as the assumption of equal distance between the cutpoints isn't realistic for my line of work. Hence, I'd never tried it.

      I can verify that after you fit a grouped version of the graded response model, my go-to model, all the IRT graphs work correctly. (Although if you didn't tell the model that any DIF existed, all the graphs will look like the single group variant, except that the test characteristic curve has two identical lines superimposed because again, there's no DIF.) When you do the same to a GPCM, Stata returns the error you described. When you fit a non-grouped GPCM, this issue doesn't surface. I'm not aware that there's a theoretical reason that graphs should be invalid after a grouped GPCM. I can't see why there would be one, but I'm not familiar with the model, and I guess I could be wrong.

      I'd suggest submitting this to Stata support.
      Be aware that it can be very hard to answer a question without sample data. You can use the dataex command for this. Type help dataex at the command line.

      When presenting code or results, please use the code delimiters format them. Use the # button on the formatting toolbar, between the " (double quote) and <> buttons.

      Comment


      • #4
        Originally posted by Weiwen Ng View Post
        ... I'm not familiar with the GPCM, as the assumption of equal distance between the cutpoints isn't realistic for my line of work...
        First, ignore this sentence. I just really demonstrated my unfamiliarity with the GPCM by confusing it with a rating scale model.

        Second, I've sent this thread to Stata tech support.

        Third, here's some code to illustrate the problem to Stata, and to show you the proper syntax for item or test characteristic graphs after a multiple-group model. This code retrieves a sample dataset with 4 ordinal items and one covariate from a different SEM example. Say you decided (after iteratively testing the items) that items 1 and 2 had non-uniform DIF, but items 3 and 4 had no DIF by sex. If you fit a graded response model and you ask for the item characteristic curves for the items with DIF, you'll get the correct graphs. I'm not attaching them because they look like lines of colored spaghetti, but you can run the code yourself, and in your own work you can decide how to handle that the presenbtation. I would normally prefer to present the item information functions for this purpose, as well as the test characteristic curve (that's expected sum score vs latent trait).

        Code:
        use http://www.stata-press.com/data/r15/gsem_issp93
        quietly irt (grm y3 y4) (0: grm y1 y2) (1: grm y1 y2), group(sex) noheader
        irtgraph icc y1
        irtgraph tcc
        If you repeat with the generalized partial credit model, you get the error you identified.

        Code:
        quietly irt (gpcm y3 y4) (0: gpcm y1 y2) (1: gpcm y1 y2), group(sex) noheader
        irtgraph icc y1
        (nothing to draw)
        irtgraph tcc
        (nothing to draw)
        I won't write code, but the appropriate graphs do appear after both the partial credit model (the one where all items have the same discrimination parameter) and the rating scale model (probably the most restrictive ordinal IRT model, this is the one where the distances between categories are constrained equal). Hence, it's only the GPCM that's broken.
        Attached Files
        Be aware that it can be very hard to answer a question without sample data. You can use the dataex command for this. Type help dataex at the command line.

        When presenting code or results, please use the code delimiters format them. Use the # button on the formatting toolbar, between the " (double quote) and <> buttons.

        Comment


        • #5
          Hi Weiwen Ng,
          thank you so much for your help and your detailed explanation. This makes a lot of sense

          Comment

          Working...
          X