How to make ICC lines for 3 groups on single graph (e.g., twoway)

[Wendy Garrard]

Hi StataList,

I am using twoway to create ICC graphs for item analysis of many variables. However, my data includes three conditions (groups). I would like to have the ICC for all three groups display on a single graph (per item). My primary goal is to use these graphs to quickly identify which items are best discriminators of differences between the TX group and the other 2 control groups.

Is it possible to do this with twoway or should I try a different approach?

Thanks very much in advance.

~~~~~~~~~~~~~~~~~~~~~~~
I am using the foreach loop below with twoway to produce three individual ICC graphs (1 per group) per item, but I can't figure out how to produce them (all 3 group) in a single graph.
~~~Stata 14, WIndows~~~~~

foreach var of varlist VAR1 VAR87 VAR9z7 VARaok VARz8 {
local GraphTitle : variable label `var'
logistic `var' StdTotal
predict p
twoway (line p StdTotal, sort lcolor(blue) by(StudyGrp) lwidth(thick) lpattern(solid) connect(direct)), ///
title("Q: `GraphTitle'", size(medsmall) color(black)) ///
ytitle("") yscale(range(1 0)) ylabel(0(0.2)1) ymtick(0(0.2)1) ///
xtitle("") xscale(range(-3 3)) xlabel(-3(1)3) xmtick(-3(1)3) ///
scheme(s1color) ///
name(`var', replace)
drop p
}

~~~~~~~~~~~~~~~~

[wbuchanan]

Not completely clear what you mean with regards to ICC (e.g., Item Characteristic Curve, Inter/Intra Class Correlation, etc...). Assuming you want to plot the predicted probabilities why not use :

foreach v of var VAR1 VAR87 VAR9z7 VARaok VARz8 {

     logistic `v' StdTotal
     predict p

     tw line p StdTotal if StudyGrp == 1, sort lc(blue) lw(medium) || ///   
     line p StdTotal if StudyGrp == 2, sort lc(orange) lw(medium) || ///   
     line p StdTotal if StudyGrp == 3, sort lc(green) lw(medthin) ///  
     ti(`"Q: `: var label `v''"', size(medium) c(black) span) yti("") xti("") ///   
     ysca(range(1 0)) xsca(range(-3 3)) ylab(0(.2)1, angle(0) nogrid) ///   
     xlab(-3(1)3) name(`v', replace) 

     drop p

}

For what it's worth, the -brewscheme- package on SSC was developed specifically to avoid some of the verbosity associated with controlling each of the aesthetic parameters here:

use http://www.stata-press.com/data/r12/lbw.dta, clear

ssc inst brewscheme, replace

brewscheme, scheme(example) linest(set1) linec(5) somest(pastel1) somec(3)

foreach v of var low smoke ht ui {

     logistic `v' age

     predict p

     tw line p bwt if race == 1, sort || line p bwt if race == 2, sort || ///  
     line p bwt if race == 3, sort scheme(example) ti(`"Q: `: var label `v''"') ///  
     xsca(range(-3 3)) ylab(#6) xlab(#7) name(`v', replace) xti("") yti("") 

     drop p
}

[Wendy Garrard]

Thanks wbuchanan,
YOur suggestion gets me a little close to what I want. BTW-sorry I was unclear about the meaning of ICC (I am using Item Response Curves in this case).

I tried both your approaches, with the same result. It produced a single graph for each item, and the legend for each graph indicated three groups (each a diff color). But, there was only 1 group ICC visible on each graph.

Cheers,
wg

[wbuchanan]

That's a bit more difficult to figure out. Why not use the IRT estimators and the

difmh

function to test for differential item function? If the data conforms to the assumptions of the Rasch model, you could also use the item-level infit, outfit, standardized infit, and standardized outfit statistics to identify poorly fitting items.

[Wendy Garrard]

Thanks again, I will try this option. And yes, the bid of code I was using was originally part of an IRT analysis. I was not aware of the command you suggested. Also, I know that Stata 14 has an IRT module, but I am not familiar enough with it to produce the output I need right now.. My immediate task is merely to produce some visual displays that will help me quickly sort out the better/worse items from an existing project that are likely to be useful for a new round of pilot testing for a new project.

Much thanks for your suggestions.

[wbuchanan]

Just asked my wife about this and she pulled out some lecture notes that took a similar approach. From what it looks like (given the example in those slides), you should also be conditioning the logistic regression on the group indicators and include a total score by group interaction effect.

That being the case, try:

logistic `v' c.StdTotal##i.StudyGrp
margins i.StudyGrp, at(StdTotal = range(-3 3))
marginsplot, scheme(example)

May need to be tweaked a bit, but I think that may be what you were looking to do.

[Wendy Garrard]

Wow! That last suggestion sent this idea "around third base & headed for home!" . . . Please thank your wife on my behalf ;-)

I tweaked the code to this:

foreach v of var var08C var10C var11C var13C var24C var26C  {
    logistic `v' c.StdTotal##i.StudyGrp
    margins i.StudyGrp, at(StdTotal = (-3(1)3))
    marginsplot, scheme(example) ///
        name(`v', replace)
    }

Thanks so much for your interest in my data puzzle. I appreciate your pushing me to think through the process. To answer your previous question about what code I was using, and for what purpose:
~~~~~~~
Initially I used an IRT approach presented at a Stata conference (Huber, 2012) on "Psychometrics Using Stata." I am tasked with sorting out good and bad items from a prior project. The items represent a test of knowledge gained pre-to-posttest in a randomized three group design. Only one group was taught the knowledge content. Unfortunately, many of the items performed poorly for a variety of reasons. I hoped that Item Char Curves would help me understand some unexpected response patterns. For example, several items have a "backward" ICC pattern (where individuals with lower overall scores [control groups] had more correct responses than those with higher overall scores). This got me curious about breaking out the individual item ICC graphs by tx & control groups as a potentially quick-but-dirty way of spotting items where the anomalous response patterns were associated with exposure to the experimental conditions.