Replication code for Stata 14 and Stata16 producing different results

[Wilfred Chow]

I ran into a problem where a data replicator could not replicate an item response variable using the predict command (they were using Stata 16 while I produced my results in Stata 14.2). I used irt grm var1 var2 var3 var4 using the following code:

quietly irt grm imm1 imm2_1 imm2_2 imm3
predict immigration1

In Stata 14.2, I got the following output and successfully created the predict values:
(option pr assumed)
(option conditional(ebmeans) assumed)
(using 7 quadrature points)

But in Stata 16, I received the following error output:
(option pr assumed)
(option conditional(ebmeans) assumed)
(using 7 quadrature points)
option pr not allowed;
no depvars were specified using family(bernoulli), family(ordinal), or family(multinomial)
r(198);

For some reason, Stata 16 indicates that the option pr is not allowed, but that should be available to the irt command since Stata 14. I am not entirely sure why there this is occurring. I am able to reproduce the results with Stata 14 on different systems, but all users for Stata 16 observe the same error (I had colleagues check this for me). Any help for this issue would be appreciated.

[Joseph Coveney]

According to the help file for -predict- after -irt grm- for the current version, "newvarsspec is stub* or newvarlist."

Have you tried

predict immigration1*

[Wilfred Chow]

I recognize that it is a stub, but I do not want the other variables as I am only interested in the first variable. This code works fine for Stata 14.2 but not for Stata 16. It is possible that using the asterisk for all predicted variables could solve this issue in Stata 16. I'll report the results.

Update: Thanks for for the suggestion. That fixed the problem right up. It does appear that Stata 16 requires a complete variable list rather than specifying a single stub.

[William Lisowski]

It appears that a problem with irt 2pl - predict disallowing a single newvar - was corrected in the 5nov2020 update to Stata 16 - and I wonder, since irt grm is described as an extension of irt 2pl, if there wasn't a common code base in predict so the problem was also corrected at that time for irt grm.

predict after irt 2pl, when specified with a single new variable name, exited with an unhelpful error
message when it should have performed the requested prediction for the first outcome variable specified
in the fitted model. This has been fixed.

But Stata 17 has evolved somewhat further.

Stata/SE 17.0 for Mac (Intel 64-bit)
Revision 01 Jun 2022

My attempt to replicate the problem on Stata 17 did not fail outright, but produced a prediction for only the first level of the first outcome variable, which does not seem a useful result.

. webuse charity
(Data from Zheng & Rabe-Hesketh (2007))

. quietly irt grm ta1 ta2

. predict prdn
(option pr assumed)
(option conditional(ebmeans) assumed)
(using 7 quadrature points)

. describe

Contains data from https://www.stata-press.com/data/r17/charity.dta
 Observations:           949                  Data from Zheng & Rabe-Hesketh (2007)
    Variables:             6                  30 Oct 2020 14:58
------------------------------------------------------------------------------------------------
Variable      Storage   Display    Value
    name         type    format    label      Variable label
------------------------------------------------------------------------------------------------
ta1             byte    %17.0g     agree      Charitable organizations more effective
ta2             byte    %17.0g     agree      Degree of trust
ta3             byte    %17.0g     agree      Charitable organizations honest/ethical
ta4             byte    %17.0g     agree      Role improving communities
ta5             byte    %17.0g     agree      Job delivering services
prdn            float   %9.0g                 Predicted probability (0.ta1)
------------------------------------------------------------------------------------------------
Sorted by: 
     Note: Dataset has changed since last saved.

.

Here I use a stub with the outcome() option to limit the predictions to the first outcome variable, and it produces predictions for all levels of the outcome. If I use a stub with no outcome() option, I get, as expected, predictions for every level of every outcome.

. webuse charity
(Data from Zheng & Rabe-Hesketh (2007))

. quietly irt grm ta1 ta2

. predict prdn*, outcome(ta1)
(option pr assumed)
(option conditional(ebmeans) assumed)
(using 7 quadrature points)

. describe

Contains data from https://www.stata-press.com/data/r17/charity.dta
 Observations:           949                  Data from Zheng & Rabe-Hesketh (2007)
    Variables:             9                  30 Oct 2020 14:58
------------------------------------------------------------------------------------------------
Variable      Storage   Display    Value
    name         type    format    label      Variable label
------------------------------------------------------------------------------------------------
ta1             byte    %17.0g     agree      Charitable organizations more effective
ta2             byte    %17.0g     agree      Degree of trust
ta3             byte    %17.0g     agree      Charitable organizations honest/ethical
ta4             byte    %17.0g     agree      Role improving communities
ta5             byte    %17.0g     agree      Job delivering services
prdn1           float   %9.0g                 Predicted probability (0.ta1)
prdn2           float   %9.0g                 Predicted probability (1.ta1)
prdn3           float   %9.0g                 Predicted probability (2.ta1)
prdn4           float   %9.0g                 Predicted probability (3.ta1)
------------------------------------------------------------------------------------------------
Sorted by: 
     Note: Dataset has changed since last saved.

.