fvexpand and fvrevar: Ordering of elements of output

[Michael Oberfichtner]

Dear Statalisters,

Do fvexpand and fvrevar order their output identically when applied to the same varlist?

I am currently working on a code that uses the output of fvexpand to refer to output obtained with variables that fvrevar generates. This procedure relies on the assumption that both fvexpand and fvrevar order factor variables identical in their respective output. The minimum working example below illustrates what I am trying to achieve.

Are there any references that the two commands order their output identically or any experiences that this is (not) the case? (As the two commands are built-in, it is not possible for me to check the code.)

Many thanks in advance!
Michael

clear all
set obs 10
set seed 1
gen cont = runiform()
gen binary = (runiform() > .5)

local varlist c.cont##i.binary // define varlist for both commands
fvexpand `varlist'
local reference "`r(varlist)'"
fvrevar `varlist'
local variables "`r(varlist)'"

disp "The following summary statistics refer to the expressions: `reference'" // if `reference’ and `variables’ have the same ordering
sum `variables'

[Clyde Schechter]

I don't know the answer to your question. Playing around with a few examples, it appears that they do follow the same ordering. But I cannot find anything in the documentation which guarantees this will always be the case. Moreover, both -fvexpand- and -fvrevar- are build-in commands, so I cannot check the code to see if one refers to the other, or if they both use the same algorithm.

That said, the -fvrevar- command wouldn't be very useful if this were not the case, as the output of commands to which its `r(varlist)' is fed would be difficult or impossible to interpret. So I expect that they do, indeed, produce outputs in the same order. But I see no way to be sure of this short of contacting StataCorp's technical support.

[Michael Oberfichtner]

Thanks, Clyde.

My attempts to create a counterexample failed, too, and I could not think of a reason, why they should not follow the same ordering. However, I will contact StataCorp's technical support.

[Michael Oberfichtner]

Stata support's reply:

They should follow the same order. As in Stata, lowest value of a factor
variable is omitted and other values are listed in ascending order.
However, the two commands are slightly different.

-fvexpand- expands a factor varlist to the corresponding expanded,
specific varlist.

-fvrevar- creates a variable list that includes equivalent, temporary
variables in place of the factor variables.

The difference is that Stata recognizes the factor variables using
-fvexpand-, and -fvrevar- creates the full list of interaction terms
you specified.

If you -summarize- your “reference” and “variables”,

. sum `reference'

Variable | Obs Mean Std. Dev. Min Max
-------------+---------------------------------------------------------
cont | 10 .3755035 .286673 .0210242 .9113581
1.binary | 10 .3 .4830459 0 1
|
binary#|
c.cont |
1 | 10 .0662047 .1429492 0 .4512149

. sum `variables'

Variable | Obs Mean Std. Dev. Min Max
-------------+---------------------------------------------------------
cont | 10 .3755035 .286673 .0210242 .9113581
__000000 | 10 0 0 0 0
__000001 | 10 .3 .4830459 0 1
__000002 | 10 0 0 0 0
__000003 | 10 .0662047 .1429492 0 .4512149


As shown above, -fvexpand- implements the factor operator and the lowest
value of a factor variable was omitted from the output. -fvrevar- gives
your the full list by adding two hypothetical base levels (second and
fourth row). If you want to have the full list and no base level omitted
from -fvexpand-, you can use -ibn.var- to keep the base level. For
example,

local varlist c.cont##ibn.binary // define varlist for both commands
fvexpand `varlist'
local reference "`r(varlist)'"
fvrevar `varlist'
local variables "`r(varlist)'"
sum `variables'
sum `reference'

. sum `variables'

Variable | Obs Mean Std. Dev. Min Max
-------------+---------------------------------------------------------
cont | 10 .3755035 .286673 .0210242 .9113581
__000000 | 10 .7 .4830459 0 1
__000001 | 10 .3 .4830459 0 1
__000002 | 10 .3092988 .3274926 0 .9113581
__000003 | 10 .0662047 .1429492 0 .4512149

. sum `reference'

Variable | Obs Mean Std. Dev. Min Max
-------------+---------------------------------------------------------
cont | 10 .3755035 .286673 .0210242 .9113581
|
binary |
0 | 10 .7 .4830459 0 1
1 | 10 .3 .4830459 0 1
|
binary#|
c.cont |
0 | 10 .3092988 .3274926 0 .9113581
1 | 10 .0662047 .1429492 0 .4512149

The two commands should give you the same list in same order.

[Jeremy Reynolds]

Thanks to Michael and Clyde for highlighting these commands. I have a small detail to add to the discussion . . .

I was really happy to learn about the fvrevar command. I was looking for a way to remove factor variable prefixes from a list of variables so the list could be used with commands that do not allow such prefixes. The fvrevar command makes that task super easy.

Unfortunately, at the moment, it does not always produce the list of modified variable names in the same order they were supplied. In the example below, the variable foreign was the first variable in the original list, but in r(varlist) it is the last variable.
I have contacted Stata support about the issue. Hopefully they will provide a solution soon.

I have not examined if the same problem occurs when using fvrevar with different options or with fvexpand.

Jeremy

 . sysuse auto
(1978 automobile data)
.
. fvrevar i.foreign c.length mpg price, list
.
. di "`r(varlist)'"
length mpg price foreign

[Jeremy Reynolds]

Here is a temporary workaround from Chris Cheng at Stata.

Jeremy

. sysuse auto
(1978 automobile data)

. local var i.foreign c.length mpg price

. 
. local j = 1

. foreach i of local var{
  2.         local pos = strpos("`i'", ".")
  3.         if `pos' > 0 {
  4.                 local pos1 = `pos' + 1
  5.                 local list`j' = substr("`i'", `pos1', .)
  6.         }
  7.         else{
  8.                 local list`j' "`i'"
  9.         }
 10.         local guts "`guts' `list`j''"
 11.         local ++j
 12. }

. 
. di "`guts'"
 foreign length mpg price