st_matrixcolstripe() does not work as expected.

[kkranker]

I am having trouble with st_matrixcolstripe() command with complex column stripe. Parts of the name are getting switched unexpectedly. Here's a trivial example:

mata:
    b = (1, 2, 3, 4)
    b
    st_matrix("b",b)
    
    s = ("eq1", "1.treat#0bn.post" \ "eq1", "1.treat#1.post" \ "eq2", "[email protected]" \ "eq2", "[email protected]" )
    s
    st_matrixcolstripe("b",s)
end
matrix list b

The output looks like this (Stata 13.1):

.. mata:
------------------------------------------------- mata (type end to exit) -------------------------------------------------------------------------------------------------------------------
:         b = (1, 2, 3, 4)

:         b
       1   2   3   4
    +-----------------+
  1 |  1   2   3   4  |
    +-----------------+

:         st_matrix("b",b)

:         
:         s = ("eq1", "1.treat#0bn.post" \ "eq1", "1.treat#1.post" \ "eq2", "[email protected]" \ "eq2", "[email protected]" )

:         s
                          1                      2
    +-----------------------------------------------+
  1 |                   eq1       1.treat#0bn.post  |
  2 |                   eq1         1.treat#1.post  |
  3 |                   eq2   [email protected]  |
  4 |                   eq2     [email protected]  |
    +-----------------------------------------------+

:         st_matrixcolstripe("b",s)

: end
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

. matrix list b

b[1,4]
            eq1:         eq1:         eq2:         eq2:
        1.treat#     1.treat#  1vs0.treat#  1vs0.treat#
         0.post       1.post       0.post       1.post
r1            1            2            3            4

As you can see, the column names I highlighted red are incorrect, at least as far as I can tell. The names before and after the "#" sign are swapped. This is not I expected based on the Stata documentation.

Any ideas or suggestions would be much appreciated!

Thanks,
Keith

[Jeff Pitblado (StataCorp)]

Keith found a bug in the matrix stripe parse logic. We hope to fix this in a future update.

In the mean time, Keith can change the order of treat and post for the first 2
elements to get the intended matrix stripe. We acknowledge that this will be difficult
if Keith is combining matrices, and thus their stripes, from estimation and postestimation
commands such as regress, margins, and contrast.

mata:
    b = (1, 2, 3, 4)
    b
    st_matrix("b",b)
    
    s = ("eq1", "0bn.post#1.treat" \ "eq1", "1.post#1.treat" \ "eq2", "[email protected]" \ "eq2", "[email protected]" )
    s
    st_matrixcolstripe("b",s)
end
matrix list b

Here is the log

. mata:
------------------------------------------------- mata (type end to exit) ------
:     b = (1, 2, 3, 4)

:     b
       1   2   3   4
    +-----------------+
  1 |  1   2   3   4  |
    +-----------------+

:     st_matrix("b",b)

:     
:     s = ("eq1", "0bn.post#1.treat" \ "eq1", "1.post#1.treat" \ "eq2", "r1vs0.p
> [email protected]" \ "eq2", "[email protected]" )

:     s
                          1                      2
    +-----------------------------------------------+
  1 |                   eq1       0bn.post#1.treat  |
  2 |                   eq1         1.post#1.treat  |
  3 |                   eq2   [email protected]  |
  4 |                   eq2     [email protected]  |
    +-----------------------------------------------+

:     st_matrixcolstripe("b",s)

: end
--------------------------------------------------------------------------------

. matrix list b

b[1,4]
           eq1:        eq1:        eq2:        eq2:
        0.post#     1.post#  1vs0.post#  1vs0.post#
       1.treat     1.treat     0.treat     1.treat
r1           1           2           3           4

[kkranker]

Thanks Jeff. I thought I was going nuts. I was, indeed, combining matrices, and thus their stripes, from estimation and postestimation
commands. Thanks for the tip.

[kkranker]

Hi Jeff,

Thanks again for your help with this. I definitely did notice some improvements when I updated Stata to the 05may2015 update.

However, I'm still getting some odd behavior with the matrix row and column names parsing logic. (I'm now using the 03 Aug 2015 version of Stata 14.) Here's three examples:

. matrix b = (1, 2, 3)
. matrix colnames b = eq1:r1vs0.var1#[email protected] eq2:1.var1 eq2:1.var2

results in
  matrix colnames b = eq1:r1vs0.var1#[email protected] eq2:r1vs0.var1 eq2:r1vs0.var2

. matrix colnames b = eq1:r1vs0.var1#[email protected] eq1:r1vs0.var1#[email protected]  eq2:0bn.var3#2.var2#1.var1

results in
  matrix colnames b = eq1:r1vs0.var1#[email protected] eq1:r1vs0.var1#[email protected] eq2:r1vs0.var1#[email protected]

. matrix colnames b = eq1:0bn.var3#2.var2#1.var1 eq2:r1vs0.var1#[email protected] eq2:r1vs0.var1#[email protected]

results in
  matrix colnames b = eq1:0bn.var3#2bn.var2#1bn.var1 eq2:0bn.var3#2vs0.var2#1vs0.var1 eq2:1.var3#2vs0.var2#1vs0.var1

In that 3rd example, it seems to lose track of the "@" sign and shuffles the order of the variables around. My actual code is more complex, but it's leading to some really odd results, with variables switching order and so on.

I don't think I know what the core issue is. However, here's one more example to give you a little inspiration for where I'm running into trouble:

sysuse auto,clear
qui regress price i.rep78##i.foreign c.weight
margins foreign
matrix a = r(b)
matrix coleq a = aaa

margins rep78
matrix b = r(b)
matrix coleq b = bbb

margins r.foreign@rep78,
matrix c = r(b)
matrix coleq c = ccc

matrix combined = (a,b,c)
matrix list combined

ereturn post combined
ereturn display

Gives the following odd results:

 . ereturn display
---------------------------------------------------------------------------------
                |      Coef.
----------------+----------------------------------------------------------------
aaa             |
      r.foreign |
      Domestic  |   5226.075
       Foreign  |          0
----------------+----------------------------------------------------------------
bbb             |
          rep78 |
             1  |          0
             2  |          0
             3  |   6163.774
             4  |   5627.083
             5  |   8411.683
----------------+----------------------------------------------------------------
ccc             |
r.foreign@rep78 |
     Foreign 1  |          0
     Foreign 2  |          0
     Foreign 3  |   3680.342
     Foreign 4  |   5427.856
     Foreign 5  |    398.453
---------------------------------------------------------------------------------

Any more help?

Thanks!
Keith

[Jeff Pitblado (StataCorp)]

This is a tough one Keith. The design for contrast operators does not allow for mixed use like this.
I have no short term solutions here, but will look into getting Stata to handle cases like this more
sensibly.

[kkranker]

Thanks for looking into it. Please let me know what you find. I can see that I'm probably going beyond the original, intended use of these operators. I also see the advantage of some of the ways that Stata rearranges the terms in each column. Using 1.A#2.B instead of 2.B#1.A is fair enough and seems sensible. What I don't like is how this changes terms in ways that changes the meaning of the expression. If anything, it would be great if you guys could document how we're "supposed' to use these operators.

[Jeff Pitblado (StataCorp)]

I see nothing wrong with your use of the matrices after margins and margins, contrast.
The real problem is that we overlooked this possibility when developing how Stata handles
contrast operators.

After looking at this more closely, we have determined how to fix this problem. The fix will change
matrix stripe behavior for a handful of Stata commands, and we believe for the better, but we
still have to preserve the old behavior under version control (so we are not breaking anyone's
current do-files). I am confident the fix will go out in a future update to Stata 14, but I'm not
sure exactly when.

[kkranker]

Great. I'll look forward to this update. I really appreciate your attention to the issue. This type of commitment is what makes Stata perennially a great software package.