Exporting values from matrices

[Giorgio Piccitto]

Dear all, I am struggling with the following issue (that I really do not understand).

I have saved (at least this is what I think to have done) 3 matrices, whose I would like to export some coefficient in a table.

I did it with the following command

matrix b_restricted    = e(b1base)
matrix b_unrestricted  = e(bfull)
matrix b_decomposition = e(b)

and it seems to have worked properly, since when asking for:

 matrix list  e(b)
 matrix list e(bfull)
 matrix list e(b1base)

Stata shows me the correct values (that I would like to export). Then, having tried with different methods (looking a bit around on the web), I always end up with a table with all 0s; is like in some way I discard the saved matrices, but I reallky don't know in which step.

Do you have any suggestion on how to deal with this?

Thanks a lot, GP

[Clyde Schechter]

matrix list  e(b)
matrix list e(bfull)
matrix list e(b1base)

does not verify that you have copied the coefficients correctly--it just shows that e(b), e(bfull), and e(b1base) contain the coefficients you want. To verify that you have copied them correctly you would have to run:

matrix list b_restricted
matrix list b_full
matrix list b_decomposition

and verify that those contain the coefficients you want.

The commands

matrix b_restricted    = e(b1base)
matrix b_unrestricted  = e(bfull)
matrix b_decomposition = e(b)

are syntactically fine and will copy those corresponding matrices, provided that the e(...) matrices mentioned actually exist at the time these commands are run. That will be the case if all of them are outputs of a single Stata estimation command. I can't, off the top of my had, think of any Stata estimation command that returns all three of e(b), e(bfull), and e(b1base). But Stata has a huge number of estimation commands, the bulk of which I am not really familiar with, not to mention the huge number of user-written estimation commands. So perhaps this is the case and running the three -matrix b_... = - commands together after running that single command is what you did. If so, the copying was correctly done.

If, however, they come from three different estimation commands, and you run all three of them at the same time, only the most recently run estimation command will still have its -e()- results intact. Each time an estimation command is run, all previous contents of -e()- are cleared out. So you would have to revise your code to run each of the -matrix b_... = - commands separately, immediately following the corresponding estimation command that creates it.

All of that said, it is also possible that the code that comes after all of these may have inadvertently modified the matrices you created, or the commands you have used to try to export them may not make be correct. As you do not show any of that code, it is anybody's guess where it might have gone wrong and how you might deal with it.

[Giorgio Piccitto]

Dear Clyde, thanks a lot for your answer.

To go along what you said: I provide belows how my data looks like.

* Example generated by -dataex-. For more info, type help dataex
clear
input float(dip female age_class edu_3) double income float speakwell
0 0 6 3 1670 0
0 1 1 2    . 0
0 1 1 2 1130 0
0 1 2 3 1320 0
0 1 1 2    . 0
1 1 1 2 1100 0
0 1 2 3  250 0
0 1 2 3 1900 0
0 1 6 3 2140 0
0 1 2 3 2200 0
end
label values edu_3 edu_3
label def edu_3 2 "n diplôme de l'enseignement primaire ou secondaire (hors BTS)", modify
label def edu_3 3 "Un diplôme de l'enseignement supérieur (y compris BTS)", modify

Here, I am running a Gelbach decomposition through the following command:

b1x2 dip  , x1all(female age_class ) ///
 x2all(edu_3  ///
income ///
speakwell)  ///
 x2delta(g1=edu_3 income ///
 : g2= speakwell ) x1only(female)

and then saving the matrices:

matrix list b_restricted
matrix list b_full
matrix list b_decomposition

When doing so, I receive a message

 matrix b_full not found

, while the other two matrices seem to be fine, and containing the proper values.

Is this maybe the error preventing me to go on? Do you have any clue on why matrix b_full is not found?

Thanks a lot in advance for your kindness.

Best, GP

[Andrew Musau]

From

matrix b_restricted = e(b1base)
matrix b_unrestricted = e(bfull)
matrix b_decomposition = e(b)

matrix e(bfull) is saved as b_unrestricted.

mat l b_unrestricted

[Clyde Schechter]

I'm having two difficulties here. One is that I have never seen, nor heard of, -b1x2- before. It is apparently user-written command available from SSC. When I run your command with your example data, I get an error message "conformability error." This in turn arises fairly deep inside -b1x2- having to do with some matrices calculated internally, and I do not know why it arises. Do you get this same message when you run it? It may well depend on the specifics of the data. If you do not get it when you run it with your full data set, it might even arise just because the example data set here is too small for the intended analysis. So I'm not sure what to make of this part. If you are getting this error message (or any other error message) from -b1x2- itself, then, of course b_full will not be found: the command that creates it didn't run! Of course, it will also be the case that Stata halts execution when giving this message, so Stata won't even go on to attempt to execute your later commands. It is a general principle in using Stata that once Stata gives you an error message, you should not attempt to proceed with executing the rest of the code: it will almost never work, and when it does, it will usually produce wrong results. Once you get an error, you have to stop what you're doing and fix that!

For my second difficulty, let me assume that when you run -b1x2- with your full data set it runs to completion without any error message. According to its help file it does return e(bfull), e(b1full), e(b2full), and e(b1base). It does not, however, return anything called e(b). So your -matrix b_decomposition = e(b)- command is doomed to fail, and any attempt to use b_decomposition will give you an error message saying that that matrix is not found. However, I see no analogous reason why you would have that problem with matrix b_full.

So, assuming -b1x2- runs without error messages with your real data on your setup, we have two overall possibilities: -b1x2- is either creating e(bfull) but you are somehow losing it, or it is not. You can distinguish these by running -matrix list e(bfull)- immediately after -b1x2-.

Case 1.-b1x2- is for some reason not returning-e(bfull)- This may mean that there is something wrong with your data that precludes calculating that matrix, or it may be a bug in -b1x2-. I can't help you distinguish these as I know next to nothing about this command and don't have the time or inclination to learn it now. My thought would be that you carefully review your inputs to make sure they are suitable for the command, and if you are satisfied that they are, contact the author of -b1x2- for advice. (You might wait a day or so before doing that because it is possible that some other Forum member is familiar with -b1x2- and can advise you here.)

Case 2. -b1x2- does return e(bfull)-. Your next command should then be -matrix b_full = e(bfull)-. You should verify that that ran correctly by next running -matrix list b_full-, verifying that it contains what you expect. What I would then do is insert a -matrix list b_full- command after each remaining command, so you will see at what point Stata finally tells you that it no longer finds that matrix. Then the command just before that will be the one that removes it. At least you will have localized the "terminal event" for b_full. Then you can focus on the commands between -b1x2- and that command to see what is going wrong.

Again, let me emphasize that it would have been much more helpful to display example data that actually reproduces the problem you are having, and to show the code all the way between -b1x2- and the error you are encountering. I could have then done the steps I'm asking you to do, and perhaps after localizing the problem I might have even been able to unravel the root cause of it. The more you help those who try to help you, the more likely it is you will get a useful response. Hiding the code that you need help debugging is self-defeating.

[Giorgio Piccitto]

Dear all, thanks a lot for your advises. It seems an issue too complicated to be debated here, probably it is my fault that I am not able to convey properly what I am running into.

Thanks anyway to all of you for your kind help.

Best, GP

[Giorgio Piccitto]

Dear all, just to come back to this issue (after a way of some attempts to follow what Clyde and Andrew kindly commented).

I really do not see what is going wrong with dealing with the matrices are created by the command -b1x2-. First of all, apologize Clyde if the data I provided did not work, but I assume that the decomposition run by such command may be quite 'expensive' in data terms, and probably this is at the origin of the warning you run into. At the same time, I thought that I couldn't provide bigger parts of my dataset (that is why I provided only such a small part). But maybe you can kindly provide me some further tips without running exactly the same -b1x2- decomposition I am running with my data, since the problem seems to be related to matrices handling.

In particular, following Andrew's comment on the correct name of the matrix, I run the following code to try to export the mediation coefficients on a table:

 
matrix b_decomposition = e(b)
matrix list b_decomposition
mkmat  b_decomposition, matrix( b_decomposition) rownames(names)
svmat  b_decomposition, names(col)
gen varname = names
order varname
export delimited varname coeffs*, using mediation_coeffs.csv, replace

Yet, Stata gives me the following warning, preventing me from executing the command:

 
mkmat  b_decomposition, matrix( b_decomposition) rownames(names)
variable b_decomposition not found

but the content and the name of the variable seems to be fine, as from:

 
matrix list b_decomposition

b_decomposition[1,3]
origin_de~P:  origin_de~P:  origin_de~P:
              g1            g2          __TC
y1     .00036102     .00228619     .00264721

I am not an expert of dealing matrices in Stata, but this really puzzles me. Is there any advise on why this happens?

Best and many thanks, Giorgio

[Andrew Musau]

matrix b_decomposition = e(b)
matrix list b_decomposition
mkmat b_decomposition, matrix( b_decomposition) rownames(names)
svmat b_decomposition, names(col)
gen varname = names
order varname
export delimited varname coeffs*, using mediation_coeffs.csv, replace

Yet, Stata gives me the following warning, preventing me from executing the command:

mkmat b_decomposition, matrix( b_decomposition) rownames(names)
variable b_decomposition not found

mkmat creates a Stata matrix from existing variables. In this case, the matrix already exists. If you want the opposite (i.e., to create variables from a matrix), you use svmat, which is what you have in the next line. So just delete the line with the mkmat command.

matrix b_decomposition = e(b)
matrix list b_decomposition
svmat b_decomposition, names(col)
gen varname = names
order varname
export delimited varname coeffs* using mediation_coeffs.csv, replace

[Giorgio Piccitto]

Dear Andrew, thanks a lot.

When running your code, I run into the following warning:

. matrix list b_decomposition

b_decomposition[1,4]
     origin_de~P:  origin_de~P:  origin_de~P:  origin_de~P:
              g1            g2            g3          __TC
y1     .00264921     .00185979      .0028175      .0073265

. svmat b_decomposition, names(col)

. gen varname = names
names not found

probably it is a silly error, and if so I apologize, but as I said I am starting right now familiarizing with matrices handling in Stata...

thanks, GP

[Andrew Musau]

I am not sure what output you want. From the matrix posted in #9, are you looking for the following?

clear
mat bdecomposition= (.00264921,     .00185979,      .0028175,      .0073265)
mat colnames bdecomposition= g1 g2 g3 TC
mat coleq bdecomposition= origin_destinationP
mat rowname bdecomposition= y1
mat l bdecomposition

frame create matframe
frame matframe{
    local col= word("`:coleq bdecomposition'", 1)
    svmat bdecomposition, name(col)
    rename (*) `col'=
    gen row=1
    reshape long `col', i(row) j(varname) string
    drop row
    list
}

Res.:

. mat l bdecomposition

bdecomposition[1,4]
     origin_de~P:  origin_de~P:  origin_de~P:  origin_de~P:
              g1            g2            g3            TC
y1     .00264921     .00185979      .0028175      .0073265

. 
. 
. 
. frame create matframe

. 
. frame matframe{
. 
.     local col= word("`:coleq bdecomposition'", 1)
. 
.     svmat bdecomposition, name(col)
number of observations will be reset to 1
Press any key to continue, or Break to abort
Number of observations (_N) was 0, now 1.
. 
.     rename (*) `col'=
. 
.     gen row=1
. 
.     reshape long `col', i(row) j(varname) string
(j = TC g1 g2 g3)

Data                               Wide   ->   Long
-----------------------------------------------------------------------------
Number of observations                1   ->   4           
Number of variables                   5   ->   3           
j variable (4 values)                     ->   varname
xij variables:
origin_destinationPTC origin_destinationPg1 ... origin_destinationPg3->origin_destinationP
-----------------------------------------------------------------------------
. 
.     drop row
. 
.     list

     +--------------------+
     | varname   origin~P |
     |--------------------|
  1. |      TC   .0073265 |
  2. |      g1   .0026492 |
  3. |      g2   .0018598 |
  4. |      g3   .0028175 |
     +--------------------+
. 
. }