Extract indices of matrix

[louise nek]

Hello!

I have a correlation matrix (called C) with dimensions [723, 723]. The matrix consist mostly of zeros (about 80%) and some non-zero correlation coefficients. I am now interested in finding the row and column indices (i.e. the id-numbers) for the non-zeros.

Is there a way to extract these indices into a list or a matrix? (Or in other case, shrink matrix C such that it only keeps the non-zeros?)
That would be great! I am very grateful for you tips!

(I am using Stata 14.)

Best,
Louise

[Nick Cox]

search corrci

points to

search for corrci (manual: [R] search)
-----------------------------------------------------------------------------------------

Search of official help files, FAQs, Examples, SJs, and STBs

SJ-10-4 pr0041_1 . . . . . . . . . . . . . . . . . Software update for corrci
(help corrci, corrcii if installed) . . . . . . . . . . . . N. J. Cox
Q4/10 SJ 10(4):691
update to fix corrci so that it always saves r-class results

SJ-8-3 pr0041 . Speaking Stata: Corr. with confidence, Fisher's z revisited
(help corrci, corrcii if installed) . . . . . . . . . . . . N. J. Cox
Q3/08 SJ 8(3):413--439
reviews Fisher's z transformation and its inverse, the
hyperbolic tangent, and reviews their use in inference
with correlations

Idea explained in 8-3; software to be downloaded from 10-4.

With corrci installed you can do things like save the correlations to a new dataset and then drop what you don't care about.

sysuse auto, clear
corr price-foreign, saving(mycorr)
u mycorr
drop if abs(r) < 0.2

[louise nek]

Hi Nick!

Thanks for you answer!
However, I still can't make it work.


I tried a):

corrci r102897 r103010 r103606, saving(mycorr2)
u mycorr
drop if abs(r)<0.8

Which worked, but I need to include all the 723 variables, all starting with r.


So I tried b):

corrci r*, saving(mycorr1)

And get the error:

corr(): matrix has zero or negative values on diagonal

I also tried c):

corr r102897-r103010, saving(mycorr1)

And get the error:

option saving() not allowed

I am also interested in a more general command that can be used for all kinds of matrices, correlation matrices or other.
Is there a general command like "drop if abs(r)<0.8" that can be used to all matrices? Or is there a way to save a matrix similar to the ", saving(mycorr)" so that I then can use the "drop if abs(r)<0.8".

Thank you!

[Oded Mcdossi]

You can post the matrix to data file using the svmat command and then you can manipulate the data to be in long form and drop cells under specific conditions.

[Nick Cox]

Working backwards: if you have a matrix, then you can drop rows or columns, but nonzero entries would get dropped either way unless rows or columns were entirely zeros. So, it is not clear what you are looking for unless you reorder your data as (row index, column index, value), which is what corrci tries to do.

Your problem with corrci is a problem with Stata's function corr(), which is straining at your data somehow. I guess you need to work in Mata for that to work.

The inbuilt command correlate has no saving() option: it would be documented if it existed.

But this correlation matrix has of the order of 500,000 correlations, so I am not surprised that you want to thin it down.