Undocumented maximum matrix size for Mata->Stata?

[Mark Schaffer]

Hi all. According to -help limits-, the maximum size for a matrix in Stata is determined by matsize. But separately the documentation states that this can be circumvented by creating the matrix in Mata and then copying it into Stata using -st_matrix-, e.g., in the FAQ here: http://www.stata.com/support/faqs/ma...-than-matsize/.

I've been unable to find any mention of a limit to the size of a Stata matrix thus made. But trial-and-error suggests the maximum number of rows/columns is 32,767 (=2^15-1) since anything bigger generates a rather entertaining error, namely a negative number of rows or columns. For example:

. mata: m=J(32767,1,1)

. mata: st_matrix("m",m)

. di rowsof(m)
32767

. 
. mata: m=J(32768,1,1)

. mata: st_matrix("m",m)

. di rowsof(m)
-32768

I've checked this on a Windows 7 machine running Stata 10, 11 and 12 IC and Stata 13 SE.

Can someone verify that this is a genuine limit, and/or that it applies under other operating systems?

[Domininkas Mockus]

Thanks for the question! I get the same error - now I know why! (Stata 13 SE running on a 64-bit Windows 7 Enterprise)

[Austin Drukker]

I also have this error. When I try to set a 40,993 x 1 vector using st_matrix the result is a -24,543 x 1 vector! Has anyone found a solution?

[Sebastian Kripfganz]

While I can replicate it for Stata 14 SE, it does not seem to be a problem anymore in Stata 16 SE.

[Kayode Olaide]

Hello Sebastian,
I appreciate the great work you have done on ARDL in Stata. I am estimating ARDL using Stata 14, and I got the following error messages in some of the estimation:
(1) # of lag permutations (312500) exceeds setting of 'maxcombs' (100000), when I ran the model ardl NRDEC NGDPPC NGDPPCG NRIR NER NMSG NUR NIR, maxlag(4) aic
(2) Maximum number of iterations exceeded, when I ran the model ardl NEDEC NGDPPC NGDPPCG NRIR NER NMSG NUR NIR, ec1 lags(4)
The variables are the Nigerian fiscal decentralization variables and some key macroeconomic variables.
Please, how can I resolve these two issues.
Thanks.

[JanDitzen]

Interesting question. I think there are two things here, first how does Stata deal with its maximum matrix size and secondly differences across Stata.

On the first, my understanding was that when posting the mata matrix to Stata, Stata should issue an error if the matrix exceeds the maximum matrix size. I definitely have seen this error. When I add to your example above matrix list m Stata 14 aborts with a mata (??) error. Thus it seems the matrix does not really exist in Stata.

With Stata 16 the maximum matrix size was increased (see #22 https://blog.stata.com/2019/06/26/stata-16-released/ ). help limits shows the following:

                                                      Stata/IC         StataMP/SE
matrix  (3)
dimension of single matrix                800x800        65,534x65,534 (MP)
                                                                          11,000x11,000 (SE)

On my Stata 16 SE the following works though:

. mata: m=J(11000,11000,1)
. mata: st_matrix("m",m).
. di rowsof(m)
11000 
. mata: m=J(12000,12000,1).
. mata: st_matrix("m",m).
. di rowsof(m)
12000

despite c(max_matdim) = 11000? So, I am a bit confused as well now....